Hi Ken:
One thing that wasn't clear in your description of your dataset is fi the
lat-lons change over time, or are constant for each time series, but not on a
regular grid.
Thanks,
-Roy
> On Mar 18, 2019, at 12:28 PM, Sourish Basu <Sourish.Basu@xxxxxxxxxxxx> wrote:
>
> Ken,
>
> I belong to a community that likes netcdf files :-) The data you
> describe is perfectly suited for packaging into netcdf. Not knowing the
> specifics of your data, I would say you could have one file per year,
> with variables 'x', 'y', 'lat', 'lon', etc., all of the same length
> (within one file/year, that is). In addition, if you make the common
> dimension of those arrays unlimited, that would allow easy operations by
> command line tools such as netcdf operators.
>
> In general, I find the set of people who like netcdf and the set of
> people who like text files to be completely disjoint. So if you know
> your target is the first set, I'd steer clear of CSV or other text-based
> formats :-)
>
> If you could tell me something more specific about your data, or perhaps
> pass me a sample dataset, I could help you with the packaging, if you'd
> like.
>
> Cheers,
>
> Sourish
>
> On 3/18/19 12:58 PM, Ken Mankoff wrote:
>> Hello,
>>
>> I'm trying to release a data set to a community that likes NetCDF. I'm
>> looking for info about if this might be an OK file format given the data
>> structure, and if so, how to go about generating the files.
>>
>> I have a 36-year daily time-series (~14,000 times). Presumably I'll break
>> this into yearly files to make working with them easier. At each time, I
>> have ~20,000 records. Fixed in time is (x,y,lon,lat,elevation), and then a
>> varying value. The lon,lat (and associated x,y) are spares points, not a
>> grid.
>>
>> Is this a data structure that can be reasonably stored in NetCDF, or should
>> I stick to a CSV file, one per year, with 365 columns and 20,000+6(header)
>> rows?
>>
>> Thanks,
>>
>> -k.
>>
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web. Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> netcdfgroup mailing list
>> netcdfgroup@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe, visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web. Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
**********************
"The contents of this message do not reflect any position of the U.S.
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
***Note new street address***
110 McAllister Way
Santa Cruz, CA 95060
Phone: (831)-420-3666
Fax: (831) 420-3980
e-mail: Roy.Mendelssohn@xxxxxxxx www: http://www.pfeg.noaa.gov/
"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
