Re: [netcdfgroup] Extracting time series from netCDF file

  • To: Jose Borrero <jborrero@xxxxxxx>
  • Subject: Re: [netcdfgroup] Extracting time series from netCDF file
  • From: Dave Allured - NOAA Affiliate <dave.allured@xxxxxxxx>
  • Date: Mon, 19 Nov 2012 16:43:35 -0700
Jose,

There is a very good chance that you already have NCO (which Steve
mentioned) installed along with NetCDF.  Test by typing "ncks -r" on
the Unix line.  If so, you can easily extract single time series in
your requested 2-column text format, all on the command line, with two
ncks commands followed by a single paste command.  There are a few
details, nothing serious.

If you don't have NCO, there is another question lurking in the
background:  What *other* software might you already have that has
NetCDF capability?

--Dave

On Mon, Nov 19, 2012 at 3:34 PM, Steve Hankin <steven.c.hankin@xxxxxxxx> wrote:
>
> On 11/19/2012 1:10 PM, Jose Borrero wrote:
>>
>>
>> Just curious, why can't it be done in the manner that I thought it could
>> be done?
>>
>> Why do I have to go install a whole other software to do what should be
>> pretty simple?
>
>
> Hi Jose,
>
> Switching outlooks from a half-empty glass to a half-full glass turns your
> lament of "have to go install a whole other software to do what should be
> pretty simple" into "have numerous free and well-supported software packages
> to do both simple and complex tasks":
> http://www.unidata.ucar.edu/software/netcdf/software.html
>
> ncdump is a rudimentary (but very useful) tool, used mostly for diagnostic
> purposes.
> NCO together with standard Unix utilities would probably achieve what you
> are after here.
> Ferret, I know, can produce the output you are after -- not an
> advertisement, just mentioning an option I know well.
> The list of options to choose from is long, but this community can guide you
> to a quick (free) solution.
>
>     - Steve
>
>
>>
>> I need to do work, not install software...
>>
>> I might as well just do it my own made up wonky way that works...
>>
>> seem like every bag of worms just contains another bag of worms...
>>
>> frustrating!
>>
>> ----- Original Message -----
>> From: "Signell, Richard" <rsignell@xxxxxxxx>
>> Date: Monday, November 19, 2012 3:36 pm
>> Subject: Re: [netcdfgroup] Extracting time series from netCDF file
>> To: Jose Borrero <jborrero@xxxxxxx>
>> Cc: netcdfgroup@xxxxxxxxxxxxxxxx
>>
>>> Jose,
>>> If you were used to using Matlab but no longer have it, you might try
>>> Python, which you can use in a similar fashion. This example shows how
>>> to extract and plot a time series of wave heights from a remote
>>> OPeNDAP dataset, but would work the same with a local NetCDF file:
>>> http://nbviewer.ipython.org/4113653/
>>>
>>> This code uses the NetCDF4-Python module, which reads NetCDF and
>>> OPeNDAP datasets using the same syntax.  It can be a pain to install,
>>> but I didn't have to build it because I have the Enthought Python
>>> Distribution, which includes NetCDF4-Python with OPeNDAP support.  (I
>>> think the full version of the EPD is free for folks with .edu
>>> addressses, and of course there are many other options for free
>>> scientific distributions as well).
>>>
>>> Good luck,
>>> -Rich
>>>
>>>
>>> On Mon, Nov 19, 2012 at 12:59 PM, Jose Borrero <jborrero@xxxxxxx>
>>> wrote:>
>>>>
>>>> Hi All,
>>>>
>>>> I have a program that produces netCDF files of water levels over
>>>
>>> a geographic region fro different times.
>>>>
>>>> the header looks like this:
>>>>
>>>> ===============================================
>>>>          lon = 720 ;
>>>>          lat = 330 ;
>>>>          grid_lon = 720 ;
>>>>          grid_lat = 330 ;
>>>>          rows = 3 ;
>>>>          clms = 3 ;
>>>>          time = UNLIMITED ; // (226 currently)
>>>> variables:
>>>>          double lon(lon) ;
>>>>                  lon:long_name = "longitude" ;
>>>>                  lon:units = "degrees_east" ;
>>>>                  lon:point_spacing = "even" ;
>>>>          double lat(lat) ;
>>>>                  lat:long_name = "latitude" ;
>>>>                  lat:units = "degrees_north" ;
>>>>                  lat:point_spacing = "uneven" ;
>>>>          double grid_lon(grid_lon) ;
>>>>                  grid_lon:long_name = "longitude" ;
>>>>                  grid_lon:units = "degrees_east" ;
>>>>                  grid_lon:point_spacing = "even" ;
>>>>          double grid_lat(grid_lat) ;
>>>>                  grid_lat:long_name = "latitude" ;
>>>>                  grid_lat:units = "degrees_north" ;
>>>>                  grid_lat:point_spacing = "uneven" ;
>>>>          double O2Geo(clms, rows) ;
>>>>                  O2Geo:long_name = "rotation matrix" ;
>>>>          float max_amp(grid_lat, grid_lon) ;
>>>>                  max_amp:long_name = "Maximum Wave Amplitude" ;
>>>>                  max_amp:units = "centimeters" ;
>>>>                  max_amp:_FillValue = -1.e+34f ;
>>>>                  max_amp:missing_value = -1.e+34f ;
>>>>          double time(time) ;
>>>>                  time:long_name = "time" ;
>>>>                  time:units = "seconds" ;
>>>>                  time:calendar = "gregorian" ;
>>>>          float ha(time, lat, lon) ;
>>>>                  ha:long_name = "Wave Amplitude" ;
>>>>                  ha:units = "centimeters" ;
>>>>                  ha:missing_value = -1.e+34f ;
>>>>                  ha:_FillValue = -1.e+34f ;
>>>> ===============================================
>>>>
>>>> i would like to extract a time series of data at a specific
>>>
>>> location.>
>>>>
>>>> I have been looking at the ncdump command and thinking there
>>>
>>> would be a way do it at the command line, i.e. something like:
>>>>
>>>> ncdump -v ha(lat,lon, time[1=>t]) datafile.nc > timeseries.cdl
>>>>
>>>> (I realize the syntax is totally wrong, i am just trying to
>>>
>>> express how I would specify a lat/lon and ask for the data over the
>>> entire time)
>>>>
>>>> then using ncgen to make a new binary netCDF that can be read by
>>>
>>> GMT's nc2xy and output a simple 2 column time series.
>>>>
>>>> but I don't  think I can specify things that way with ncdump.
>>>>
>>>> I then looked in to using the ncBrowse software.  I have managed
>>>
>>> to use that to specify a time series of data at a point, and even
>>> write that data out to a CDL file.  However, the file generated by
>>> ncBrowse (pasted below in an edited form) cannot be read by nc2xy
>>> and has characters that are not compatible with ncgen
>>>>
>>>> nc2xy tells me: nc2xy: NetCDF: Unknown file format [ha.cdl]
>>>>
>>>> and ncgen tells me:
>>>>
>>>> ncgen: ha.cdl line 7: syntax error, unexpected '=', expecting ','
>>>
>>> or ')'
>>>>
>>>> ===================================================
>>>> netcdf ha.cdl {
>>>> dimensions:
>>>>       time = UNLIMITED ; // (226 currently)
>>>>       lat = 1 ;
>>>>       lon = 1 ;
>>>> variables:
>>>>     double time(time=226);
>>>>       time:long_name = "time";
>>>>       time:units = "seconds";
>>>>       time:calendar = "gregorian";
>>>>     double lat(lat=330);
>>>>       lat:long_name = "latitude";
>>>>       lat:units = "degrees_north";
>>>>       lat:point_spacing = "uneven";
>>>>     double lon(lon=720);
>>>>       lon:long_name = "longitude";
>>>>       lon:units = "degrees_east";
>>>>       lon:point_spacing = "even";
>>>>     float ha(time=226, lat=330, lon=720);
>>>>       ha:long_name = "Wave Amplitude";
>>>>       ha:units = "centimeters";
>>>>       ha:missing_value = -1.0E34f; // float
>>>>       ha:_FillValue = -1.0E34f; // float
>>>> data:
>>>>
>>>>   time = 0.0, <ALL THE TIMES>, 14400.0 ;
>>>>
>>>>   lat = 24.00833 ;
>>>>
>>>>   lon = 60.00833 ;
>>>>
>>>>   ha = 1.0723767, <ALL THE ha VALUES>1.6242466 ;
>>>>
>>>> }
>>>> ===================================================
>>>>
>>>> so then, I go in to the ha.cdl file and edit the header in a text
>>>
>>> editor to remove the =226, =330 and =720 bits and change the line:
>>>>
>>>>     float ha(time=226, lat=330, lon=720);
>>>>
>>>> to
>>>>
>>>>     float ha(time);
>>>>
>>>> I then run run:
>>>>
>>>> ncgen -b -o ha.nc ha.cdl
>>>>
>>>> and make a file ha.nc which then can be accessed by nc2xy through
>>>
>>> the command:
>>>>
>>>> nc2xy ha.nc -Ftime/ha > ha.txt
>>>>
>>>> to create the file ha.txt which looks like:
>>>>
>>>> ===========================
>>>> 0.000   17.423
>>>> 64.000  61.289
>>>> 128.000 36.575
>>>> <...SNIP...>
>>>> 14336.000       -2.210
>>>> 14400.000       -2.147
>>>> ===========================
>>>>
>>>> and is exactly what i want! Two columns, time and ha at that
>>>
>>> particular lat/lon So I know it is possible...
>>>>
>>>> So, if you are still with me, how can i do this more simply,
>>>
>>> through the command line preferably, or even a small script that goes:
>>>>
>>>> ncdump (args)
>>>> ncgen (args)
>>>> nc2xy (args)
>>>>
>>>> would be fine, but I just need to know the right arguments to
>>>
>>> make it all talk to each other.
>>>>
>>>> if you can help me,  MANY, MANY, THANKS!!!!
>>>>
>>>> -jose
>>>>
>>>> (p.s. i used to do this with MATLAB, but I recently had my
>>>
>>> computer with MATLAB die, and i can't get it up and running again,
>>> so i am trying to do this an alternative way...)
>>>>
>>>>
>>>> _______________________________________________
>>>> netcdfgroup mailing list
>>>> netcdfgroup@xxxxxxxxxxxxxxxx
>>>> For list information or to unsubscribe,  visit:
>>>
>>> http://www.unidata.ucar.edu/mailing_lists/
>>>
>>>
>>> --
>>> Dr. Richard P. Signell   (508) 457-2229
>>> USGS, 384 Woods Hole Rd.
>>> Woods Hole, MA 02543-1598
>>>
>> _______________________________________________
>> netcdfgroup mailing list
>> netcdfgroup@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/



  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: