Re: [netcdfgroup] Extracting time series from netCDF file

  • To: Dave Allured - NOAA Affiliate <dave.allured@xxxxxxxx>
  • Subject: Re: [netcdfgroup] Extracting time series from netCDF file
  • From: Jose Borrero <jborrero@xxxxxxx>
  • Date: Mon, 19 Nov 2012 19:05:58 -0500
  • Authentication-results: msg-ironport1.usc.edu; dkim=neutral (message not signed) header.i=none
Dave and others,

NCO looks like the winner here...

just installed it via fink and am looking in to how it works now.

thanks and I will let you know.

-jose

----- Original Message -----
From: Dave Allured - NOAA Affiliate <dave.allured@xxxxxxxx>
Date: Monday, November 19, 2012 6:43 pm
Subject: Re: [netcdfgroup] Extracting time series from netCDF file
To: Jose Borrero <jborrero@xxxxxxx>
Cc: netcdfgroup@xxxxxxxxxxxxxxxx

> Jose,
> 
> There is a very good chance that you already have NCO (which Steve
> mentioned) installed along with NetCDF.  Test by typing "ncks -r" on
> the Unix line.  If so, you can easily extract single time series in
> your requested 2-column text format, all on the command line, with two
> ncks commands followed by a single paste command.  There are a few
> details, nothing serious.
> 
> If you don't have NCO, there is another question lurking in the
> background:  What *other* software might you already have that has
> NetCDF capability?
> 
> --Dave
> 
> On Mon, Nov 19, 2012 at 3:34 PM, Steve Hankin 
> <steven.c.hankin@xxxxxxxx> wrote:
> >
> > On 11/19/2012 1:10 PM, Jose Borrero wrote:
> >>
> >>
> >> Just curious, why can't it be done in the manner that I thought 
> it could
> >> be done?
> >>
> >> Why do I have to go install a whole other software to do what 
> should be
> >> pretty simple?
> >
> >
> > Hi Jose,
> >
> > Switching outlooks from a half-empty glass to a half-full glass 
> turns your
> > lament of "have to go install a whole other software to do what 
> should be
> > pretty simple" into "have numerous free and well-supported 
> software packages
> > to do both simple and complex tasks":
> > http://www.unidata.ucar.edu/software/netcdf/software.html
> >
> > ncdump is a rudimentary (but very useful) tool, used mostly for 
> diagnostic> purposes.
> > NCO together with standard Unix utilities would probably achieve 
> what you
> > are after here.
> > Ferret, I know, can produce the output you are after -- not an
> > advertisement, just mentioning an option I know well.
> > The list of options to choose from is long, but this community 
> can guide you
> > to a quick (free) solution.
> >
> >     - Steve
> >
> >
> >>
> >> I need to do work, not install software...
> >>
> >> I might as well just do it my own made up wonky way that works...
> >>
> >> seem like every bag of worms just contains another bag of worms...
> >>
> >> frustrating!
> >>
> >> ----- Original Message -----
> >> From: "Signell, Richard" <rsignell@xxxxxxxx>
> >> Date: Monday, November 19, 2012 3:36 pm
> >> Subject: Re: [netcdfgroup] Extracting time series from netCDF file
> >> To: Jose Borrero <jborrero@xxxxxxx>
> >> Cc: netcdfgroup@xxxxxxxxxxxxxxxx
> >>
> >>> Jose,
> >>> If you were used to using Matlab but no longer have it, you 
> might try
> >>> Python, which you can use in a similar fashion. This example 
> shows how
> >>> to extract and plot a time series of wave heights from a remote
> >>> OPeNDAP dataset, but would work the same with a local NetCDF file:
> >>> http://nbviewer.ipython.org/4113653/
> >>>
> >>> This code uses the NetCDF4-Python module, which reads NetCDF and
> >>> OPeNDAP datasets using the same syntax.  It can be a pain to 
> install,>>> but I didn't have to build it because I have the 
> Enthought Python
> >>> Distribution, which includes NetCDF4-Python with OPeNDAP 
> support.  (I
> >>> think the full version of the EPD is free for folks with .edu
> >>> addressses, and of course there are many other options for free
> >>> scientific distributions as well).
> >>>
> >>> Good luck,
> >>> -Rich
> >>>
> >>>
> >>> On Mon, Nov 19, 2012 at 12:59 PM, Jose Borrero <jborrero@xxxxxxx>
> >>> wrote:>
> >>>>
> >>>> Hi All,
> >>>>
> >>>> I have a program that produces netCDF files of water levels over
> >>>
> >>> a geographic region fro different times.
> >>>>
> >>>> the header looks like this:
> >>>>
> >>>> ===============================================
> >>>>          lon = 720 ;
> >>>>          lat = 330 ;
> >>>>          grid_lon = 720 ;
> >>>>          grid_lat = 330 ;
> >>>>          rows = 3 ;
> >>>>          clms = 3 ;
> >>>>          time = UNLIMITED ; // (226 currently)
> >>>> variables:
> >>>>          double lon(lon) ;
> >>>>                  lon:long_name = "longitude" ;
> >>>>                  lon:units = "degrees_east" ;
> >>>>                  lon:point_spacing = "even" ;
> >>>>          double lat(lat) ;
> >>>>                  lat:long_name = "latitude" ;
> >>>>                  lat:units = "degrees_north" ;
> >>>>                  lat:point_spacing = "uneven" ;
> >>>>          double grid_lon(grid_lon) ;
> >>>>                  grid_lon:long_name = "longitude" ;
> >>>>                  grid_lon:units = "degrees_east" ;
> >>>>                  grid_lon:point_spacing = "even" ;
> >>>>          double grid_lat(grid_lat) ;
> >>>>                  grid_lat:long_name = "latitude" ;
> >>>>                  grid_lat:units = "degrees_north" ;
> >>>>                  grid_lat:point_spacing = "uneven" ;
> >>>>          double O2Geo(clms, rows) ;
> >>>>                  O2Geo:long_name = "rotation matrix" ;
> >>>>          float max_amp(grid_lat, grid_lon) ;
> >>>>                  max_amp:long_name = "Maximum Wave Amplitude" ;
> >>>>                  max_amp:units = "centimeters" ;
> >>>>                  max_amp:_FillValue = -1.e+34f ;
> >>>>                  max_amp:missing_value = -1.e+34f ;
> >>>>          double time(time) ;
> >>>>                  time:long_name = "time" ;
> >>>>                  time:units = "seconds" ;
> >>>>                  time:calendar = "gregorian" ;
> >>>>          float ha(time, lat, lon) ;
> >>>>                  ha:long_name = "Wave Amplitude" ;
> >>>>                  ha:units = "centimeters" ;
> >>>>                  ha:missing_value = -1.e+34f ;
> >>>>                  ha:_FillValue = -1.e+34f ;
> >>>> ===============================================
> >>>>
> >>>> i would like to extract a time series of data at a specific
> >>>
> >>> location.>
> >>>>
> >>>> I have been looking at the ncdump command and thinking there
> >>>
> >>> would be a way do it at the command line, i.e. something like:
> >>>>
> >>>> ncdump -v ha(lat,lon, time[1=>t]) datafile.nc > timeseries.cdl
> >>>>
> >>>> (I realize the syntax is totally wrong, i am just trying to
> >>>
> >>> express how I would specify a lat/lon and ask for the data over 
> the>>> entire time)
> >>>>
> >>>> then using ncgen to make a new binary netCDF that can be read by
> >>>
> >>> GMT's nc2xy and output a simple 2 column time series.
> >>>>
> >>>> but I don't  think I can specify things that way with ncdump.
> >>>>
> >>>> I then looked in to using the ncBrowse software.  I have managed
> >>>
> >>> to use that to specify a time series of data at a point, and even
> >>> write that data out to a CDL file.  However, the file generated by
> >>> ncBrowse (pasted below in an edited form) cannot be read by nc2xy
> >>> and has characters that are not compatible with ncgen
> >>>>
> >>>> nc2xy tells me: nc2xy: NetCDF: Unknown file format [ha.cdl]
> >>>>
> >>>> and ncgen tells me:
> >>>>
> >>>> ncgen: ha.cdl line 7: syntax error, unexpected '=', expecting ','
> >>>
> >>> or ')'
> >>>>
> >>>> ===================================================
> >>>> netcdf ha.cdl {
> >>>> dimensions:
> >>>>       time = UNLIMITED ; // (226 currently)
> >>>>       lat = 1 ;
> >>>>       lon = 1 ;
> >>>> variables:
> >>>>     double time(time=226);
> >>>>       time:long_name = "time";
> >>>>       time:units = "seconds";
> >>>>       time:calendar = "gregorian";
> >>>>     double lat(lat=330);
> >>>>       lat:long_name = "latitude";
> >>>>       lat:units = "degrees_north";
> >>>>       lat:point_spacing = "uneven";
> >>>>     double lon(lon=720);
> >>>>       lon:long_name = "longitude";
> >>>>       lon:units = "degrees_east";
> >>>>       lon:point_spacing = "even";
> >>>>     float ha(time=226, lat=330, lon=720);
> >>>>       ha:long_name = "Wave Amplitude";
> >>>>       ha:units = "centimeters";
> >>>>       ha:missing_value = -1.0E34f; // float
> >>>>       ha:_FillValue = -1.0E34f; // float
> >>>> data:
> >>>>
> >>>>   time = 0.0, <ALL THE TIMES>, 14400.0 ;
> >>>>
> >>>>   lat = 24.00833 ;
> >>>>
> >>>>   lon = 60.00833 ;
> >>>>
> >>>>   ha = 1.0723767, <ALL THE ha VALUES>1.6242466 ;
> >>>>
> >>>> }
> >>>> ===================================================
> >>>>
> >>>> so then, I go in to the ha.cdl file and edit the header in a text
> >>>
> >>> editor to remove the =226, =330 and =720 bits and change the line:
> >>>>
> >>>>     float ha(time=226, lat=330, lon=720);
> >>>>
> >>>> to
> >>>>
> >>>>     float ha(time);
> >>>>
> >>>> I then run run:
> >>>>
> >>>> ncgen -b -o ha.nc ha.cdl
> >>>>
> >>>> and make a file ha.nc which then can be accessed by nc2xy through
> >>>
> >>> the command:
> >>>>
> >>>> nc2xy ha.nc -Ftime/ha > ha.txt
> >>>>
> >>>> to create the file ha.txt which looks like:
> >>>>
> >>>> ===========================
> >>>> 0.000   17.423
> >>>> 64.000  61.289
> >>>> 128.000 36.575
> >>>> <...SNIP...>
> >>>> 14336.000       -2.210
> >>>> 14400.000       -2.147
> >>>> ===========================
> >>>>
> >>>> and is exactly what i want! Two columns, time and ha at that
> >>>
> >>> particular lat/lon So I know it is possible...
> >>>>
> >>>> So, if you are still with me, how can i do this more simply,
> >>>
> >>> through the command line preferably, or even a small script 
> that goes:
> >>>>
> >>>> ncdump (args)
> >>>> ncgen (args)
> >>>> nc2xy (args)
> >>>>
> >>>> would be fine, but I just need to know the right arguments to
> >>>
> >>> make it all talk to each other.
> >>>>
> >>>> if you can help me,  MANY, MANY, THANKS!!!!
> >>>>
> >>>> -jose
> >>>>
> >>>> (p.s. i used to do this with MATLAB, but I recently had my
> >>>
> >>> computer with MATLAB die, and i can't get it up and running again,
> >>> so i am trying to do this an alternative way...)
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> netcdfgroup mailing list
> >>>> netcdfgroup@xxxxxxxxxxxxxxxx
> >>>> For list information or to unsubscribe,  visit:
> >>>
> >>> http://www.unidata.ucar.edu/mailing_lists/
> >>>
> >>>
> >>> --
> >>> Dr. Richard P. Signell   (508) 457-2229
> >>> USGS, 384 Woods Hole Rd.
> >>> Woods Hole, MA 02543-1598
> >>>
> >> _______________________________________________
> >> netcdfgroup mailing list
> >> netcdfgroup@xxxxxxxxxxxxxxxx
> >> For list information or to unsubscribe,  visit:
> >> http://www.unidata.ucar.edu/mailing_lists/
> >
> >
> > _______________________________________________
> > netcdfgroup mailing list
> > netcdfgroup@xxxxxxxxxxxxxxxx
> > For list information or to unsubscribe,  visit:
> > http://www.unidata.ucar.edu/mailing_lists/
> 



  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: