Re: [netcdfgroup] Extracting time series from netCDF file

Hi Jose,

Just thinking that there are some data visualisation software that also allow 
essentially netcdf data manipulation.

I like to use ferret http://ferret.wrc.noaa.gov/Ferret/ for this kind of stuff.
Netcdf is the "native" format for ferret - ferret will read netcdf data with a 
very simple command (use ha.nc) and interpret the dimensions correctly (judging 
from your cdl below). You can then just extract points by referring to their 
lat, lon: save/file="pointdat.nc" ha[x=x0,y=y0] - you can have that also in 
ascii...

You can also use R, though you have to jump through quite a lot of hops to read 
netcdf (as in matlab) - quite a big difference to ferret.

But you still have to install software.... ;-)
... though I think it is worthwhile (R and ferret that is)

Good luck,
Cheers,

Joerg

-----Original Message-----
From: netcdfgroup-bounces@xxxxxxxxxxxxxxxx 
[mailto:netcdfgroup-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Russ Rew
Sent: 19 November 2012 21:31
To: Jose Borrero
Cc: netcdfgroup@xxxxxxxxxxxxxxxx
Subject: Re: [netcdfgroup] Extracting time series from netCDF file

Hi Jose,

> Just curious, why can't it be done in the manner that I thought it 
> could be done?
> 
> Why do I have to go install a whole other software to do what should 
> be pretty simple?
> 
> I need to do work, not install software...
> 
> I might as well just do it my own made up wonky way that works...  
> 
> seem like every bag of worms just contains another bag of worms...
> 
> frustrating!

I sympathize, but the approach you were suggesting is not supported by the C 
ncdump, though there is a netCDF-Java ncdump that comes closer to what you want:

  http://www.unidata.ucar.edu/software/netcdf-java/reference/manPages.html

Changes to the C ncdump to support a query language on the command line are 
non-trivial.

For those familiar with Python, I think the solution Rich Signell provided is 
very elegant and worth studying, because it's far more powerful than what can 
be accomplished with relatively primitive command-line tools.

--Russ
 
> ----- Original Message -----
> From: "Signell, Richard" <rsignell@xxxxxxxx>
> Date: Monday, November 19, 2012 3:36 pm
> Subject: Re: [netcdfgroup] Extracting time series from netCDF file
> To: Jose Borrero <jborrero@xxxxxxx>
> Cc: netcdfgroup@xxxxxxxxxxxxxxxx
> 
> > Jose,
> > If you were used to using Matlab but no longer have it, you might 
> > try Python, which you can use in a similar fashion. This example 
> > shows how to extract and plot a time series of wave heights from a 
> > remote OPeNDAP dataset, but would work the same with a local NetCDF file:
> > http://nbviewer.ipython.org/4113653/
> > 
> > This code uses the NetCDF4-Python module, which reads NetCDF and 
> > OPeNDAP datasets using the same syntax.  It can be a pain to 
> > install, but I didn't have to build it because I have the Enthought 
> > Python Distribution, which includes NetCDF4-Python with OPeNDAP 
> > support.  (I think the full version of the EPD is free for folks 
> > with .edu addressses, and of course there are many other options for 
> > free scientific distributions as well).
> > 
> > Good luck,
> > -Rich
> > 
> > 
> > On Mon, Nov 19, 2012 at 12:59 PM, Jose Borrero <jborrero@xxxxxxx> 
> > wrote:>
> > >
> > > Hi All,
> > >
> > > I have a program that produces netCDF files of water levels over
> > a geographic region fro different times.
> > >
> > > the header looks like this:
> > >
> > > ===============================================
> > >         lon = 720 ;
> > >         lat = 330 ;
> > >         grid_lon = 720 ;
> > >         grid_lat = 330 ;
> > >         rows = 3 ;
> > >         clms = 3 ;
> > >         time = UNLIMITED ; // (226 currently)
> > > variables:
> > >         double lon(lon) ;
> > >                 lon:long_name = "longitude" ;
> > >                 lon:units = "degrees_east" ;
> > >                 lon:point_spacing = "even" ;
> > >         double lat(lat) ;
> > >                 lat:long_name = "latitude" ;
> > >                 lat:units = "degrees_north" ;
> > >                 lat:point_spacing = "uneven" ;
> > >         double grid_lon(grid_lon) ;
> > >                 grid_lon:long_name = "longitude" ;
> > >                 grid_lon:units = "degrees_east" ;
> > >                 grid_lon:point_spacing = "even" ;
> > >         double grid_lat(grid_lat) ;
> > >                 grid_lat:long_name = "latitude" ;
> > >                 grid_lat:units = "degrees_north" ;
> > >                 grid_lat:point_spacing = "uneven" ;
> > >         double O2Geo(clms, rows) ;
> > >                 O2Geo:long_name = "rotation matrix" ;
> > >         float max_amp(grid_lat, grid_lon) ;
> > >                 max_amp:long_name = "Maximum Wave Amplitude" ;
> > >                 max_amp:units = "centimeters" ;
> > >                 max_amp:_FillValue = -1.e+34f ;
> > >                 max_amp:missing_value = -1.e+34f ;
> > >         double time(time) ;
> > >                 time:long_name = "time" ;
> > >                 time:units = "seconds" ;
> > >                 time:calendar = "gregorian" ;
> > >         float ha(time, lat, lon) ;
> > >                 ha:long_name = "Wave Amplitude" ;
> > >                 ha:units = "centimeters" ;
> > >                 ha:missing_value = -1.e+34f ;
> > >                 ha:_FillValue = -1.e+34f ; 
> > > ===============================================
> > >
> > > i would like to extract a time series of data at a specific
> > location.>
> > > I have been looking at the ncdump command and thinking there
> > would be a way do it at the command line, i.e. something like:
> > >
> > > ncdump -v ha(lat,lon, time[1=>t]) datafile.nc > timeseries.cdl
> > >
> > > (I realize the syntax is totally wrong, i am just trying to
> > express how I would specify a lat/lon and ask for the data over the 
> > entire time)
> > >
> > > then using ncgen to make a new binary netCDF that can be read by
> > GMT's nc2xy and output a simple 2 column time series.
> > >
> > > but I don't  think I can specify things that way with ncdump.
> > >
> > > I then looked in to using the ncBrowse software.  I have managed
> > to use that to specify a time series of data at a point, and even 
> > write that data out to a CDL file.  However, the file generated by 
> > ncBrowse (pasted below in an edited form) cannot be read by nc2xy 
> > and has characters that are not compatible with ncgen
> > >
> > > nc2xy tells me: nc2xy: NetCDF: Unknown file format [ha.cdl]
> > >
> > > and ncgen tells me:
> > >
> > > ncgen: ha.cdl line 7: syntax error, unexpected '=', expecting ',' 
> > or ')'
> > >
> > > ===================================================
> > > netcdf ha.cdl {
> > > dimensions:
> > >      time = UNLIMITED ; // (226 currently)
> > >      lat = 1 ;
> > >      lon = 1 ;
> > > variables:
> > >    double time(time=226);
> > >      time:long_name = "time";
> > >      time:units = "seconds";
> > >      time:calendar = "gregorian";
> > >    double lat(lat=330);
> > >      lat:long_name = "latitude";
> > >      lat:units = "degrees_north";
> > >      lat:point_spacing = "uneven";
> > >    double lon(lon=720);
> > >      lon:long_name = "longitude";
> > >      lon:units = "degrees_east";
> > >      lon:point_spacing = "even";
> > >    float ha(time=226, lat=330, lon=720);
> > >      ha:long_name = "Wave Amplitude";
> > >      ha:units = "centimeters";
> > >      ha:missing_value = -1.0E34f; // float
> > >      ha:_FillValue = -1.0E34f; // float
> > > data:
> > >
> > >  time = 0.0, <ALL THE TIMES>, 14400.0 ;
> > >
> > >  lat = 24.00833 ;
> > >
> > >  lon = 60.00833 ;
> > >
> > >  ha = 1.0723767, <ALL THE ha VALUES>1.6242466 ;
> > >
> > > }
> > > ===================================================
> > >
> > > so then, I go in to the ha.cdl file and edit the header in a text
> > editor to remove the =226, =330 and =720 bits and change the line:
> > >
> > >    float ha(time=226, lat=330, lon=720);
> > >
> > > to
> > >
> > >    float ha(time);
> > >
> > > I then run run:
> > >
> > > ncgen -b -o ha.nc ha.cdl
> > >
> > > and make a file ha.nc which then can be accessed by nc2xy through
> > the command:
> > >
> > > nc2xy ha.nc -Ftime/ha > ha.txt
> > >
> > > to create the file ha.txt which looks like:
> > >
> > > ===========================
> > > 0.000   17.423
> > > 64.000  61.289
> > > 128.000 36.575
> > > <...SNIP...>
> > > 14336.000       -2.210
> > > 14400.000       -2.147
> > > ===========================
> > >
> > > and is exactly what i want! Two columns, time and ha at that
> > particular lat/lon So I know it is possible...
> > >
> > > So, if you are still with me, how can i do this more simply,
> > through the command line preferably, or even a small script that goes:
> > >
> > > ncdump (args)
> > > ncgen (args)
> > > nc2xy (args)
> > >
> > > would be fine, but I just need to know the right arguments to
> > make it all talk to each other.
> > >
> > > if you can help me,  MANY, MANY, THANKS!!!!
> > >
> > > -jose
> > >
> > > (p.s. i used to do this with MATLAB, but I recently had my
> > computer with MATLAB die, and i can't get it up and running again, 
> > so i am trying to do this an alternative way...)
> > >
> > >
> > > _______________________________________________
> > > netcdfgroup mailing list
> > > netcdfgroup@xxxxxxxxxxxxxxxx
> > > For list information or to unsubscribe,  visit: 
> > http://www.unidata.ucar.edu/mailing_lists/
> > 
> > 
> > -- 
> > Dr. Richard P. Signell   (508) 457-2229
> > USGS, 384 Woods Hole Rd.
> > Woods Hole, MA 02543-1598
> > 
> 
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit: 
> http://www.unidata.ucar.edu/m ailing_lists/

_______________________________________________
netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit: 
http://www.unidata.ucar.edu/mailing_lists/ 



  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: