Re: [netcdfgroup] Extracting time series from netCDF file

  • To: "Kaduk, Joerg (Dr.)" <jk61@xxxxxxxxxxxxxxx>
  • Subject: Re: [netcdfgroup] Extracting time series from netCDF file
  • From: "David W. Pierce" <dpierce@xxxxxxxx>
  • Date: Mon, 19 Nov 2012 15:57:43 -0800
Sorry, can't resist clearing up this misconception:

> You can also use R, though you have to jump through quite a lot of hops
to read netcdf (as in matlab) - quite a big difference to ferret.

To read data from a netcdf file in R, you can use the ncdf4 library and do
this:

ncid <- nc_open( 'filename.nc' )
data <- ncvar_get( ncid, 'varname' )

So not so many hoops after all. :)

--Dave



On Mon, Nov 19, 2012 at 3:45 PM, Kaduk, Joerg (Dr.) <jk61@xxxxxxxxxxxxxxx>wrote:

> Hi Jose,
>
> Just thinking that there are some data visualisation software that also
> allow essentially netcdf data manipulation.
>
> I like to use ferret http://ferret.wrc.noaa.gov/Ferret/ for this kind of
> stuff.
> Netcdf is the "native" format for ferret - ferret will read netcdf data
> with a very simple command (use ha.nc) and interpret the dimensions
> correctly (judging from your cdl below). You can then just extract points
> by referring to their lat, lon: save/file="pointdat.nc" ha[x=x0,y=y0] -
> you can have that also in ascii...
>
> You can also use R, though you have to jump through quite a lot of hops to
> read netcdf (as in matlab) - quite a big difference to ferret.
>
> But you still have to install software.... ;-)
> ... though I think it is worthwhile (R and ferret that is)
>
> Good luck,
> Cheers,
>
> Joerg
>
> -----Original Message-----
> From: netcdfgroup-bounces@xxxxxxxxxxxxxxxx [mailto:
> netcdfgroup-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Russ Rew
> Sent: 19 November 2012 21:31
> To: Jose Borrero
> Cc: netcdfgroup@xxxxxxxxxxxxxxxx
> Subject: Re: [netcdfgroup] Extracting time series from netCDF file
>
> Hi Jose,
>
> > Just curious, why can't it be done in the manner that I thought it
> > could be done?
> >
> > Why do I have to go install a whole other software to do what should
> > be pretty simple?
> >
> > I need to do work, not install software...
> >
> > I might as well just do it my own made up wonky way that works...
> >
> > seem like every bag of worms just contains another bag of worms...
> >
> > frustrating!
>
> I sympathize, but the approach you were suggesting is not supported by the
> C ncdump, though there is a netCDF-Java ncdump that comes closer to what
> you want:
>
>   http://www.unidata.ucar.edu/software/netcdf-java/reference/manPages.html
>
> Changes to the C ncdump to support a query language on the command line
> are non-trivial.
>
> For those familiar with Python, I think the solution Rich Signell provided
> is very elegant and worth studying, because it's far more powerful than
> what can be accomplished with relatively primitive command-line tools.
>
> --Russ
>
> > ----- Original Message -----
> > From: "Signell, Richard" <rsignell@xxxxxxxx>
> > Date: Monday, November 19, 2012 3:36 pm
> > Subject: Re: [netcdfgroup] Extracting time series from netCDF file
> > To: Jose Borrero <jborrero@xxxxxxx>
> > Cc: netcdfgroup@xxxxxxxxxxxxxxxx
> >
> > > Jose,
> > > If you were used to using Matlab but no longer have it, you might
> > > try Python, which you can use in a similar fashion. This example
> > > shows how to extract and plot a time series of wave heights from a
> > > remote OPeNDAP dataset, but would work the same with a local NetCDF
> file:
> > > http://nbviewer.ipython.org/4113653/
> > >
> > > This code uses the NetCDF4-Python module, which reads NetCDF and
> > > OPeNDAP datasets using the same syntax.  It can be a pain to
> > > install, but I didn't have to build it because I have the Enthought
> > > Python Distribution, which includes NetCDF4-Python with OPeNDAP
> > > support.  (I think the full version of the EPD is free for folks
> > > with .edu addressses, and of course there are many other options for
> > > free scientific distributions as well).
> > >
> > > Good luck,
> > > -Rich
> > >
> > >
> > > On Mon, Nov 19, 2012 at 12:59 PM, Jose Borrero <jborrero@xxxxxxx>
> > > wrote:>
> > > >
> > > > Hi All,
> > > >
> > > > I have a program that produces netCDF files of water levels over
> > > a geographic region fro different times.
> > > >
> > > > the header looks like this:
> > > >
> > > > ===============================================
> > > >         lon = 720 ;
> > > >         lat = 330 ;
> > > >         grid_lon = 720 ;
> > > >         grid_lat = 330 ;
> > > >         rows = 3 ;
> > > >         clms = 3 ;
> > > >         time = UNLIMITED ; // (226 currently)
> > > > variables:
> > > >         double lon(lon) ;
> > > >                 lon:long_name = "longitude" ;
> > > >                 lon:units = "degrees_east" ;
> > > >                 lon:point_spacing = "even" ;
> > > >         double lat(lat) ;
> > > >                 lat:long_name = "latitude" ;
> > > >                 lat:units = "degrees_north" ;
> > > >                 lat:point_spacing = "uneven" ;
> > > >         double grid_lon(grid_lon) ;
> > > >                 grid_lon:long_name = "longitude" ;
> > > >                 grid_lon:units = "degrees_east" ;
> > > >                 grid_lon:point_spacing = "even" ;
> > > >         double grid_lat(grid_lat) ;
> > > >                 grid_lat:long_name = "latitude" ;
> > > >                 grid_lat:units = "degrees_north" ;
> > > >                 grid_lat:point_spacing = "uneven" ;
> > > >         double O2Geo(clms, rows) ;
> > > >                 O2Geo:long_name = "rotation matrix" ;
> > > >         float max_amp(grid_lat, grid_lon) ;
> > > >                 max_amp:long_name = "Maximum Wave Amplitude" ;
> > > >                 max_amp:units = "centimeters" ;
> > > >                 max_amp:_FillValue = -1.e+34f ;
> > > >                 max_amp:missing_value = -1.e+34f ;
> > > >         double time(time) ;
> > > >                 time:long_name = "time" ;
> > > >                 time:units = "seconds" ;
> > > >                 time:calendar = "gregorian" ;
> > > >         float ha(time, lat, lon) ;
> > > >                 ha:long_name = "Wave Amplitude" ;
> > > >                 ha:units = "centimeters" ;
> > > >                 ha:missing_value = -1.e+34f ;
> > > >                 ha:_FillValue = -1.e+34f ;
> > > > ===============================================
> > > >
> > > > i would like to extract a time series of data at a specific
> > > location.>
> > > > I have been looking at the ncdump command and thinking there
> > > would be a way do it at the command line, i.e. something like:
> > > >
> > > > ncdump -v ha(lat,lon, time[1=>t]) datafile.nc > timeseries.cdl
> > > >
> > > > (I realize the syntax is totally wrong, i am just trying to
> > > express how I would specify a lat/lon and ask for the data over the
> > > entire time)
> > > >
> > > > then using ncgen to make a new binary netCDF that can be read by
> > > GMT's nc2xy and output a simple 2 column time series.
> > > >
> > > > but I don't  think I can specify things that way with ncdump.
> > > >
> > > > I then looked in to using the ncBrowse software.  I have managed
> > > to use that to specify a time series of data at a point, and even
> > > write that data out to a CDL file.  However, the file generated by
> > > ncBrowse (pasted below in an edited form) cannot be read by nc2xy
> > > and has characters that are not compatible with ncgen
> > > >
> > > > nc2xy tells me: nc2xy: NetCDF: Unknown file format [ha.cdl]
> > > >
> > > > and ncgen tells me:
> > > >
> > > > ncgen: ha.cdl line 7: syntax error, unexpected '=', expecting ','
> > > or ')'
> > > >
> > > > ===================================================
> > > > netcdf ha.cdl {
> > > > dimensions:
> > > >      time = UNLIMITED ; // (226 currently)
> > > >      lat = 1 ;
> > > >      lon = 1 ;
> > > > variables:
> > > >    double time(time=226);
> > > >      time:long_name = "time";
> > > >      time:units = "seconds";
> > > >      time:calendar = "gregorian";
> > > >    double lat(lat=330);
> > > >      lat:long_name = "latitude";
> > > >      lat:units = "degrees_north";
> > > >      lat:point_spacing = "uneven";
> > > >    double lon(lon=720);
> > > >      lon:long_name = "longitude";
> > > >      lon:units = "degrees_east";
> > > >      lon:point_spacing = "even";
> > > >    float ha(time=226, lat=330, lon=720);
> > > >      ha:long_name = "Wave Amplitude";
> > > >      ha:units = "centimeters";
> > > >      ha:missing_value = -1.0E34f; // float
> > > >      ha:_FillValue = -1.0E34f; // float
> > > > data:
> > > >
> > > >  time = 0.0, <ALL THE TIMES>, 14400.0 ;
> > > >
> > > >  lat = 24.00833 ;
> > > >
> > > >  lon = 60.00833 ;
> > > >
> > > >  ha = 1.0723767, <ALL THE ha VALUES>1.6242466 ;
> > > >
> > > > }
> > > > ===================================================
> > > >
> > > > so then, I go in to the ha.cdl file and edit the header in a text
> > > editor to remove the =226, =330 and =720 bits and change the line:
> > > >
> > > >    float ha(time=226, lat=330, lon=720);
> > > >
> > > > to
> > > >
> > > >    float ha(time);
> > > >
> > > > I then run run:
> > > >
> > > > ncgen -b -o ha.nc ha.cdl
> > > >
> > > > and make a file ha.nc which then can be accessed by nc2xy through
> > > the command:
> > > >
> > > > nc2xy ha.nc -Ftime/ha > ha.txt
> > > >
> > > > to create the file ha.txt which looks like:
> > > >
> > > > ===========================
> > > > 0.000   17.423
> > > > 64.000  61.289
> > > > 128.000 36.575
> > > > <...SNIP...>
> > > > 14336.000       -2.210
> > > > 14400.000       -2.147
> > > > ===========================
> > > >
> > > > and is exactly what i want! Two columns, time and ha at that
> > > particular lat/lon So I know it is possible...
> > > >
> > > > So, if you are still with me, how can i do this more simply,
> > > through the command line preferably, or even a small script that goes:
> > > >
> > > > ncdump (args)
> > > > ncgen (args)
> > > > nc2xy (args)
> > > >
> > > > would be fine, but I just need to know the right arguments to
> > > make it all talk to each other.
> > > >
> > > > if you can help me,  MANY, MANY, THANKS!!!!
> > > >
> > > > -jose
> > > >
> > > > (p.s. i used to do this with MATLAB, but I recently had my
> > > computer with MATLAB die, and i can't get it up and running again,
> > > so i am trying to do this an alternative way...)
> > > >
> > > >
> > > > _______________________________________________
> > > > netcdfgroup mailing list
> > > > netcdfgroup@xxxxxxxxxxxxxxxx
> > > > For list information or to unsubscribe,  visit:
> > > http://www.unidata.ucar.edu/mailing_lists/
> > >
> > >
> > > --
> > > Dr. Richard P. Signell   (508) 457-2229
> > > USGS, 384 Woods Hole Rd.
> > > Woods Hole, MA 02543-1598
> > >
> >
> > _______________________________________________
> > netcdfgroup mailing list
> > netcdfgroup@xxxxxxxxxxxxxxxx
> > For list information or to unsubscribe,  visit:
> > http://www.unidata.ucar.edu/m ailing_lists/
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe,  visit:
> http://www.unidata.ucar.edu/mailing_lists/
>



-- 
David W. Pierce
Division of Climate, Atmospheric Science, and Physical Oceanography
Scripps Institution of Oceanography, La Jolla, California, USA
(858) 534-8276 (voice)  /  (858) 534-8561 (fax)    dpierce@xxxxxxxx
  • 2012 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: