Re: [netcdfgroup] storing sparse matrices data in NetCDF

  • To: Ken Mankoff <mankoff@xxxxxxxxx>
  • Subject: Re: [netcdfgroup] storing sparse matrices data in NetCDF
  • From: Chris Barker - NOAA Federal <chris.barker@xxxxxxxx>
  • Date: Mon, 25 Mar 2019 11:24:53 -0500
I’m a bit confused about exactly how this data is structured— you used the
word “sparse”, but I don’t think it’s sparse in the sense of “sparse
matrix” — that would be data on a rectangular grid, where most of the
values are zero. If that is the case, than storing it as a full array and
compressing it would be a good way to go. (Other than compression, I don’t
know if s standard say to store sparse arrays— maybe there should be one?

But, if rather than a regular grid with missing or zero values, what you
have is data at essentially arbitrary locations, then “discrete geometries”
may make sense. But another option is to consider the data as an
unstructured grid, and use the UGRID convention:

https://github.com/ugrid-conventions/ugrid-conventions/blob/master/README.md#ugrid-conventions

Note that you can use it with only points— it doesn’t need to specify the
mesh itself.

This would make sense if you have:

A time series of data at each point — that is the locations are constant in
time. Ideally, the data would be at the same times at all locations. That
is, structured in time, but not in space.

In that case, your locations would be the nodes, and you’d have a time
coordinate, and then the data itself would be in a (time, node_num) indexed
array.

HTH,

Chris

On Mar 25, 2019, at 8:17 AM, Ken Mankoff <mankoff@xxxxxxxxx> wrote:

Hi Robert!

I hope all is well at GISS.

On 2019-03-18 at 20:30 -0700, Schmunk, Robert B. (GISS-611.0)[SciSpace LLC]
<robert.b.schmunk@xxxxxxxx> wrote...

Pursuant to Dave’s suggestion, check out the CF convention for netCDF

metadata. In particular see section 9 on discrete sampling geometries,

and its discussion of the timeSeries featureType at multiple

locations.


Appendix H.2.1 shows example CDL notation for an case that sounds

fairly similar to what you are looking for.


Yes, I had found that and am modeling my data format after H2.1:

http://cfconventions.org/cf-conventions/cf-conventions.html#_orthogonal_multidimensional_array_representation_of_time_series

But I have a Panoply-specific question related to this data. When open the
file I'm creating in Panoply, runoff variable type is "2D" and dimension
are "cat" and "time", and the Panoply-displayed info is:

double runoff(time=365, cat=31240);
 :units = "m3 s-1";
 :long_name = "RACMO runoff";
 :standard_name = "water_volume_transport_in_river_channel";
 :description = "..."
 :_ChunkSizes = 1U, 31240U; // uint

I'd like to also be able to plot data on (lon,lat). If I add

 :coordinates = "lon lat"

then the data is "Geo2D", but I can only make a "longitude-time"
georeferenced plot (and again cat v. time 2D plot). I've been trying and
searching for what attributes I need to add to help your software display
the results on a (lon,lat) grid/map. Can you offer any suggestions?

Thanks,

-k.

_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web.  Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.


netcdfgroup mailing list
netcdfgroup@xxxxxxxxxxxxxxxx
For list information or to unsubscribe,  visit:
http://www.unidata.ucar.edu/mailing_lists/
  • 2019 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdfgroup archives: