Re: [netcdf-java] Adding Support for Simple Geometries CF Convention

  • To: cschroedl@xxxxxxxx
  • Subject: Re: [netcdf-java] Adding Support for Simple Geometries CF Convention
  • From: Sean Arms <sarms@xxxxxxxx>
  • Date: Tue, 24 Jul 2018 08:48:17 -0600
Greetings Carl!

Appolgies for the delay in response. I was out of the office all last week
with extremely limited internet connection.

First off, most certainly any effort towards adding Simple Geometry support
to netCDF-Java and the TDS would be welcome! We had planned on doing this
ourselves in the near future (actually, the September timeline is what we
had envisioned), but the developer most suited to do that work is no longer
at Unidata as of last month. Our development team is quite...pressed...at
the moment, and so outside efforts are even more so welcome!

In general, the first step would be to represent a simple geometry in the
Common Data Model.

https://www.unidata.ucar.edu/software/thredds/current/netcdf-java/CDM/

The machinery for doing that would start off at the IOSP layer. Given that
this is an addition to CF and presumably stored in a netCDF file, then
supporting the data access layer of the CDM would be very simple as it's
already done. The next step would be to add coordinate system layer
support. Again, given that this is going to be part of CF, the coordinate
system layer should be pretty straight forward, if not working as is. The
tricky part would be supporting things at the Scientific Feature Type
layer. It might be the case that the GRID feature type would take you a
long way, but it's very likely that a new, Simple Geometry Feature Type
would need to be created.

Once the new specification is supported in the CDM, the next step would be
to talk about how you envision interacting with simple geometries via the
TDS. I can think of many cool things to support, for example looking at the
intersection of a simple geometry and a GRID and performing an extraction
of some sort, but they might not be all that useful. There are also more
basic things, like subsetting data by requesting specific geometries by
index or id. These things would likely involve extending the netCDF Subset
Service API to allow for such queries.

For the past few years, we have been working to get the next major release
of netCDF-Java and the TDS out the door, and are getting very close. That
said, it would be best to target any work on the 5.0 branch of the THREDDS
repository, as opposed to the 4.6 branch (i.e. master).

Supporting the Simple Geometries addition to CF would almost certainly
involve adding dependencies. Some of the dependencies may already rely on
artifacts from netCDF-Java, but  might be pinned to older versions. In
addition to adding a circular dependency, the change in major version could
pose a stumbling block to using them in the 5.0 branch. Neither of those
are show stoppers, but are issues to be aware of. Also, many of the users
of netCDF-Java rely on the netCDFAll jar file, which has grown quite a bit
over the years. When adding new dependencies one would need to keep the
size of the netCDFAll jar file in mind. One way to work with this would be
to do the work in a simple geometry module. Projects that want to use, say,
the simple geometry IOSP would need to include that artifact in their maven
pom or download it in addition to netCDFAll.jar, although the "all" at this
point would be misleading...

I hope this gives you an idea of the work involved in support Simple
Geometries. I'd be happy to have a teleconference at some point if you
think that would be helpful. Once again, thank you for reaching out!

Cheers,

Sean




On Tue, Jul 17, 2018 at 7:59 AM Carl Schroedl <cschroedl@xxxxxxxx> wrote:

> Hi NetCDF Java Folks!
>
> Thanks for you hard work on this important open source project. I am Carl
> Schroedl, a software developer for the US Geological Survey. We will be
> working on a surface water model archiving platform this summer. Our
> surface water model archives will be leveraging a new CF Convention on
> Simple Geometries to represent things like rivers and watersheds. If you
> would like to dig into the details on the new CF Convention, check out the
> specification pull request
> <https://github.com/cf-convention/cf-conventions/pull/115>, the attached
> poster, or the Simple Geometry portion of this lecture
> <http://wiki.esipfed.org/index.php/Interoperability_and_Technology/Tech_Dive_Webinar_Series#10_May_2018:_.22NetCDF-CF_Advances_-_Simple_Geometries.2C_Swaths.2C_and_Groups.22:_Dave_Blodgett_.28USGS.29.2C_Tim_Whiteaker_.28UT_Austin.29.2C_Aleksander_Jelanek_.28HDF_Group.29_and_Daniel_Lee_.28EUMETSAT.29>
> .
>
> We would like to contribute support for this new convention to open source
> tools, possibly including THREDDS, and the CDM. I know from experience that
> projects are not always in a great position to accept contributions (large
> in-flight refactor branches, closely-related concurrent work, etc.). Do you
> foresee any problems with an influx of these types of contributions to
> between now and September? Do you have any advice on how our team could
> best contribute support for this convention to THREDDS, the CDM, or related
> projects?
>
> Thanks again for your work on these important open source efforts.
>
> Wishing you all the best,
>
> --
>
> *Carl Schroedl *| Computer Scientist | USGS | Water Mission Area | Integrated 
> Modeling and Prediction Division | Modeling Support and Coordination Branch | 
> 8505 Research Way | Middleton, WI 53562 | 608-821-3836 | 
> https://www.usgs.gov/science/mission-areas/water
>
> _______________________________________________
> NOTE: All exchanges posted to Unidata maintained email lists are
> recorded in the Unidata inquiry tracking system and made publicly
> available through the web.  Users who post to any of the lists we
> maintain are reminded to remove any personal information that they
> do not want to be made public.
>
>
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
  • 2018 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the netcdf-java archives: