IDV with RAMADDA

Status Report: September 2012 - April 2013

Yuan Ho, Julien Chastang, Sean Arms

This report updates the status of Unidata's Integrated Data Viewer (IDV) development efforts since the last report (September, 2012). We also highlight RAMADDA achievements that are relevant to the IDV.

Strategic Focus Areas

The IDV group's work supports the following Unidata funding proposal focus areas:

  1. Enable widespread, efficient access to geoscience data
    The IDV is a state of the art geoscience visualization application. It gives users the ability to view and analyze a rich set of geoscience data, including real time data, in a seamless and integrated fashion. This analysis is captured in IDV bundles. RAMADDA is a content management system and service specifically tailored towards the sharing and distribution of IDV bundles facilitating distribution of scientific data and analysis.
  2. Develop and provide open-source tools for effective use of geoscience data
    The IDV has been an open-source project for several years. The IDV is available on the github version control platform for greater open-source collaboration. The IDV provides users the unparalleled ability to analyze, integrate, and visualize heterogeneous geoscience data in two, three, and four dimensions. The IDV coupled with RAMADDA enables geoscience specialists the capability to share and collaborate their IDV analysis via social scientific networks.
  3. Provide cyberinfrastructure leadership in data discovery, access, and use
    RAMADDA allows geoscience specialists the ability to search and publish their IDV bundles on-line. Unidata's RAMADDA installation enables the IDV team to communicate more effectively to our users concerning their IDV issues. Specifically, during support ticket conversations, the IDV team requests that users upload pertinent data to RAMADDA for analysis. The IDV team also takes advantage of RAMADDA to share instructional IDV screencasts with users.
  4. Build, support, and advocate for the diverse geoscience community
    Unidata offers yearly multi-day training and occasionally regional workshops for IDV and RAMADDA. The IDV coupled with RAMADDA enables our earth science community partners to distribute geoscience data and metadata through web-based technologies thereby fostering scientific collaborations. Moreover, the IDV's ability to share bundles through RAMADDA creates a scientific social and collaborative network for the geoscience community.

Activities Since the Last Status Report

System Changes

Latest netCDF-Java Version: Collaborated with the THREDDS group to incorporate a new version of the netCDF-Java library (4.3.16) in the IDV. This version of netCDF-Java includes major changes to the way GRIB 1 and 2 files are handled. See the netCDF-Java GRIB documentation for more details on these changes.


New Minor Version of Java: For improved security and performance, a new minor version of Java is now distributed with the IDV.

Data Changes

Nonstandard calendars: An additional change in the netCDF-Java 4.3 library is support for non-standard calendars (e.g. 365 or 360 day years used in climate datasets). The IDV has been modified to support these calendars and display dates correctly from those datasets.


Unidata THREDDS Data Server Changes: Due to the changes in the way netCDF-Java handles GRIB files, the THREDDS Data Server (TDS) has also changed the way data are served. While we feel the IDV is able to interface with the new server transparently, we have provided a way for academic users of the IDV to finish out the spring semester without the fear of bundles breaking. The 4.0 release of the IDV allows users to keep accessing the old TDS server (motherlode.ucar.edu) via a plugin until June 1st, at which point IDV users should upgrade to at least IDV 4.0u1. The latest release (currently 4.0u1) automatically requests data from the new TDS (thredds.ucar.edu) when loading bundles and handles all of the url path and variable name changes under the hood.

Display Improvements

Grid Trajectory Display: The IDV now can plot 2D and 3D forward and backward gridded trajectory. The forward and backward 2D and 3D grid trajectory are derived data choices based on the horizontal velocity components and vertical velocity component. The trajectory is colored by a user selected parameter. There are three options in the Grid Trajectory Control to initialize the trajectory initial area.


Contour Labelling Frequency: Collaborated with the VisAD group to add a user interface to change the frequency of the contour labels in the IDV. The density of the labels can now be increased or decreased along the contours or horizontally across the display.


WMS Server: USGS Shaded Relief has been reinstated as "CONUS Shaded Relief" in the list of map background images.


Webcams: Fixed broken webcam links available in the IDV.

IDV and RAMADDA Training and Conference Attendance

  • IDV with RAMADDA regional workshop at San Jose State University on March 14, 15, 2013.
  • IDV with RAMADDA oral presentation at the EGU 2013 conference in Vienna, Austria.
  • IDV with RAMADDA demo at visualization splinter session at the EGU 2013 conference in Vienna, Austria.
  • Showcased the IDV with RAMADDA at the Unidata booth at the AMS 2013 Conference in Austin, Texas.

RAMADDA

The IDV team created a new major one day section on RAMADDA for the IDV with RAMADDA training workshop. This section was presented at the last IDV training workshop (October 2012).


The RAMADDA group fixed some problems with "latest" opendap link handling, and addressed password security issues.

Planned Activities

Ongoing Activities

Ongoing performance work in the IDV and VisAD will be folded into the version control trunk of the IDV project.


The IDV team continues to provide the geoscience community high-quality IDV support and guidance through the e-support question and answer forum.


We plan to continue to work with our steering committee to set IDV priorities.


New Activities

Preparing for WRF IDV workshop.


Preparing for Unidata summer workshop.


Following up on IDV short course application for AMS 2014.

Relevant Metrics

The number of both casual and regular IDV users is increasing. For example, in August 2011, there were 459 IDV users starting the IDV more than 5 times per month compared with 528 users for the same period in 2012. In August 2011, there were 72 IDV users starting the IDV more than 30 times per month compared with 89 users for the same period in 2012. Notably, there are large numbers of new IDV users are from China. Readers can find the raw metrics at http://www.unidata.ucar.edu/software/idv/logging/left.html.


In the area of greater collaborative development, since the migration of the IDV project to github, we have accepted 29 "pull requests" or code contributions from internal and external collaborators.