IDV with RAMADDA

Status Report: September 2013 - April 2014

Yuan Ho, Julien Chastang, Sean Arms

This report updates the status of Unidata's Integrated Data Viewer (IDV) development efforts since the last report (September, 2013). In the last six months, the IDV project reached a number of important milestones including the Java 7 / Java3D 1.6 transition, an early prototype of GEMPAK upper air format viewing capability, and a well attended Atlanta 2013 AMS IDV workshop. Moreover, there were several important display changes aimed at improving the user experience and system efficiency.

Strategic Focus Areas

The IDV group's work supports the following Unidata funding proposal focus areas:

  1. Enable widespread, efficient access to geoscience data
    The IDV is a state of the art geoscience visualization application. It gives users the ability to view and analyze a rich set of geoscience data, including real time data, in a seamless and integrated fashion. This analysis is captured in IDV bundles. RAMADDA is a content management system and service specifically tailored towards the sharing and distribution of IDV bundles facilitating distribution of scientific data and analysis.
  2. Develop and provide open-source tools for effective use of geoscience data
    The IDV has been an open-source project for several years. The IDV is available on the github version control platform for greater open-source collaboration. The IDV provides users the unparalleled ability to analyze, integrate, and visualize heterogeneous geoscience data in two, three, and four dimensions. The IDV coupled with RAMADDA enables geoscience specialists the capability to share and collaborate their IDV analysis via social scientific networks.
  3. Provide cyberinfrastructure leadership in data discovery, access, and use
    RAMADDA allows geoscience specialists the ability to search and publish their IDV bundles on-line. Unidata's RAMADDA installation enables the IDV team to communicate more effectively to our users concerning their IDV issues. Specifically, during support ticket conversations, the IDV team requests that users upload pertinent data to RAMADDA for analysis. The IDV team also takes advantage of RAMADDA to share instructional IDV screencasts with users.
  4. Build, support, and advocate for the diverse geoscience community
    Unidata offers yearly multi-day training and occasionally regional workshops for IDV and RAMADDA. The IDV coupled with RAMADDA enables our earth science community partners to distribute geoscience data and metadata through web-based technologies thereby fostering scientific collaborations. Moreover, the IDV's ability to share bundles through RAMADDA creates a scientific social and collaborative network for the geoscience community.

Activities Since the Last Status Report

System Changes

Java 7 / Java3D 1.6 migration: Java 7 / Java3D 1.6 migration: Java 6 reached end-of-life in 2013, and Java 3D < 1.6, JOGL < 2.0 are no longer supported. Because of these changes and to take advantage of new Java 7 features, we migrated the IDV to Java 7 and Java3D 1.6. This process was somewhat difficult and time consuming on Apple OS X where we transitioned from an Apple or Oracle Java Runtime Environment. There were three steps involved in this effort:

  • Collaborated with the Java OpenGL (JOGL) open-source community to resolve issues in Java3D 1.6 on the Apple OS X operating system. In particular, we reported multi-tab rendering and deadlock problems and worked with the community to resolve them.
  • Identified and resolved Java Swing (non-Java3D) user interface issues such as font antialiasing and look and feel problems.
  • Modified and enhanced our install4j workflow to deploy the IDV with Java 7.

This transition ensures a smooth collaboration with the netCDF-Java project which must make use of Java 7 for improved I/O. Moreover, this migration also benefits the VisAD project (the scientific 3D rendering library used by the IDV) and any project that leverages VisAD. In sum, these improvement will benefit the IDV, McIDAS-V, and VisAD user communities. Lastly, this work will ensure the IDV will continue to work on Mac OS X, Linux, Windows and Solaris platforms for the foreseeable future.


Latest netCDF-Java Version: 4.3.21: Please see the netCDF-Java and TDS update item for more details on these changes.


Install4J deployment workflow improvements: Improved the deployment workflow of the IDV via Install4J. Users are now presented with a current installation warning after they have selected a directory where the IDV may be already installed. When a current installation directory is detected and the user does not want to overwrite, they are presented with a new directory.


IDV Memory Usage: We have added a command line argument to disable explicit garbage collection within the IDV code. As the Java Virtual Machine evolves its already sophisticated memory management algorithms, it is simply best to let Java manage its own memory.


Display Changes

New ADDE Image Chooser: Added an image preview panel in the data subset window, and moved the image property widgets from the original chooser to the Advanced panel in the subset window. The result is a more intuitive and efficient user experience when selecting the area of interest before creating the final display.


Progressive Resolution (PR): This capability is a new advanced feature in the IDV. When loading large datasets with PR enabled, the IDV calculates the resolution of the map view window, dynamically sets the magnification, and loads sufficient data to generate the image. The quality of the image is not compromised. The result of this improvement is more efficient use of both client and server system resources thereby reducing network traffic significantly.


Match Display Area (MD): When the user selects this option in the data source property or subset panel, the IDV will automatically spatially subset to match the display area in the view window. This feature will allow IDV users to switch to new areas of interest by simply selecting the new projection.


Data Changes

GEMPAK Upper Air format Support: Satisfying a request from the September 2013 User Committee Executive Session, the IDV and netCDF-Java groups collaborated to arrive at an early prototype GEMPAK upper air data format IDV display capability.

IDV and RAMADDA Training and Conference Attendance

RAMADDA

No activity from Unidata although the ramadda.org team continues to make progress on RAMADDA.


Planned Activities

New Activities

Preparing for the IDV Regional Workshop at Miami University on April 18-19.


Relevant Metrics

The number of both casual and regular IDV users is increasing. For example, in October 2012, there were 596 IDV users starting the IDV more than 5 times per month compared with 651 users for the same period in 2013. In November 2012, there were 88 IDV users starting the IDV more than 30 times per month compared with 91 users for the same period in 2013. Notably, there are large numbers of new IDV users are from China. Readers can find the raw metrics at http://www.unidata.ucar.edu/software/idv/logging/left.html.


In the area of greater collaborative development, since the migration of the IDV project to github, we have accepted 65 "pull requests" or code contributions from internal and external collaborators.


In the area of online IDV training, the Youtube IDV instructional videos have been viewed a total of ~2000 times or roughly a 3 fold increase from six months ago.