McIDAS, ldm-mcidas, Satellite Data Ingest

Status Report: October 2014 - March 2015

Tom Yoksas

Strategic Focus Areas

McIDAS activities support the following Unidata funding proposal focus areas:

1. Enable widespread, efficient access to geoscience data
   McIDAS remains the application of choice for the satellite meteorology community.
   The Advanced Data Distribution Environment (ADDE) componet of McIDAS was the first application offered by Unidata to provide remote, programmatic access to a wide variety of data that is important to the atmospheric science community.


2. Develop and provide open-source tools for effective use of geoscience data The fifth generation of McIDAS, McIDAS-V, unlike its predecessors, is a fully open source application that is is in wide scale and growing use in the satellite meteorology community.
   McIDAS ADDE continues to evolve and provide access to increasing volumes of image and non-image data.


3. Provide cyberinfrastructure leadership in data discovery, access, and use
   Concepts articulated in ADDE inspired the development of THREDDS (to address the lack of rich metadata available in ADDE) and RAMADDA. ADDE remains one of the most used data services in the Unidata suite serving over 3 TB per month.


4. Build, support, and advocate for the diverse geoscience community
   McIDAS is sought for use by those interested in satellite meteorology worldwide.

Activities Since the Last Status Report

Unidata McIDAS-X/XCD is in mostly in maintenance mode. Aside from routine updates/bugfixes to existing code and tables, the main thrust of development is to add indexing to ADDE datasets to speed access into large datasets.

Prior Activities

• Unidata McIDAS version 2014 includes all SSEC versions up to and including the current release, v2014.1 and Unidata updates and bugfixes. Changes to Unidata McIDAS continue to be made through an addendum process. Unidata McIDAS-X/XCD was released in September, 2014.

Ongoing Activities

• SSEC McIDAS Advisor Committee (MAC)

  The UPC (Yoksas, Ho) continues to participate as the Unidata representative to the McIDAS Advisory Committee (MAC) that is operated by SSEC.

  The MAC was assembled by UW/SSEC to advise SSEC on McIDAS-X users needs/concerns/desires for development in the next generation McIDAS, McIDAS-V. The MAC was modeled after the Unidata IDV Steering Committee.

• Interest in McIDAS by Non-core Users

  The UPC continues to receive requests for McIDAS from international university users, U.S. government agencies and other non-traditional Unidata users (e.g., private businesses, etc.). Government agencies and non-traditional Unidata users are referred to UW/SSEC for access to McIDAS; international educational community user requests are granted on a case-by-case basis after they provide a clear statement of their acceptance of the terms of use provided by SSEC.

Planned Activities

Ongoing Activities

Continued support of existing and new community members.

New Activities

Implement indexing for ADDE image datasets to speed up access especially in large and archive datasets. A preliminary design for ADDE image dataset indexing has been made. Investigations for how to integrate the new capbilities in to the suite of existing ADDE servers is in progress.

Add support for new types of data when they become available, otherwise McIDAS-X support is in maintenance mode.

Relevant Metrics

• Internet2 (I2) bandwidth usage by the McIDAS ADDE protocol routinely exceeds several TB/week.
• McIDAS Inquiry Metrics

ldm-mcidas Decoders Activities

Development

ldm-mcidas releases are made when needed to support changes in software development and operating system environments. ldm-mcidas v2012 was released at the end of September, 2012. This release addressed building on newer OS versions.

Geostationary Satellite Data Ingest

Unidata continues to ingest GOES-East and GOES-West imager data at the UCAR Foothills Lab campus in Boulder. GOES-South (GOES-South America) was decommissioned on August 16, 2013, and there appears to be no current plans for repurposing an existing GOES platform for South American surveillance.

• Direct, programmatic access to real-time GOES-East (GOES-13) data via McIDAS ADDE is being downloaded an average of approx. 1.8 TB/month.
• Direct, programmatic access to real-time GOES-West (GOES-15) data via McIDAS ADDE is being downloaded an average of approx.1.5 TB/month.