McIDAS, ldm-mcidas, Satellite Data Ingest
Status Report: April 2014 - September 2014
Tom Yoksas
Strategic Focus Areas
McIDAS activities support the following Unidata funding proposal focus areas:
- Enable widespread, efficient access to geoscience data
McIDAS remains the application of choice for the satellite meteorology community.
The Advanced Data Distribution Environment (ADDE) componet of McIDAS was the first application offered by Unidata to provide remote, programmatic access to a wide variety of data that is important to the atmospheric science community. - Develop and provide open-source tools for effective use of
geoscience data
The fifth generation of McIDAS, McIDAS-V, unlike its
predecessors, is a fully open source application that is
is in wide scale and growing use in the satellite meteorology
community.
McIDAS ADDE continues to evolve and provide access to increasing volumes of image and non-image data. - Provide cyberinfrastructure leadership in data discovery, access,
and use
Concepts articulated in ADDE inspired the development of THREDDS (to address the lack of rich metadata available in ADDE) and RAMADDA. ADDE remains one of the most used data services in the Unidata suite serving over 3 TB per month. - Build, support, and advocate for the diverse geoscience community
McIDAS is sought for use by those interested in satellite meteorology worldwide.
Activities Since the Last Status Report
Unidata McIDAS v2014 released on September 8, 2014
Unidata McIDAS version 2014 includes all SSEC versions up to and including the current release, v2014.1 and Unidata updates and bugfixes. Changes to Unidata McIDAS continue to be made through an addendum process.
SSEC McIDAS Advisor Committee (MAC)
The UPC (Yoksas, Ho) continues to participate as the Unidata representative to the McIDAS Advisory Committee (MAC) that is operated by SSEC.
The MAC was assembled by UW/SSEC to advise SSEC on McIDAS-X users needs/concerns/desires for development in the next generation McIDAS, McIDAS-V. The MAC was modeled after the Unidata IDV Steering Committee.
Interest in McIDAS by Non-core Users
The UPC continues to receive requests for McIDAS from international university users, U.S. government agencies and other non-traditional Unidata users (e.g., private businesses, etc.). Government agencies and non-traditional Unidata users are referred to UW/SSEC for access to McIDAS; international educational community user requests are granted on a case-by-case basis after they provide a clear statement of their acceptance of the terms of use provided by SSEC.
Planned Activities
Ongoing Activities
Continued support of existing and new community members.
New Activities
Implementing an indexing scheme for ADDE image datasets to speed up access especially in large and archive datasets. A preliminary design for ADDE image dataset indexing has been made. Investigations for how to integrate the new capbilities in to the suite of existing ADDE servers is in progress.
Add support for new types of data when they become available, otherwise McIDAS-X support is in maintenance mode.
Relevant Metrics
- Internet2 (I2) bandwidth usage by the McIDAS ADDE protocol routinely exceeds several TB/week.
- McIDAS Inquiry Metrics
ldm-mcidas Decoders Activities
Development
ldm-mcidas releases are made when needed to support changes in software development and operating system environments. ldm-mcidas v2012 was released at the end of September, 2012. This release addressed building on newer OS versions.
Geostationary Satellite Data Ingest
Unidata continues to ingest GOES-East and GOES-West imager data at the UCAR Foothills Lab campus in Boulder. GOES-South (GOES-South America) was decommissioned on August 16, 2013, and there appears to be no current plans for repurposing an existing GOES platform for South American surveillance.
- Direct, programmatic access to real-time GOES-East (GOES-13) data via McIDAS ADDE is being used by approx. 35 users who downloaded an average of approx. 0.75 TB/week over the past year.
- Direct, programmatic access to real-time GOES-West (GOES-15) data via McIDAS ADDE is used by approx. 25 users have downloaded an average of 0.4 TB/week for the past year.
Planned Activities
Ongoing Activities
Continued ingest and serving of GOES-East and GOES-West imagery. This effort sporadic requires maintenance of the satellite ingest and computer data equipment.
New/future Activities
Repurpose former USAN dish at Mesa Lab to operation as a remotely controllable ingester for any of the GOES platforms. This is a moderately low priority activity.
Proposed Activities
Begin planning for the resources it will take to ingest and disseminate GOES-R data (which is currently scheduled to become available in second quarter 2016). This activity will proceed with cooperation/coordination of NCAR/RAL, NCAR/EOL and NOAA. A draft DRAFT Executive Summary and Budget (i.e., a non-proposal "proposal") was developed in cooperation with RAL and EOL, and submitted to Steve Goodman who is in NOAA's GOES-R office. This funding effort will be resurrected for fiscal year 2014.