McIDAS, ldm-mcidas, GOES Ingest and NOAAPort Status Report
Winter 2008/2009

Overview

The use of McIDAS in the U.S. educational community remains largely unchanged since the last User Committee meeting in October, 2008.

• Unidata McIDAS-X v2008 was made available on August 17, 2008. The UPC continues to make bugfix and enhancement upgrades available through an addendum process
• The UPC continues to receive sporadic requests for McIDAS from international university users and U.S. government agencies. Government agencies are referred to UW/SSEC for access to McIDAS; international educational community user requests are granted on a case-by-case basis after they provide a clear statement of their acceptance of the terms of use provided by SSEC.
• ldm-mcidas v2008 includes support for USPLN lightning data
• Unidata-ingested GOES-10 imagery has been included in Unidata-Wisconsin GOES-East composite image sectors since January 31, 2007.
  Unidata's GOES-10 ingester, goessouth.unidata.ucar.edu, serves, on-average, ~1 TB/month of image data via McIDAS ADDE.
• Unidata NOAAPort ingest package continues as a community resource for IDD HDS, IDS|DDPLUS, NGRID, NIMAGE, and NEXRAD3 data

Unidata McIDAS-X, -XCD (Yoksas)

• McIDAS-X/-XCD version 2008 were released by SSEC on May 28, 2008, and June 5, 2008 respectively. The Unidata McIDAS-X/-XCD v2008 distribution was released on August 17, 2008.
  • The most significant feature of v2008 is the support of NCEP-created GRIB/GRIB2 fields in the GRIB ADDE server. Speed of access to model fields using the GRIB filer approach (which uses MySQL) is greatly improved over the decode option.

    Problems uncovered with GRIB2 processing on Solaris 10 x86-64 systems are being addressed in Unidata.

  • Several new modules were added and existing ones were modified in preparation for an upcoming McIDAS-X BUFR server and McIDAS-XCD BUFR filer; the server and filer are *not* complete in version 2008; a separate, alpha version, McIDAS BUFR package that requires Java is planned for later in 2008.

• The McIDAS v2008 training workshop was officially cancelled due to sparse sign-ups. Since the cancellation, however, there have been several inquiries about the availability of a training session focusing on McIDAS' image capaibilites. The UPC is reconsidering holding a training workshop for a half dozen people from NREL in the slot made available by the cancellation of the GEMPAK training workshop.

• The number of US sites using Unidata McIDAS continues at around two dozen. For a historical comparison, 102 sites downloaded Unidata McIDAS-X/-XCD v2004.

  The majority of new interest in Unidata McIDAS is coming from organizations outside of the US. Three new international universities have requested access to Unidata McIDAS-X v2008 since its release in August:

  • Balearic Islands University (Spain)
  • University of the West Indies (Barbados, affiliated with the Caribbean Institute for Meteorology and Hydrology)
  • Eötvös Loránd University (Hungary)
  • Universidad de Buenos Aires (Argentina; reactivating their use of McIDAS)

• The first bugfix/enhancement update v2008 is being prepared. Modifications are related to support for 64-bit architectures.

• Continuing support/bug fixes for Unidata and SSEC McIDAS-X, -XCD.

• Internet2 bandwidth usage by McIDAS ADDE ranks as the second highest Advanced Application user of I2 bandwidth at approx. 0.5 TB/week. Notably the LDM ranks first in I2 bandwidth use for Advanced Applications.

Unidata McIDAS-X/-XCD Distribution License (Yoksas)

Concerns about distribution of Unidata McIDAS-X to institutions outside of the United States (e.g., CPTEC/INPE in Brazil and CONAE in Argentina) resulted in a letter a letter of clarification for distribution of Unidata McIDAS-X to better identify who may be allowed to legally download Unidata's distribution of McIDAS-X. The letter reaffirms Unidata's right to distribute McIDAS-X to the "international university community" while providing guidelines for which institutions may be regarded to be members of that community.

Prospective non-.edu users now have to read and agree to the following:

"SSEC considers permissible users of the international university community to be those who are using McIDAS-X for academic classroom instruction or academic research by students related to obtaining a degree. Research institutes and government agencies that use McIDAS-X for meteorological research or operations must purchase a license from SSEC for the non-academic use. Given this clarification, and if Unidata users remain within the licensing terms that include no re-distribution of any portion of the software, SSEC is agreeable to providing this software to the international university Unidata user community.

We need a clear statement from you that specifies all of the following:

• you are accessing McIDAS from Unidata for use in a university
• McIDAS must be used for academic classroom instruction or academic research by students related to obtaining a degree
• your installation of McIDAS is not for use by a government agency, non-academic research institute, or private concern
• you agree that you will not re-distribute any portion of the software

As soon as you state that your use of Unidata McIDAS will conform to the items above, we will add your nangyalka@yahoo.com email address to the list that allows downloading of Unidata McIDAS."

Sites that can not meet these criteria (e.g., U.S. government agencies, private corporations, non-educational international users, etc.) are referred to the McIDAS Users Group (MUG) at UW/SSEC for assistance.

ldm-mcidas Decoders (Yoksas)

• A new version, v2008, includes support for decoding of USPLN lightning data that is now available via IDD EXP feed from WSI Corporation (and Embry Riddle Aeronautical University. Release of the new version will be made in tandem with a new distribution of GEMPAK (v5.11.4) as both packages contain support for the two different datastream formats available for USPLN data.

• McIDAS-specific decoders (nldn2md and proftomd) are being integrated package into the Unidata McIDAS-X/-XCD package. After the integration, the ldm-mcidas package will only contain decoders for the compressed imagery being distributed in the IDD (Zlib-compressed NOAAPort GINI imagery; PNG-compressed Nexrad Level III composites in GINI format; and PNG-compressed Unidata-Wisconsin imagery in AREA format).

GOES Ingest Capabilities (Yoksas, Schmidt)

• During the Fall 2006 Policy Committee meeting, a response to the call for internal proposals for the UOP Director's STORM fund was developed. The proposal, representing a collaboration among Unidata, and NCAR's Earth Observing and Research Applications Laboratories, requested funding for the following activities:
  • Purchase of equipment needed to ingest satellite imagery from the GOES-10 spacecraft using an existing satellite dish installed at the Mesa Lab
  • Purchase of new equipment, and upgrades to leverage existing equipment to ingest GOES-East/West imagery using the satellite dishes installed at FL-2
  • Provision of real-time, programmatic access to the data ingested from all three GOES platforms using Unidata THREDDS technologies

  In October, 2006, UCAR submitted this proposal to NESDIS which funded it for the full requested amount of $30K.

  On the evening of January 30, Unidata successfully began ingesting GOES-10 (referred to internally as GOES-South) imagery using equipment that had been surplused by various groups in NCAR/UCAR and the Unidata community (e.g., retired USAN satellite dish from NCAR/CISL; TeraScan (tm) satellite receiver (SeaSpace Inc.) from NCAR/RAL; PC satellite ingest card from the Weather Underground, Inc., etc.), and McIDAS SDI software licensed from the Space Science and Engineering Center (SSEC) of the University of Wisconsin-Madison.

  On Wednesday, January 31 GOES-South imager sectors were integrated into the Unidata-Wisconsin datastream (IDD feedtype UNIWISC) for the benefit of the US and increasingly international Unidata community, especially university participants in Latin America. Any system participating in the IDD has access to the data (currently over 400 systems at over 210 sites). GOES-South imagery is now accessible to any ADDE-enabled application (e.g., McIDAS, IDV, etc.) with Internet access.

  The establishment of GOES-10 ingest capabilities in UCAR was a large portion of Rick Anthes President's Corner article in the March 2007 UCAR newsletter.

  SDI interface cards donated to the UPC by the LSU/SRCC and St. Louis University will be used to establish a GOES-East and GOES-West ingest and ADDE serving capability in UCAR and as spare parts (active, Schmidt, Yoksas).

  Activities of note:

  March-May, 2007:

  • recertify the satellite dish de-ice subsystem (complete)
  • advertise the availability via McIDAS ADDE of real-time GOES-South data to the Unidata community (complete)
  • incorporate real-time GOES-South, GOES-East, and GOES-West data into THREDDS and ADDE server installations on motherlode.ucar.edu (in planning)

  Summer-Fall, 2007:

  • new LNBs were installed on the GOES-East and GOES-West dishes located at the front of FL-2. The LNBs were purchased by RAL and EOL as part of their contribution to the STORM project.

  • move GOES-South ingest to satellite dish located between FL-1 and FL-2 that was given to Unidata by RAL for this activity. This dish is newer and easier to motorize than the Mesa USAN dish that was originally used for GOES-South ingest.

  Winter-Summer, 2008:

  • motorize GOES-South ingest dish to track GOES-10 as its orbit degrades (complete).

    In order to do this while keeping within the constraints of a limited budget, Unidata personnel (Schmidt) fabricated and installed a new mount for the GOES-10 dish.

  • tap signal and data off of existing NCAR/RAL TeraScan GOES-East and GOES West reception systems (using equipment surplused by UCAR/JOSS; LSU/SRCC; and St. Louis University), and ingest the data directly into formats that can be accessed by McIDAS ADDE (complete).

  • Real-time GOES-East, GOES-West, and GOES-South imagery is now being made available via ADDE off of the GOES ingest systems in UCAR.

    Full resolution GOES-East and GOES-West imagery is also available via ADDE from Unidata's toplevel IDD relay node housed in the SSEC Data Center. This same machine hosts a rolling multi-month, realtime archive of images broadcast in the IDD Unidata-Wisconsin datastream.

  • work with NCAR/EOL to make the GOES-South data available in TeraScan TD format (complete)

  • archive GOES-South, GOES-East, and GOES-West data on the NCAR MassStore (in planning)
  Winter, 2008:
  • Using monies remaining in the STORM budget, the Mesa dish will be repurposed to support ingest from all current GOES platforms: GOES-East (GOES-12), GOES-West (GOES-11), GOES-South (GOES-10), and GOES-13.

NOAAPort Ingest (Schmidt, Emmerson, Yoksas, Weber)

• The Unidata-developed DVB-S NOAAPort ingest system as been In use at the UPC and several (8 operational + 2 in progress) Unidata sites since the NWS transition to DVB-S broadcast. At least 4 non-Unidata sites are known to be using the ingest system, but no effort is spent in tracking its use.

  SSEC provides and supports the Unidata DVB-S NOAAport ingest system for its licensed users of McIDAS-XCD.

  User Support occasionally gets questions on the NOAAPort ingest software package, but the support load due to the inquiries is insignificant.

  The NOAAPort ingest system installed at the Universidad de Costa Rica (UCR) in San Jose, Costa Rica continues to function as their primary source of NOAAPort data. This NOAAPort reception system installation is a high profile undertaking at the UCR, so its continued success remains important for Unidata. (Yoksas)

  This augmentation of internet connectivity at UCR will allow for the receipt of high resolution model data (e.g., CONDUIT data) that will be used in local modeling efforts. It will also allow the UCR to take a leading role in the provision of real-time data in Central America through the establishment of the IDD-Caribe.

by Tom Yoksas.