McIDAS, GOES Ingest, IDD Unidata-Wisconsin Datastream, ldm-mcidas, and Collaborations Status Report
Fall-Winter 2006/2007

Tom Yoksas
March 2, 2007

Vision

Efforts referenced within this report are based on five of the six endeavors articulated in the Unidata 2008 Proposal:

• Endeavor 1: Responding to a broader and more diverse community
• Endeavor 2: Comprehensive Support Services
• Endeavor 4: Software to analyze and visualize geoscience data
• Endeavor 5: Distributed, organized collections of digital material
• Endeavor 6: Improved data access infrastructure

---

Unidata McIDAS-X, -XCD

• Version 2006 released on August 4, 2006. As of March 2, 2006 40 sites have downloaded Unidata McIDAS 2006. For a historical comparison, 102 sites downloaded version 2004 and approx. 60 downloaded v2005.

  The majority of new interest in Unidata McIDAS is coming from sites outside of the US.

• six bugfix/enhancement updates have been released for v2006.

• Direct access of PNG-compressed AREA format images in the IDD UNIWISC (Unidata-Wisconsin) and GINI format Nexrad composites in the IDD FNEXRAD datastreams was added in addendum v2006f.

• Highlights of the v2006 release include:
  • A new command, IMGRBG was added to McIDAS-X. IMGRGB combines images and basemaps into a multi-banded RGB image.

  • WWDISP was modified to correctly plot severe thunderstorm and tornado watch boxes. WWDISP in versions prior to 2006 will not plot the boxes because of a February 2006 change to the National Weather Service product headers.

  • GRDLIST output was modified because another space was required in the LEVEL field. To accommodate the extra space there while keeping the overall output width to 80 characters or less, the DAY, TIME and SRC fields were moved one space to the right, and the FHOUR field was reduced by one character.

  • The remote ADDE server installation script was removed from the Windows XP distribution.

  • A major change for the Unidata v2006 MCGUI is the reformulation of the image display definition file ADDEIMAGE.CORE. In an effort to make ADDEIMAGE.CORE (and ADDEIMAGE.SITE and ADDEIMAGE.USER) entries easier to create/modify, the format of the file has been completely changed. v2005 versions of ADDEIMAGE.(CORE|SITE|USER) are incompatible with MCGUI v2006. If you created any of these files, you need to move them out of your MCPATH directories before attempting to run MCGUI v2006.

  • A GRIB ADDE server is now available in v2006. One may continue to serve McIDAS GRID files as always, but may want to switch to the new GRIB server for faster data transfers and less disk space usage. The real-time GRIB server utilizes a MySQL database. Only GRIB files from the NOAAPORT data stream have been tested and are fully supported.

• training workshop was held on October 30 - November 2, 2006 at the CTTC training facility in UCAR. The McIDAS training session was held in parallel with the netCDF and THREDDS workshops.

• continuing support/bug fixes for Unidata and SSEC McIDAS-X, -XCD.

---

GOES Ingest, IDD datastreams, ldm-mcidas, and NOAAPORT

• GOES Ingest Capabilities (Yoksas, Schmidt)

  During the Fall 2006 Policy Committee meeting, a response to the call for internal proposals for the UOP Director's STORM fund was developed. The proposal, representing a collaboration among Unidata, and NCAR's Earth Observing and Research Applications Laboratories, requested funding for the following activities:

  • Purchase of equipment needed to ingest satellite imagery from the GOES-10 spacecraft using an existing satellite dish installed at the Mesa Lab
  • Purchase of new equipment, and upgrades to leverage existing equipment to ingest GOES-East/West imagery using the satellite dishes installed at FL-2
  • Provision of real-time, programmatic access to the data ingested from all three GOES platforms using Unidata THREDDS technologies

  In October, 2006, UCAR submitted this proposal to NESDIS which funded it for the full requested amount of $30K.

  On the evening of January 30, Unidata successfully began ingesting GOES-10 (referred to internally as GOES-South) imagery using equipment that had been surplussed by various groups in NCAR/UCAR and the Unidata community (e.g., retired USAN satellite dish from NCAR/CISL; TeraScan (tm) satellite receiver (SeaSpace Inc.) from NCAR/RAL; PC satellite ingest card from the Weather Underground, Inc., etc.), and McIDAS SDI software licensed from the Space Science and Engineering Center (SSEC) of the University of Wisconsin-Madison.

  On Wednesday, January 31 GOES-South imager sectors were integrated into the Unidata-Wisconsin datastream (IDD feedtype UNIWISC) for the benefit of the US and increasingly international Unidata community, especially university participants in Latin America. Any system participating in the IDD has access to the data (currently over 400 systems at over 210 sites). GOES-South imagery is now accessible to any ADDE-enabled application (e.g., McIDAS, IDV, etc.) with Internet access.

  The establishment of GOES-10 ingest capabilities in UCAR will be a large portion of Rick Anthes President's Corner article in the March 2007 UCAR newsletter.

  SDI interface cards donated to the UPC by the LSU/SRCC and St. Louis University will be used to establish a GOES-East and GOES-West ingest and ADDE serving capability in UCAR and as spare parts (active, Schmidt, Yoksas).

  Future Activities:

  March-May, 2007:

  • recertify the satellite dish de-ice subsystem (in progress)
  • advertise the availability via McIDAS ADDE of real-time GOES-South data to the Unidata community (done)
  • tap signal and data off of existing NCAR/RAL TeraScan GOES-East and GOES West reception systems (using equipment surplussed by UCAR/JOSS; LSU/SRCC; and St. Louis University), and ingest the data directly into formats that can be accessed by McIDAS ADDE (next step)
  • incorporate real-time GOES-South, GOES-East, and GOES-West data into THREDDS and ADDE server installations on motherlode.ucar.edu (upcoming)
  • work with NCAR/EOL to make the GOES-South data available in TeraScan TD format (upcoming)

  Spring-Summer, 2007:

  • work with NCAR/EOL to archive GOES-South, GOES-East, and GOES-West data on the NCAR MassStore
  • motorize GOES-South ingest dish to track GOES-10 as its orbit degrades (give the small reserve of fuel onboard GOES-10, NOAA has decided to not perform North-South station maneuvers, so the inclination of its orbit will continue to increase)

  Full resolution GOES-East/West imagery is also available via ADDE from a Unidata-owned, toplevel IDD relay node housed in the SSEC Data Center. This same machine hosts a rolling multi-month, realtime archive of images broadcast in the IDD Unidata-Wisconsin datastream.

• Unidata-Wisconsin (IDD feedtype UNIWISC) (Yoksas)

  IDD UNIWISC datastream continues to supply Unidata users with realtime GOES data in native projection.

  Even though GOES-South imagery has successfully been incorporated into the half-hourly GOES-East image sectors in the Unidata-Wisconsin IDD datastream, it is believed that inclusion of the available 15-minute scans would be of use to Unidata community members, especially those in South America. The additions of GOES-South imager sectors to the Unidata-Wisconsin datastream will be proposed to the Unidata User's Committee in advance of its May, 2007 meeting.

• ldm-mcidas (Yoksas)

  v2004 was released on October 15, 2004. A new, as yet unreleased, version, v2006, includes support for decoding of USPLN lightning data that could be made available from Embry Riddle University (active).

• NOAAPORT Ingest (Chiswell, Schmidt, Yoksas)

  The Unidata-developed DVB-S NOAAPORT ingest system has been in use at the UPC and several (7) Unidata sites since the NWS transition to DVB-S broadcast. At least 4 non-Unidata sites are know to be using the ingest system, but we don't spend effort tracking use.

  SSEC provides and supports the Unidata DVB-S NOAAPORT ingest system for its licensed users of McIDAS-XCD. We occasionally get support questions on the NOAAPORT ingest software package, but the support burden caused by the inquiries is minimal.

  The NOAAPORT ingest system installed at the Universidad de Costa Rica in San Jose, Costa Rica continues to function as designed. Problems, however, regularly occur as oxidation builds up in a N-connector coupling that was added by the company that re-aimed the satellite for NOAAPORT use. After the oxidation is removed by UCR personnel, data ingestion returns to normal.

  This NOAAPORT reception system installation is a high profile undertaking at the UCR, so its continued success remains important for Unidata. (Yoksas)

---

Collaborations

Collaboration activities remain largely unchanged since the May 1, 2003 Policy Committee Report.

Collaborations that are most active include:

• Unidata is actively involved in the UCAR-wide Africa Initiative. Activities include the planning of the Sahel 2007 conference/workshop in Ougadougou, Burkina Faso on April 2-6, 2007.

• Collaborations with EUMETSAT aimed at promoting use of remote access methodologies (ADDE and THREDDS), first in Darmstadt and then in member states, have resulted in free ADDE access to real-time METEOSAT-8 data (after users register with EUMETSAT) for research and training. EUMETSAT personnel (Marianne Koenig) reported on the status of this effort at the Spring 2006 AGU Joint Assembly held Baltimore, MD in May.

• INPE/CPTEC is now ingesting GOES-10 data at their installation in Cachoeira Paulista, SP, Brazil. NOAA has offered CPTEC the full suite of processing software used for GOES-East/West imagery. Since that GOES processing software is McIDAS-based, CPTEC is investigating the possibility of inviting Unidata and SSEC/CIMSS staff to visit Cachoeira Paulista and hold a focused McIDAS training workshop. The latest target date for the workshop is June 11-15, 2007.

• INPE/CPTEC is converting satellite imagery captured by its TeraScan (tm) system to McIDAS AREA format. Evaluation of the images being produced by CPTEC by the UPC showed a systematic error in each band. CPTEC was informed of the problems (see CPTEC TDF -> McIDAS AREA imagery compared to SSEC McIDAS AREA imagery for one comparison and is taking corrective action. The UPC has been promised the code so it can evaluate the effort of turning it into an ADDE TDF server.

• MeteoForum. This collaboration resulted in the creation in the active South American extension of the IDD, the IDD-Brazil. This effort captured and still holds the interest in the highest levels in Brazilian meteorological circles. More information on the IDD-Brazil can be found in the International status report.

---

by Tom Yoksas.