McIDAS, ldm-mcidas, Collaborations, Support, and Cooperating Community Server Activities
Spring 2003

Tom Yoksas
May 8, 2003

Unidata McIDAS

• Version 2002 released on August 1. As of April 30, 2003 89 sites have downloaded Unidata McIDAS 2002. For comparison, 93 sites downloaded version 7.8x (several sites have chosen to not upgrade to v2002 for some reason).
• since August 1 release of Unidata McIDAS Version 2002, there have been 2 addenda that fixed bugs, added functionality. Addendum v2002b, released on April 1, added support of the latest version of Tcl/Tk, Version 8.4.1.
• continuing support/bug fixes for Unidata and SSEC McIDAS-X, -XCD (supported version: 2002x)
• McIDAS is now able to serve sounding data through the remote ADDE server, vpserv, under Linux. Given the number of sites that have migrated to Linux, finding this bug was made a top priority.
• ADDE grib server as imagery (proof of concept done for serving NEXRAD composite grib messages produced by GEMPAK included in FNEXRAD LDM datastream)
• ADDE server for grid and point source data in GEMPAK format(s) is a priority. This work will allow COMET case studies to be directly usable by ADDE enabled applications (McIDAS and IDV).
• porting of McIDAS-X to MacOS-X was worked on actively for a short period of time. Investigations revealed problems with the OS-X loader, and an incompatibility with the version of X running on the test machine. Further development has been taken over by SSEC.
• there has been some work on the McIDAS ADDE server for data in netCDF format that has been folded back into the SSEC McIDAS code base.
• effort will be needed to support ADDE serving of zlib compressed GINI imagery. The NWS is moving to a zlib compression method for shrinking the size of NOAAPORT broadcast GOES imagery. Lobbying for adoption of the Unidata (Chiswell) PNG compression technique failed.

ldm-mcidas and Unidata-Wisconsin datastream

• current release is v2002b (release announced on January 7).
• upgraded both Zlib and libPNG components to match current releases (important given security hole found in Zlib release in older distribution)
• the NEXRAD radar composites (sent the FNEXRAD IDD feed and available by 6 cooperating community ADDE servers) continue to be some of the most popular data products being offered by Unidata
• replacement of the MDR product in the Unidata-Wisconsin datastream with the 6 km version of the base reflectivity composite producted by Unidata put on hold.
• new Sun Enterprise 450 installed in the SSEC machine is now operational. This machine took over creation and dissemination of Unidata-Wisconsin products on April 18. It is also acting as a top level IDD source for NOAAPORT NWSTG data and is configured to provide ADDE access to substantial sequences (currently 3 weeks) of GOES-East and GOES-West imagery (both imager and sounder products). Provision of ADDE access to non-realtime (greater than 3 days old) METEOSAT imagery, running of a DODS server; and provision of THREDDS services in progress (Yoksas, Whittaker).
• a Sun 280R (dual 900 Mhz SPARC with 8 GB RAM and substantial disk) was installed at the NSF offices in Reston, VA. This new machine uses Unidata-developed, 4-channel NOAAPORT ingestion capabilities to provide top level IDD injection and ADDE serving duties from the PC machine known as 'atm'.
• two CIMSS Wildfire ABBA products are scheduled to be added to the Unidata-Wisconsin datastream.

Collaborations

• NOAA/FSL
  
  • Met with FSL representatives to explore the issues in the UPC setting up an FX-Net server installation to support the increasing number of universities requesting such a service from FSL. Comments by the FX-Net technical lead (Sean Madine) indicate that operation of a server installation would likely require 2 FTEs and at least 5 properly configured PCs running RedHat Linux.
  • We continue to track FX-Linux software developments at FSL.
  • we will be setting up a meeting with an FSL wavelet compression expert to discuss issues related to compression of satellite imagery and 4D model data for IDD distribution.

• AMRC, AMRC/SPAWAR/NCAR/NSF
  • ADDE list, get, and put servers for Level 1B formats (unofficial release in McIDAS by SSEC)
  • AMRC ADDE server for Antarctic point and imagery data (imagery data operational; point source data will be added to mix as SPAWAR/AMRC data exchange evolves)
  • collaboration with NCAR, NSF, AMRC, and SPAWAR to include SPAWAR Antarctic data on AMRC hosted ADDE server (in progress) and inject that data into IDD (future) and AMPS 30 km model output into the IDD (coordination needed with Bill Kuo)

• COMET - MeteoForum
  • continued working with MeteoForum WMO RMTC representatives to assess potential for LDM feed of conventional and select model data).
  • Realtime feeds operational to the Universidad Federal do Rio de Janerio and the Universidad Federal do Para (Belem).

    Success of current data delivery is based on LDM-6 advances coupled with the international networking initiative AMPATH (Florida International University, Global Crossing, and Internet-2).

  • Universidad Federal do Rio de Janerio (UFRJ) is now acting as the top level IDD distribution node for Brazil and Argentina. UFPA and other Brazilian universities will feed directly from UFRJ after they have been upgraded to the version of the LDM that is currently in short cycle development and testing. The resulting data distribution network will be known as IDD-Brazil.
  • presented an overview of MeteoForum objectives at the Internet-2 workshop to be held in Washington, D.C. in April. The presentation is available online from the Internet-2 web site in MeteoForum: An International Network of Meteorological Training Centers for the 21st Century

• Litton/TASC
  • continue to provide Unidata McIDAS-X port for Digital Unix (Note: Digital Unix/Tru64 support is slated to be phased out by HP-Compaq)
  • provide package support beyond issues related to OS support

• NASA/JSFC
  • Adapted ADDE NEXRAD Level III product server to work on files FTPed from NWS servers
  • McIDAS ADDE server, display, analysis and user interface issues (ongoing)

• NOAA/NESDIS/DESA
  • continue to provided Unidata/Plymouth FreeBSD port of McIDAS

• NOAA/SATEPS
  • continued to maintain communications for NOAAPORT-related issues.
  • provide support for FreeBSD implementation of McIDAS

• SSEC
  • ADDE server dynamic dataset design (Unidata)
  • McIDAS-X support for little-endian platforms (Digital Unix, Linux, FreeBSD, MacOS-X; Linux "support" adopted by SSEC)
  • v2002 bug fixes and enhancements (Yoksas, various)
  • provide programatic enhancements and bugfixes to core McIDAS (Yoksas)

• UCAR/JOSS
  • consulted with UCAR/JOSS personnel on TerraScan (tm) to AREA imagery conversion (Gallant, Yoksas)

User Support

• continuing support of over 150 sites
• growing participation by foreign universities (McIDAS-X, LDM) (Argentina, Brazil, China (Hong Kong), Costa Rica, France, Portugal, United Kingdom, Jamaica, Mexico, Spain, Russia (Institute of Atmospheric Optics in Siberia), Thailand)

Cooperating Community Data Servers - Prelude to THREDDS

Status unchanged since February Policy Committee Status Report

• the list of collaborating servers for ADDE access to McIDAS-XCD products, Unidata-Wisconsin and FNEXRAD imagery, NOAAPORT GINI imagery, NEXRAD Level III imagery, Antarctic point source and image data, and METEOSAT imagery will grow with the addition of the E450 at SSEC. Servers are being used as realtime data source and near realtime backup. They are accessible to ADDE-enabled applications (e.g., McIDAS, IDV).

  Examples of all imagery available by IDD to Unidata sites can be seen in: http://www.unidata.ucar.edu/data/datastream.html. and: http://www.unidata.ucar.edu/data/radar.html.

  Site	Contact	Datasets Served
  AMRC	Lazzara	Antarctic image and point source datasets; will be expanded to include decoded SPAWAR datasets as they become available
  College of DuPage (in negotiation)	Sirvatka Yoksas	Unidata-Wisconsin imagery, XCD-produced point source data, NOAAPORT NEXRAD Level III and GOES-East GINI imagery
  CU/CIRES	Hsie Yoksas	everything except Antarctic and METEOSAT datasets
  Embry Riddle Aeronautical University	Herbster Yoksas	everything except Antarctic and METEOSAT datasets (as soon as ERAU's NOAAPORT reception system is installed and operational (soon))
  University of Georgia	Mote Yoksas	everything except Antarctic and METEOSAT datasets
  University of Nebraska-Lincoln	Rowe Yoksas	everything except Antarctic and METEOSAT datasets
  Plymouth State College	Koermer Yoksas	Unidata-Wisconsin imagery, NOAAPORT GINI and NEXRAD Level III imagery. Eventually will serve McIDAS POINT and weather text data and possibly METEOSAT imagery.
  UCAR: Unidata/SCD	Yoksas	everything except Antarctic and METEOSAT datasets
  NSF: Unidata	Yoksas	everything except Antarctic and METEOSAT datasets
  Universite du Quebec a Montreal	Page	Unidata-Wisconsin imagery, NOAAPORT GINI and NEXRAD Level III imagery.
  Northern Illinois University	Sebenste Yoksas	Unidata-Wisconsin imagery, XCD-produced point source data, NOAAPORT NEXRAD Level III imagery
  SSEC: Unidata/SSEC	Yoksas	everything except Antarctic datasets; includes service for full resolution GOES East/West imagery (in progress)

• Unidata has ADDE access to METEOSAT, INDOSAT (METEOSAT-5), GMS, GOES, and polar orbiter imagery through NOAA/SATEPS. Some of this data will be made available to the Unidata community on the the new Sun E450 installed in the SSEC Data Center and potentially on one or more of the cooperating community servers.

• incorporation of THREDDS XML catalog services use in Unidata McIDAS GUI, MCGUI slated for future McIDAS-X release Required upgrade of Tcl/Tk version to 8.4.1 in Unidata McIDAS is now complete; upgrade of Tcl/Tk requires rework of some McIDAS services that were added to McIDAS Tcl/Tk windowing shell, mcwish. Use of THREDDS catalog information will be incorporated when cataloging has stabilized.

• NCAR/SCD- Unidata-SCD data server project. (Yoksas, Chiswell, Schmidt, Davis, Caron, Worley)
  • act as top-level IDD node (operational)
  • act as ADDE server for McIDAS-XCD, NOAAPORT GINI, NEXRAD Level III product, and NEXRAD composite base reflectivity imagery (operational and active)
  • act as DODS server for NWSTG data in netCDF form (operational and expanding)
  • Web and FTP service to raw and decoded (GEMPAK, McIDAS, and netCDF) IDD data (announced to select users with statement of intention to not use this as a replacement for a real time feed of data)
  • web page access to IDD data being ingested provided (Chiswell)
  • LAS access to NCEP Reanalysis II (SCD, active but not advertised)
  • archive IDD raw data to MSS (operational, DDPLUS and HRS are stored on MSS for 1 year (SCD); possibility of saving CONDUIT data has been raised with SCD (Worley): comments are that CONDUIT contains "a significant amount of data and would require substantial personnel time and MSS resources, so we must consider the various impacts carefully")
  • access to NCAR Mass Store data archives (future?, SCD? there are no Unidata resources and apparently no SCD resources to dedicate to this effort)

• SSEC
  • Sun Enterprise E450 (4x450 Mhz, 4x1 GB RAM, 1 TB disk) installed and running in SSEC Data Center. This machine will took over Unidata-Wisconsin datastream preparation activities on April 18. This machine will also be used to provide the Unidata community with access to high resolution GOES (East/West), METEOSAT (7 and MSG after non-6-hour imagery is 3 days old), and GMS (with MTSAT follow-on)

This page was prepared by Tom Yoksas.
Send questions/comments to <support@unidata.ucar.edu>.