LDM

Status Report: May - Sep 2011

Steve Emmerson, Tom Yoksas, Mike Schmidt

Strategic Focus Areas

The LDM group's work supports five of the funding proposal's six focus areas:

  1. Broadening participation and expanding community services
    Dispite the existence of "pull" clients, (e.g., the IDV), people still want the ability to subscribe to meteorological data and have it automatically processed as soon as it's available. The LDM is one of the few mechanisms for doing this with high-volume data flows.
  2. Advancing data services
    As institutions adopt the LDM to receive IDD data and to distribute their own locally-generated data, more data becomes available to the larger community (quid pro quo).
  3. Developing and deploying useful tools
    The LDM is, apparently, useful and is being deployed more and more.
  4. Providing leadership in cyberinfrastructure
    You know you're providing leadership in cyberinfrastructure when organizations like the NWS and Raytheon start coming to you for advice on using the LDM.
  5. Promoting diversity by expanding opportunities
    Even small colleges and high-schools can use the LDM to receive and process massive amounts of data.

Activities Since the Last Status Report

Released Version 6.10.0

Major differences from the version in the last report are

  • Changes that Raytheon made to the LDM to enable it to provide data to AWIPS-II were incorporated so that our AWIPS-II users won't have to manage two different LDM-s.
  • The performance of the product-queue was improved
  • A -metadata option was added to the FILE action to cause a binary representation of the data-product's metadata to be written to the output file
  • The NOAAPORT and GRIBINSERT packages were merged into the LDM package in order to ease maintainence. The NOAAPORT package is used to create LDM data-products from the NOAAPORT satellite broadcast and insert them into an LDM product-queue and the GRIBINSERT package is used to create LDM data-products from a GRIB file (such as from a model run) and insert them into an LDM product-queue.
    • Neither subpackage is installed by default
    • A new NOAAPORT ingester (noaaportIngester) was created that consolidates the previous two programs (dvbs_multicast and readnoaaport) into a single, multi-threaded program with better performance
    • The GRIB tables used by these subpackages were updated
    • The error messages printed by these subpackages were improved
  • The command ldmadmin check was improved
  • The documentation on running multiple LDM-s on the same system was improved
  • Many bugs were fixed

Planned Activities

Ongoing Activities

We plan to continue the following activies:

  • Support LDM users
    • Email, phone, etc.
    • Training workshops
  • Incrementally improve the LDM as necessary