LDM

Status Report: October 2013 - March 2014

Steve Emmerson, Mike Schmidt, Tom Yoksas

Strategic Focus Areas

The LDM group's work supports the following Unidata funding proposal focus areas:

  1. Enable widespread, efficient access to geoscience data
    The LDM powers the Unidata Internet Data Distribution (IDD) system.
  2. Provide cyberinfrastructure leadership in data discovery, access, and use
    The LDM allows sites to move data in their own environments.
  3. Build, support, and advocate for the diverse geoscience community
    The LDM is used by US universities and by entities throughout the world.

Activities Since the Last Status Report

LDM-7 proposal approved by NSF

As previously reported, NSF approved a 2 year project to integrate into the LDM the Virtual Circuit Multicast Protocol (VCMTP) previously developed in an EAGER grant with the University of Virginia and to deploy the modified LDM (LDM-7) to various test universities that are equiped with DYNES routers. If successful, this has the potential to greatly reduce the bandwidth used by the UPC to distribute data via the Internet Data Distribution (IDD) system.

Work has started on the multicast component of LDM-7. The VCMTP receiver component is about half done.

Work on hardening NOAAPORT GRIB ingestion

On Feb 25 at 15:41:54 UTC LDM processes at several sites that were ingesting binary data-products from NOAAPORT crashed due to a segmentation violation (basically, the programs attempted to access an invalid memory location). Subsequent examination of the GRIB library used by the ingester programs revealed places where insufficient verification of the input data was occurring. The working hypothesis is that a poorly-constructed GRIB message was broadcast that activated a latent bug in the GRIB library.

The GRIB ingestion software has been enhanced to reduce this risk and a new version of the LDM will be released shortly.

Incorporation of retransmission-request code from Raytheon

For the AWIPS-II system, Raytheon added the capability for the LDM to request retransmission of NOAAPORT data-products that are missed. This code has been merged into the LDM codebase. Unfortunately, this capability is useless for non-NWS sites.

Planned Activities

Ongoing Activities

We plan to continue the following activies:

  • Support LDM users
    • Email, phone, etc.
    • Training workshops
  • Work on multicast-capable LDM-7
  • Incrementally improve the LDM as necessary
  • Incorporate additional AWIPS-II-related changes into the LDM
  • Update table-driven decoding of GRIB products as necessary