IDD Status

Anne Wilson
May 13, 2002

This report updates the IDD Status report from the February, 2002, Unidata Policy Committee Meeting.

Participation

Date Number of Institutions Number of hosts
October, 2001 131 244
February, 2002 155 268
May, 2002 165 292

The international reach of the IDD is growing as it is now serving sites in Australia and Brazil.

Data Feed Volumes

Feeds by popularity, of 142 sites reporting, are virtually unchanged from the February report.

Since February, although the data volumes of most feeds have remained about the same, there have been significant increases on the following feeds:

Feed Type Feb MB/hour May MB/hour Comments
CONDUIT 399.4 862.1 Due to the addition of 20km RUC output, and the extension of the AVN forecast from 120 hours to 180 hours
HDS 91.4 99.1 Additional products
NEXRAD 79.0 139.1 Seasonal weather
FNEXRAD 4.5 17.8 Due to addition of 1 km mosaics generated every 10 minutes
GEM 0.0 .7 New feed (see below)
Total - all feeds 597.6 1142.5 A cumulative increase of 91%

Impact of Increased Data Volumes

One impact of the increase in data volumes is that servers must increase their queue size in order to be able to store the recommended minimum of one hour's worth of data. A new problem, however, is that many sites are reaching or have reached the maximum queue size of 2 gigabytes.

To remedy this situation on Unidata's own server (thelma) it was necessary to rebuild the LDM with a 7 gigabyte queue. Building such a large queue requires that a host have large file support, that is, that it can support 64 bit addresses. Not all machines have this capability. Yet, it may soon be necessary to require at least top tier sites to have hardware that supports such large queues.

Using a large queue incurs a performance cost. In particular, upon establishment of a new connection, the time for a server to find its place in the queue is greater.

New Data Feeds

The Canadian Meteorological Center's Global Environmental Multiscale (GEM) model is now being relayed on a point to point basis via the IDD. The volume of the feed is relatively small: 8.3 megabytes, 430 products, twice per day.

Unidata is now receiving the entire CRAFT feed on an experimental basis. Currently 38 radar sites are participating. Regarding data volumes, one small sample showed 163 megabytes/hour, about 10,400 products per hour.

LDM 5.1.5

The new version of the LDM, 5.1.5, is still under development. It is anticipated to be released in June.


Anne Wilson
Last modified: Mon May 13 16:22:44 MDT 2002