Internet Data Distribution Administration

Robb Kambic
June 1994

Description

The Unidata IDD system will eventually comprise over 100 participating sites distributing a large variety of types of data injected into the network from several separate sources. In order to deal with the growth of and changes to the resulting network we will need a system to:

• represent and report on the current IDD network routing;
• reconfigure the IDD network routing;
• detect, report, and send notices on changes in the network; and
• provide information necessary to assist in adding new source, relay, and leaf sites into the network.

The current routing will be extracted by gathering information from all the nodes in the IDD topology, similar to the LDM statistics gathering process which is done by scanning the log files. Each node will report on the nodes that it is feeding immediately downstream and then send this information back to a centralized file at UPC. By processing the centralized file, a complete routing by feed type will be produced.

Some of the routing reconfiguration will be done by scanning the centralized routing file for source nodes that need to be replaced with an alternate source node. An action for the downstream node will be constructed and a notification will be sent to each node. The remote site administrator will be responsible for implementing the changes. Software tools will be available to automate the needed changes at the remote sites. Another tool will be a third party feedhere that changes the source node for a remote destination node. In the future a fail-safe node will be part of the feedhere; if the primary source stops feeding, an auxiliary source node will start feeding. This mechanism will probably handle most temporary routing changes.

Because it is expected that the IDD routing will be in flux, there will be three types of files to store the routing states. The first type will be named routing.cur and will be a repository to capture all raw, incoming, routing data from the remote nodes. Once a day, the routing.cur file will be merged with the previous day's file to create a routing.yymmdd.file that will be the current routing configuration. The merge process will eliminate duplications and update the upstream-downstream links to the current configuration. At this point different routing files can be compared to detect, report, and send notices about the current configuration. The archive of routing.yymmdd files will record the state of the routing at any time for the IDD. The last file type, routing.target will be the desired target of the IDD routing. It will be constructed by using information from the routing.yymmdd files and the LDM statistics reports.

UPC will provide the initial routing configurations for new LDM installations by examing the current routing configuration and the LDM statistics of nodes that will be feeding the new site. An important consideration is not to overburden any one site with downstream nodes because it will cause network overloads and high machine usage.

Site Information

The site information needed to construct the system will be obtained by two different methods. An almost-static file needs to be built that contains the name of the machine the LDM server is running on and the email address for each site`s maintainer. Other information:

Upstream Node   Downstream Node  Feed Type  Pattern 

will be extracted from the LDM server's log files and then sent back to UPC.

Pros

All of the variable information will be extracted in an automated manner and then merged into reports on a daily basis.

• Reports will reflect the current routing.
• Reports will only have to run on the source and relay nodes to extract the complete IDD routing.
• The system will be developed as an enhancement to the already implemented LDM Statistic program.

Cons

Sites with complex feed type variables such as (HRS|IDS) will require extra attention if the source would be split into two different sources, one for HRS and one for IDS. At this time, this is expected to be an infrequent occurence.

Sites with complex patterns will have an even greater problem if a split should occur between different sources.

Output

Report showing a feed routing using a routing file as input.

Report showing the change notices that will be emailed to the remote sites.

The report can be edited before it is sent.