Abstract Data Distribution Environment - Tom Yoksas

In 1993, SSEC began work on a Data Distribution Environment (DDE) for
McIDAS initially to help lessen data supply burdeons on their IBM MVS
mainframe.  Previous to the DDE, mainframe commands were run to produce
products that were distributed to end-users; this is the way that the
products in Unidata-Wisconsin datastream have been created and
transmitted.  The DDE significantly lessens the impact on the MVS machine for the
same amount of data transferred since individual commands do not have
to be scheduled on the mainframe.
 
In 1994, SSEC renamed the DDE to ADDE because of Microsoft's use of DDE
and OSF's use of DCE and expanded its scope.  ADDE, the Abstract Data
Distribution Environment, extends the goals and concepts in the DDE in
a variety of ways.  A quick checkbox comparision of the packages is
presented in Table 1.
 
                              Table 1
                            DDE vs ADDE
                Preliminary Goals/Requirements comparison
 
Goal/Requirement                                                 DDE
ADDE
------------------------------------------------------------------------------
Make data available to McIDAS from distributed databases          X      X
Make data available outside McIDAS from distributed databases            X
Make data in McIDAS format available                              X      X
Make data in a variety of formats available                              X
Reduce resources on the mainframe per workstation                 X      ?
Allow a smooth transition to a distributed system                 X      X
Solidify a higher level API for data access across platforms      X      X
Make this client/server software freely available                        X
Simple, operational administration                                       X
Robust, rigorous error handling                                          X
Comprehensive set of applications                                 X      X
Named data locations                                              X
Named data sets                                                          X
Highest possible performance                                      X      X
Handle more data types, e.g. text                                 X      X
 
The most important aspect of ADDE for Unidata users at this moment is
the development of a new set of McIDAS applications that use the ADDE
to get data for plots/contours/displays.  The ADDE versions of McIDAS
commands follow a client-server model:  applications (clients) make
requests for data to standalone processes (servers) via an API
developed by SSEC.  The job of understanding the data file structures
is, therefore, relegated to the data servers.  The servers need not be
colocated with the machine running the applications or even exist in
the local machine's domain.  This structure lays the groundwork a
truly distributed data processing system.
 
The fact that the application has been freed from having to understand
the structure of individual data files, and since the data request API
is established, we believe that it will be straightforward to add
servers to data of interest to the Unidata community (e.g. netCDF,
grib, raw NIDS, etc.).  At several times, SSEC has expressed in the
addition of GRIB, HDF, and netCDF data servers to McIDAS.  If that
development effort is not forthcoming, or if the timeframe for the
development does not fit with Unidata needs, then we will undertake
the development of the netCDF server.