LDM-6 users,
Everyone here at the Unidata Program Center has been reading ** with
great interest ** the posts regarding our efforts aimed at developing a
Next Generation LDM. We are _very_ pleased that several community
members have shared their thoughts on this development. We sincerely
hope to hear from more of you on this important topic!
In order to provide some background on this development, we offer
the following short overview:
Why develop a Next Generation LDM when the current one works so well?
=====================================================================
The LDM has grown to be a robust, reliable and portable base on which
to build data distribution networks. As design or implementation
limitations were identified, innovative developments have been employed
to keep the LDM viable.
The current implementation of the LDM when coupled with cluster
technology may be able to effectively relay all of the data desired
by the expanding Unidata community for several more years. However,
input on the limitations in the current LDM design from you, the
user community, has convinced us that new, alternate approaches to
data distribution must be thoroughly investigated and pursued.
The need for a more scalable technology is apparent from several
trends:
- the volume of useful data appears to be growing faster than
increases in network bandwidth
- approaches to setting priorities for bandwidth by "shaping"
its use, charging departments for bandwidth, or other institutional
data policies may disrupt the model of essentially free bandwidth
currently in use for the IDD
- competition for network bandwidth from the explosive growth in
video distribution may demand more dynamic routing than is now
possible with current LDM technologies
To avoid interruption in data delivery, future needs must be
anticipated and reliable software must be developed well before the
need is apparent.
Our experience suggests the current LDM-6 may have advanced about as
far as it can given the constraints of its architecture.
Please be assured that the development of a Next Generation LDM
does not imply that incremental improvements to the current LDM-6
will cease!
However, further major advances in the LDM may require a new
protocol that is not tied to the client/server approach but instead
uses peer-to-peer concepts.
A new protocol and implementation could allow for the following
improvements:
- more dynamic creation and destruction of data-product streams
- support for access to "one-time" data products (i.e.,
data-products that are not continuously generated)
- better load balancing of communication links
- more adaptive and flexible dynamic routing of data-products with
steady-state results that are relatively independent of the
configuration of initial connections
- a better interface for selecting data-products. For example,
reception of a data-product stream could be started by clicking
on a hyperlink in a web page.
- support for use on computing platforms beyond those being used
in the community for data relay tasks (e.g., Windows)
Naturally, minimizing disruption -- both to individual sites and
to the flow of data -- is a major concern of any new implementation
and deployment. The LDM software is open source, so no one will be
forced to upgrade to different and incompatible software. In fact,
there could be two competing distribution systems running in
parallel for some time, during which the new system will have to
prove itself capable of satisfying users of the current LDM as well
as providing solutions to problems the current LDM cannot handle.
Users interested in the history of the LDM/IDD and that would like
to learn more about ongoing efforts at creating a Next Generation
LDM should take a look at the (14 page) paper we presented at the
AMS 2006 Annual Meeting:
"The Unidata Internet Data Distribution (IDD) System: A Decade of
Development"
http://ams.confex.com/ams/Annual2006/techprogram/paper_105113.htm
http://ams.confex.com/ams/pdfpapers/105113.pdf
Again, we are seeking your input on this new, and exciting development!
In order to make it as easy as possible for you to share your thoughts,
we are providing several ways for you to comment:
- send email to the ldm-users@xxxxxxxxxxxxxxxx list
- post to the ldm-users forum:
Unidata HomePage
http://www.unidata.ucar.edu
Support
http://www.unidata.ucar.edu/support/
Participate in topical forums
http://www.unidata.ucar.edu/forums/index.jspa
LDM Users Forum
http://www.unidata.ucar.edu/forums/forum.jspa?forumID=19
- provide additions-to/comments-for the wiki that Steve Emmerson
mentioned in his request for users stories/high-level usage
scenarios:
https://wiki.ucar.edu/display/unidata/Next-Generation+LDM
- send email to the Unidata User's Committee <usercomm@xxxxxxxxxxxxxxxx>
- send email to Unidata User Support <support@xxxxxxxxxxxxxxxx>
- contact any Unidata staff member directly
Please share your thoughts on the development of a Next Generation LDM
and what features you would like to see incorporated in its design!!
Cheers,
The Unidata Staff
--
+----------------------------------------------------------------------------+
* Unidata User Support UCAR Unidata Program *
* (303) 497-8643 P.O. Box 3000 *
* support@xxxxxxxxxxxxxxxx Boulder, CO 80307 *
* Unidata WWW Service http://www.unidata.ucar.edu/ *
+----------------------------------------------------------------------------+
