RAMADDA Status Report
Spring 2009

Jeff McWhirter & Jeff Weber

RAMADDA development continues. We are planning a non-alpha (beta) release by the 1st of April. Some of the development highlights include:

• RAMADDA has been installed on the motherlode server: http://motherlode.ucar.edu/repository.
  • A number of Unidata staffers and a number of community members have accounts on the server and have been using it for case study creation, bundle publishing, etc. We have a number of project areas including:
    • IDV Community Resources area
    • GALEON
    • Antarctic IDD
    • UCAR Africa Initiative.
  • We provide access to the full IDD feed.
  • The full COMET Case Study suite of data is loaded on new motherlode, and RAMADDA is serving up the data.
  • We are harvesting a collection of satellite and weather imagery.
  • We are providing access to a number of FTP Sites.
• Other sites
  • Matias Bonet from the IMEDEA Marine Technologies, Operational Oceanography and Sustainability Department has a RAMADDA installation serving up oceanographic data and is also using RAMADDA as a general content management system.
  • Sean Arms has installed RAMADDA at the University of Oklahoma
  • Leigh Orf is planning on a RAMADDA install.
  • Dallas Masters, the MOPPITT data manager at NCAR/ACD is planning on installing RAMADDA.
• Collaboration
  We have been exploring the role of collaboration facilities within RAMADDA. The basic idea is we want to view the artifacts of collaboration (e.g., wiki pages, forums, chat, etc) as content in the repository. This collaborative content can have descriptive metadata and can be organized, cataloged and searched just like any other content. Some of the highlights include:
  • Wiki pages - the user can create a new entry type - a Wiki page. This supports almost the full wikipedia wiki syntax and includes the ability to easily import other RAMADDA content views into the Wiki page. This provides the ability to easily create and evolve textual content along with have close integration with the data and data services that RAMADDA provides.
    Some examples.
  • Chat/whiteboard - RAMADDA now provides an integrated chat server and a Java-based Chat/Whiteboard applet. This allows for synchronous collaboration among a group of users, integrated with the other data content within RAMADDA. See here for an example.
  • Facebook integration. RAMADDA can be configured to incorporate a Facebook comment facility.
  • Future plans include implementing a weblog and forum facility.
• New data views.
  • The Local file view provides an easy way to provide RAMADDA mediated services to an arbitraty directory tree on the server machine. This is similar to the TDS catalog facility.
  • TDS Catalog View. This facility allows the user to create a new entry within RAMADDA that points to an external TDS catalog. This facility provides RAMADDA services to these external TDS sources.
  • FTP View. This is similar to the two services above. The suser creates an entry that points to an external FTP server. This can be configured to automatically fetch data files from the FTP server as needed and cache them locally so that RAMADDA can provide its full suite of services (including data services like OpenDAP). For example, here is the Iowa State MT Data Archive.
• Entry monitor and notification. The end user can create an entry monitor, specifying a set of search criteria. When a new entry is added to the RAMADDA repository it is checked against the monitor search criteria and an action is executed. The actions incude:
  • Email a specified message to a group of users.
  • Insert the file into a LDM running on the server. This would allow one to have web-based access into an LDM queue. There is also a new output type for any file entry within RAMADDA that lets one inject the file (or files) into an LDM queue.
  • Post a message to Twitter. This allows for integration into a number of other social network services (e.g., Facebook).
  • Copy the entry into another part of RAMADDA. This provides the ability to have a "record" button. For example, tapping into the real time IDD stream and automatically routing certain entries into a permanent storage area.
  • FTP the entry. This will do an FTP put of the new file to a specified ftp server.
• Security and Access control.
  • SSL support. The administrator can now specify to RAMADDA when the containing server supports SSL.RAMADDA routes all sensitive web requests (e.g., login form) to the SSL URLs.
  • Anonymous upload. A group within RAMADDA can be configured to allow for anonymous upload of new files. The group's owner is notified via email. This is implemented through the RAMADDA access control mechanism as an access type.
  • "File" access control. A new access control type that allows users to view the basic information about a file entry but does not allow them to access the file.
  • IP based access control. One can grant or deny certain access rights to certain IP prefixes.
  • A plugin based user authentication mechanism that allows for 3rd parties to integrate their own user authentication facilites (e.g., ldap).
• Miscellaneous.
  • The user interface has undergone a complete overhaul based on extensive feedback from users. This also includes a UI skin feature that allows site administrators to customize the look and feel of a RAMADDA installation and allows end users to choose their preferred look.
  • A new web URL harvester that supports harvesting web based products, e.g., weather imagery.
  • WGET script - This generates a shell script for downloading a set of RAMADDA files.
  • NCML templates for data files. One can attach an NCML file to a data file or group of files. This is used to change how the data is accessed.
  • Associations. Based on user feedback we have implemented a simple drag-and-drop mechanism to create associations between entries. There is also a search interface for associations.
  • Copy/move/etc. Big improvements on how users can manage the content.
  • User favorites. This acts as bookmarks for various areas within RAMADDA.
  • User home groups. Acts as home directories for users.
  • Annotated text view - shows line oriented views of text files.

  by Jeff McWhirter.