To maintain a vibrant program, the UPC, from time to time, participates in
certain projects that are tied to its overall mission but funded separately
from the core program. Such synergistic
activities are both essential and complementary to the core effort, and both
the Policy Committee and NSF encourage them. Such projects have played a vital
role in advancing the program in new directions, creating new capabilities for
the community, enhancing interoperability of Unidata software, providing new
datasets to the community, and entraining and diffusing innovative ideas and
technologies into the community.
The
Cooperative Arctic Data and Information Service (CADIS) is such a project,
supporting the Arctic Observing Network (AON) community. AON is an NSF Office of Polar Programs funded initiative
for the International Polar Year (IPY) to
improve observational capabilities in the
CADIS
is intended as a development project to bring community standards, data
submission, archival and stewardship expertise, analysis and visualization
tools, and vision to develop a comprehensive data management capability to
support AON and other IPY-related projects.
The CADIS team is made up of 23 people from NCAR (EOL and CISL), Unidata,
and NSIDC. Additional
information about the CADIS project and data provided by CADIS can be found at:
http://www.eol.ucar.edu/projects/aon-cadis/
Unidata is providing middleware (THREDDS), data
transport (LDM) for real-time AON data transmission, and analysis and
visualization capabilities (IDV). Unidata's Network Common Data Form with CF
conventions is a strong contender for use as the "compatible format,"
so Unidata is assisting in format conversion to netCDF.
The
main development areas and accomplishments to date pertaining to Unidata’s work
are described below:
• Metadata: members from all CADIS partners institutions participated in a metadata working group, which was tasked with identifying the requirements for capturing all information relevant to discover, describe and access data for the Arctic domain. The result of this effort was the definition of a CADIS-specific metadata profile, which is based on the IPY guidelines, and includes additional elements either to meet the specific requirements of the CADIS project, or to facilitate export to other schemas such as ISO and DIF. The CADIS metadata profile is meant as a specification of the general THREDDS schema, which is used as container and interoperability framework.
• Metadata Authoring Application: CISL and EOL developers have been collaborating towards the creation of a web-based metadata authoring tool, and its integration into the current CDP publication infrastructure, which is meant to be used by CADIS PIs to capture and share the information to describe their data holdings. A first prototype of this application has been completed and will be released to the PIs for feedback in early 2008.
• Prototype CADIS Web Portal: a CADIS-specific area has been established within the existing CDP web portal in order to prototype hierarchical browsing of CADIS data collections, data access, and integration with the metadata authoring tool. This work included the definition of science-based categories to organize the data holdings, as well as branding of the CDP user interface to expose a CADIS-specific look-and-feel.
• Dataset format translation: Unidata developers have provided data format translation between native formats such as GRIB, EASE Grid, and Office Note X to netCDF so that data sets in those formats can be accessed via the CADIS portal. Thus far, six candidate datasets have been identified. Unidata tools have been applied to perform those translations confirming the efficiency of this method to standardize CADIS data.
• Data Visualization: As an Interim measure, EOL has implemented a GIS Mapserver capability for locating all AON measurement locations on a polar project map. This is the first and modest step to providing interactive perusal and access via the CADIS Portal. Unidata has been working to develop visualizations of those translated datasets using the IDV. IDV will be one of the tools available to users for browsing CADIS data in the future. VETS engineers have engaged in exploratory work aimed at providing visualization and sub-setting capabilities for CADIS data, based on integration of community standard applications like LAS (Live Access Server from NOAA/PMEL) and THREDDS Data Server from Unidata. A few examples of CADIS datasets (either gridded or in-situ data) have been successfully configured and visualized, thus showing the long-term viability of this approach. As a precondition for data access, gridded datasets had to be converted from GRIB format to NetCDF, and in-situ data ingested from ASCII files into a relational database. Unidata staff are working to have example visualizations of all six types of data (mentioned above), using the IDV
• Real
time data ingest: Unidata has also been working with the CADIS team and in
consultation a couple of the AON PIs to determine the need and process for
setting up a real-time data distribution system using the LDM. As the need
arises, Unidata is ready to help AON PIs to install the LDM to move real-time
data from Arctic observatory sites to the CADIS system in