CONDUIT/CRAFT (C2) Steering Meeting
14 January 2004
Seattle, WA

Attendees:
Kelvin Droegemeier, Univ of Oklahoma
Bob Gall, UCAR, USWRP
Louis Uccellini, NCEP, USWRP
Allan Darling, NWS
Fred Branski, NWS
Larry Curran, NWS
Janelle Janish, NSSL
Kevin Kelleher, NSSL
Phil Sharfstein, FNMOC
Bill Callahan, WSI
Geoff DiMego, NCEP
John Ward, NCEP
Chris MacDermaid, FSL
Jim Steenburgh, Univ of Utah
Steve Chiswell, Unidata
Ben Domenico, Unidata
Linda Miller, Unidata

Kelvin Droegemeier and Bob Gall led a discussion of the history of CONDUIT and CRAFT. Both use Unidata's LDM for data distribution. The project needs a change of strategy by creating a formal advisory committee comprising university, government and private sector members. The group would be responsible for discussion of CONDUIT and CRAFT planning, and implementation strategies, etc. This group could also facilitate entraining the private sector into the USWRP planning process. CRAFT can inform CONDUIT through "practices learned."

The merged group should:

• be loosely organized
• be a grass roots effort with broad community participation; participation by the private sector can be facilitated by working with the AMS Private Sector Board and the CWSA
• meet twice annually, once at the AMS Annual Meeting.
• the meetings will be open to the public and announced in advance, though C2 may hold a closed meeting in advance in order to conduct its business.

Discussion:

Fred Branski said that the NWS is looking into a Family of Services "operational" solution for delivery of the Level II data, along with the four hubs being used in conjunction with Internet2/Abilene and universities. When finalized, the FOS delivery will be one of the data delivery solutions available, which will include a charge for the cost recovery. The NWS will offer 24/7 support, customary for FOS services.

Jim Steenburgh, representing the Unidata Users Committee, presented the idea of providing the National Digital Forecast Database (NDFD) in the CONDUIT data stream. Discussion among the group revealed that the NDFD includes about 50 MB/hr (about 255 files), but plans are in place to continue expansion of the data stream. The data products are released in one lump (about 26-50 MB/hr) with the process taking about 30-90 seconds. This would increase the queing delay of products (such as the GFS) already seen by CONDUIT.

The CONDUIT data stream is sending approximately 1.0 GB/hr, peaks at 1.8 GB/hr.

Louis Uccellini emphasized the need to get NCEP grids to the community. He endorses the C2 effort and would like to see it become a main vehicle for achieving this goal for the broad community of users. C2 is a good forum to bring diverse groups together.

The WRF component is important, but must not impede operations at NWS. The 3D Var is an important aspect of WRF that the community would be of interest to the university community. Geoff DiMego pointed out one of the differences for WRF is the BUFR, versus GRIB format used for CONDUIT. NetCDF will not used for data distribution.

Some of the issues to address for the CONDUIT feed from NCEP include:

• restricted data, e.g. ECMWF
• defining data sets to be included in the data stream
• procedures for proposing data sets, etc.

CRAFT radars now number 102, comprised primarily of NWS radars, with a few DOD sites. By the end of February, there should be 124. The total radars will be 142, including NWS and DOD radars. Kevin Kelleher reported that the 4 NWS Regional HQ collection hubs might share data among one another such that each hub would contain data from all radars. A final decision about this issue has not been made.

Level II Distribution:

A "Call for Participation" asked for University proposals to provide top tier distribution of freely available Level II data to university sites on a cost-recovery basis for private industry sites were submitted to the UPC. There will be four top-tier sites, MOAs with the four will be signed and the names simultaneously released with a 1 March 2004 target date. The four top-tiers must provide 24x7 support. University of Oklahoma will continue providing the data via CRAFT until the NWS announces the selection of four sites for data distribution. NOAA is investigating the idea of providing a Family of Service solution for distribution of the Level II data. This service would be provided through a cost recovery method for the NWS.

Model Data Distribution:

• We need some procedures to determine priorities on datasets to include and which to delete for the CONDUIT feed.
• What is the target latency?
• Many of the models are on a six-hour schedule, so that needs to be considered.
• Unidata has been working on development of statistics reporting. Data can get to Brazil in 1-2 seconds. There is no longer a problem of long delays due to long distances.
• NOMADS, THREDDS and NCEP FTP servers need to be considered and included in information for the community, as options for obtaining the data.
• A hybrid system of push-pull could be the appropriate answer for making the data useful for the diverse community.
• The question of opening the project to the private sector and government users was discussed. Due to the use of Internet2-Abilene, private sector and government parties could work closely with university institutions to access the data.

Issues:

• Do not duplicate grids data stream.
• GFS being set to 384 (4 times per day)
• GFS ensemble
• Need to revisit the restricted data issue, ECMWF
• AVN is migrating to GFS file names
• Need to explore the threshold for delay in survey
• Coalesce needs
• What is the "shelf-life" of a model? 21/2 hrs?
• Deletions should be performed immediately
• Need to define a process
• Partnerships between the private sector and academia are encouraged

Action Item: NDFD not presently possible without queing delays of other CONDUIT datasets. CONDUIT will explore alternatives to making NDFD distribution possible in the future.

Action Item: Research queing delay issue: 30 minutes is unacceptable.

Action Item: Remove the RUC 40km grids (RUC 20km is being sent) and Eta 215 (20km) when ETA 218 (12km) is made available (pending research on queuing delay issue, above).

Action Item: A survey could be conducted with the community for guidance on products to be included in data stream.

Action Item: Give community 30-day notice prior to changes.

Action Item: Develop a schematic of CONDUIT similar to that for CRAFT - so that possible linkages can be better planned and understood.

Action Item: C2 needs to explore ways in which CONDUIT can benefit from the NWS version of CRAFT (i.e., the one used for operations).

Recorder:

Linda Miller - lmiller@unidata.ucar.edu
External Liaison, Unidata
University Corporation for Atmospheric Research
P.O. Box 3000
Boulder, CO 80307-3000
303-497-8646 fax: 303-497-8690