Policy Committee Meeting Summary:

17-18 April 2006

Unidata Program Center, Boulder, Colorado

Participants

Members

UPC Staff

Melanie Wetzel (Chair) Mohan Ramamurthy

Steve Ackerman

John Caron (part of meeting)

Steven Businger

Ben Domenico

Jim Koermer

Joanne Graham

David Maidment

Jo Hansen

Gene Takle

Linda Miller

 

Don Murray (part of meeting)

Agency Representatives

Russ Rew

David Herring (NASA)

Tom Yoksas (part of meeting)

Clifford Jacobs (NSF/ATM) (Monday only)

 

Leroy Spayd (NOAA/NWS)

NCAR and UOP Representatives

 

Hanne Mauriello, UOP Observer

Committee Representatives

Tim Spangler, UOP/COMET, Observer

Jim Steenburgh (Users Committee Chair) Roger Wakimoto, NCAR/EOL, Observer
  Steve Worley, NCAR/CISL, Observer

Present via phone conference for parts of meeting

 

Philip Bogden

Not Attending

Paul Ruscher

Jack Fellows, UOP Observer

   

Thursday, 22 September 2005

Administrative Matters (Wetzel)

  • Approve Agenda
  • Set next Meeting Dates
    • 13-14 September if meeting to be in Arlington
    • 6-7 September if meeting to be at the University of Hawaii in conjunction with the 50th anniversary celebration of its Department of Meteorology.
  • Approve September 2005 Meeting Notes. The notes were approved as submitted.
  • Welcome new NASA representative: Melanie extended a welcome to the committee's new representative from NASA, David Herring.

Director's Report (Ramamurthy)

Highlights include:

  • 326 hosts in 171 network domains are running LDM-6. IDD average has increased to 3.4 GB/hr with a maximum rate of 5.5 GB/hr.
  • IDV selected as visualization tool for T-REX Field Project
  • TDS/WCS interface for NAM and WCS interface for EPA air quality data
  • 2006 UsersWorkshop-10-14 July 2006
  • THORPEX-TIGGE-LDM use for data distribution
  • LDM-6 was selected for data collection activities in the THORPEX
    Interactive Grand Global Ensemble (TIGGE).
    • During testing, the LDM demonstrated ability to relay over 17 GB/hr in a single datastream originating at the ECMWF and being received at NCAR/SCD
  • STORM funds case study underway
  • Unidata Assessment to include survey, focus groups, one-on-one interviews, evaluation of Unidata software development, and case studies to determine Unidata's transformative impact on the community
    • survey is aligned with NSF goals of people, tools and ideas
  • Community services and outreach include AGU, AMS, cyberinfrastructure/informatics meetings
  • Equipment Awards Review Panel taking place 11-12 April 2006
  • Staffing changes include termination of student assistant (planned), LEAD developer on term position, netCDF developer transferring to ESMF project, but will continue on netCDF at 25% level
  • Unidata Vision for 2013 - "Unidata will be a premier developer and provider of cyberinfrastructure for the geosciences..... "
  • Proposal activity:
    • Funded: NSF Gateway; NSF Data Access; Users Workshop; NSF/OISE: travel to AGU Joint assembly; NOAA: IOOS-DMAC
    • Pending: NSF CEO: air quality; NSF CEO: Cyberinfrastructure; NSF: Virtual Ops Center
    • Submitted: NSF/OPP: A Cooperative Arctic Data and Information Service
    • Two or three others are under discussion or development
  • The general health of the program is adjudged to be "good" other than a partially cautionary notation in software development due to reduction in staffing.

Discussion:

  • Committee members expressed concern about the loss of netCDF development time with the departure of netCDF developer
  • There was a suggestion that Unidata should attempt to establish an endowment for itself.

Budget Report (Graham)

The $400K gap in funding that was anticipated in last fall's report has been greatly reduced due to a number of factors that include staff attrition and lower than anticipated expenditures in non-labor categories. Additionally, funds for Russ Rew to chair the IOOS/DMAC subcommittee will bring in fairly substantial funding this year which will defray some costs from our core. As things stand now our budget picture looks relatively comfortable through the end of FY07. If current projections come to fruition, however, we can expect to spend down our reserves by the end of FY08, going into FY09 (the new proposal period) with a $900K gap between anticipated expenditures (given current staffing load in and out of the core) and current funding in the core. Joanne and Mohan do not feel too worried about this given there is plenty of time to plan to write new proposals, which can help to leverage our resources and ultimately help fill that gap. This was all qualified with: the budget is dynamic and this is what is known right now; there is continued uncertainty about additional funding and staffing levels; updates will be provided at future meetings..

Discussion

  • Budget picture looks better since the fall 05. Although this can be attributed in some measure to staff attrition, the committee expressed concern about the increased burden, especially on netCDF with the decrease of Ed Hartnett’s time from the Unidata payroll. The committee wondered who will pick up that work.
  • There's an increased proposal success rate from proposals written by senior staff, though on most (2/3) of the proposals collaborators from outside Unidata take the lead role.
  • Although the proposal success rate is good, it diminishes staff development time, and in addition while staff attrition makes the budget picture rosier, there is the ancillary lost work issue.
  • International collaborations enhance UPC's prestige though in general they are harder to get funded.

Users Committee Report (Steenburgh)

In its last meeting, the Users committee discussed how to engage the community in the IDV's development. NCAR's CCSM was cited as a good example of community involvement in tool development. Users committee members will review the previous two or three community e-letters and be ready to discuss how better to engage the community in the use of this communication tool. Committee members continue to contact individual sites for feedback on the program and the community.

The committee's discussion of the USPLN data led to a resolution (below) which now needs to be reviewed by the Policy Committee.

Discussion

  • Is significant development for McIDAS and GEMPAK occurring in the community?
  • Is the path for contributing IDV development code obvious? If not, it should be made obvious.
  • Users are usually reluctant to change the packages they are currently using.. However, the hope is that the community will catch up.
  • Word of mouth seems to be the best means of diffusion.
  • Forums and a wikipedia article were suggested as two means for disseminating information about the IDV,
  • The lightning data policy will be reviewed by the UPC and Policy Committee.
  • It's possible that input from the UCAR General Counsel will be required.

The Users Committee made the following resolution regarding the Task Force Report on Unidata's Analysis and Visualization Applications at its April 10-11 Meeting

Resolution: The Users Committee supports the recommendation of the Report of the Task Force on the Future of Unidata Analysis and Visualization Applications to continue to maintain a suite of analysis and visualization packages to meet the needs of the community. To accelerate IDV development we recommend that the Unidata Program Center aggressively seek external funds and examine options to streamline IDV support and increase programmer development time.

Agency Reports

NSF (Jacobs).

The President's "American competiveness" initiative recommends doubling the NSF budget over the next ten years.

The agency's FY07 budget request is 6.02B an increase of 7.9% over 2006. Here is a breakdown:

  • Advancing research at the frontiers: 4.7B
    • IPY: 13.5M
  • Providing world-class facilities and infrastructure: 1.7B
  • ARRV: 56.0M
  • OOI: 13.5M

Cyberinfrastructure is targeted for 597M. Since a November 05 meeting NSF's office of cyberinfrastructure (OCI) has selected a director, Dan Atkins, identified a chair for its advisory committee, Jim Duderstadt, which will meet in June 06. The CI vision document which will be completed at that meeting and released in the summer of 2006, describes:

  • Virtual organizations with common technological frameworks
    • Operations
    • Evaluation and assessment

Learning and workforce development comprises developing and deploying CI-enabled learning and research environments and developing new skills and professions to support CI-enabled opportunities

A panel for the on-site review of NCAR management comprised 8 members representing universities and government labs. The panel evaluated the quality and effectiveness of NCAR's performance of an NSF funded R&D center based on the following points:

  • Facilitating a strategic partnership w/ NSF
  • Promoting and sustaining cross-divisional and interdisciplinary programs
  • Establishing productive national and international links with agencies, institutions, GSOs, and the global research community
  • Facilitating participation by underrepresented groups

Governing principles in the competition to manage NCAR are: minimize disincentives for potential applicants, demonstrate openness to new ideas, and explore alternative models for NCAR M&O. Note that it is NCAR management that is being competed, not UCAR.

NASA (Herring)

NOTE: this was David's first meeting as NASA representative to the committee. He was introduced on the first day; his presentation was made on the second day. David is with NASA-Goddard's Earth-Sun Exploration Division whose specialty is science and communication.

Herring reported that NASA's Earth Science Education and Public Outreach group seeks to provide information about NASA's missions, new science results and satellite data products to a wide range of target audiences, including decision makers and public stakeholders, educators, students, informal education institutions, public media, and the science attentive public. Common messages to each audience are that NASA conducts Earth-system science, it collects and shares unique data sets, and it drives new technology development.

Herring reported on a new image and data gateway under development called NASA Earth Observations (or NEO, at http://neo.sci.gsfc.nasa.gov). This new image data resource seeks to significantly increase the demand for NASA remote sensing data while dramatically simplifying public access to georeferenced imagery. NEO is designed for relatively unsophisticated and non-traditional data user communities. With NEO as an image data resource, and the open-source ICE Tool (icetool.nasa.gov) as a Web-based analytic tool, Herring hopes to help build a "citizen science" community of amateur Earth observers (analogous to amateur astromers).

NWS (Spayd)

An Integrated Upper-air Observing System (IUOS) is a complete network of all upper-air systems available in NOAA whose mission is to cost-effectively meet existing NOAA upper-air observation requirements as well as future validated requirements. At this point plans are for IUOS to expand from 1700 aircraft to 2000 by 2008. It will eliminate redundancies between radiosondes and aircraft observations starting in 2008. The Profiler network will be expanded and refurbished.To achieve this goal a phased strategy will be implemented that includes: integrating insitu regional soundings, radar and adaptive observations and future satellite obs. The ultimate goal is to have data management functionally streamlined.

The Integrated Ocean Observing System (IOOS) is a NOAA major project. Funds for the its global component have been earmarked, and the system is 55% complete. Among its responsibilities is development of the IOOS Data Management and Communications (DMAC) system.

In the near future, NWS's Digital Services will include enhancement elements for Alaska.

Budget-wise, NWS has a budget deficit. Among mitigation measures are a 10-15 percent labor reduction at HQ and a 3% reduction in the field. FY07 will also see labor reductions amounting to 491M, but there will be increases for buoys, tsunamis, and facilities. There will be unfunded pay raises, earmarks, and recissions.

The Future of Unidata Software and Applications (Rew)

A document on the Future of Unidata Analysis and Visualization report was prepared by a task force that included developers representing each of the analysis and visualization packages. It was initiated in response to Action Item 1 from the Policy Committee's September 2005 meeting: Review the IDV's current development status including its long-range plans and how this affects the long-term plans for GEMPAK and McIDAS.

Important points in the report:

  • The IDV was not meant to replace all other packages.
  • GEMPAK is very popular among its constituents, and McIDAS still has its place.
    • NCEP develops the decoders for GEMPAK. IDV and McIDAS both use ADDE.
  • Redirecting resources to IDV does not make sense.

IDV - support is cutting into development time; the user guide updates are instrumental but time consuming.

The task force's recommendation was to continue the status quo for the short term.

Discussion

  • Rather than replacing GEMPAK and McIDAS, at this time, the IDV's capabilities should complement them.
  • The community would suffer without GEMPAK.
  • A course for potential IDV community developers during the 2007 Training Workshops would give users the opportunity to acquire skill in how to contribute to its development

Unidata's Near-term Goals and Priorities (Ramamurthy)

To establish Unidata's near-term goals and priorities, (i.e., until the end of the 2008 proposal period), each Unidata management group created a document detailing their views. (The groups are: Data access and infrastructure; THREDDS/CDM Project; Data Transport; Analysis and visualization tools; Community building; Unidata services; Unidata web site; LEAD-UPC; Support services; Sys Admin.)

These documents not only detailed the goals, but included alignment and priorities. The ultimate goal of course is for all management groups to be on the same page at the end of the current 2008 proposal period, 30 September 2007.

The IDV and THREDDS documents contain parallel goals, so that each can leverage from the work of the other. There's a concern for the LEAD project and whether or not it will be able to meet its commitments minus one FTE.

In the Data Services area, an LDM will be deployed in support of the IPY. Does this require decoders?

The program center will seek to determine whether there are synergies among priorities so that existing resources can be leveraged effectively.

As creation of the vision document and the strategic plan go forward, constant revision of the near-term priorities documents will occur.

Discussion

  • Milestones: accomplishments on road to goals and priorities.

Developing Unidata's Vision

Unidata's vision will be influenced by community need, its core competencies and capabilities, and factors external to the program. Both Unidata's vision and mission must be congruent with NSF's CI vision and mission, and they must, in addition, be consistent with NSF's overall goals. Unidata will continue to provide the services to its core community has come to rely upon, while reaching out to a broader community which is consistent with the progress of the 2008 proposal..

Other factors that will influence Unidata's mission and vision (and ultimately the Strategic Plan) include the data deluge, community broadening, the evolution of service oriented architectures, data and web services development.

Web services dramatically increase the possibilities of integrating disparate applications. This evolution validates the UPC's development of same during the proposal period.

Discussion

  • Shrink vision statement to a sentence or two; those proposed seemed too long from the committee's point of view.
  • The committee recommends that we have a completed vision by the fall meeting along with about six goals that can be shared with the entire community,
    • The vision and direction statements are not so different
    • The goals should encompass a course for the IDV and other packages and tools
  • The UPC should meet the core competencies as mentioned in the presentation, and these should defined
  • The vision must align with the organization's capabilities
  • The vision and the projected direction for the UPC must also align
  • The directions in the Unidata community and the Program Center's vision have to line up.
  • The strategy for implementation which is not entirely commensurate with the program plan should be developed next after the vision and goals are established; the "How" comes still later.

Unidata Assessment: Progress Report (Sarah Nelson)

The presentation by Nelson Consulting, the group contracted by the UPC to conduct the assessment, described the process used to develop the Unidata metric. First, a survey has been developed that will be deployed to the community ~April 17. Next will be software interviews which will examine Unidata's software engineering practices. A representative sample, comprising netCDF, GEMPAK, and IDV will be examined and staff interviews will occur in May. Next, focus groups will be convened during lunch at the Users Workshop in July, and case studies will be developed during August. The final report is due during December 2006.

UOP Strategic Plan

The plan's structure and contents seemed appropriate from the committee's perspective.

Discussion

  • There was an observation that UOP's programs seem to come and go, but really there haven't been that many that have left the UOP (e.g., INO, UNAVCO, SOS). There are several acknowledged reasons for this:
    • Maturity of the program
    • Loss of funding
    • Transformation
  • Unidata's direction and its overlap with UOP's values, vision and mission, and goals and objectives were discussed.

Tuesday April 18

Web Services for Weather Data Products (Maidment)

Among CUAHSI's Hydrologic Information Systems (HIS) goals is better access to large volumes of high-quality hydrologic data. Water data comes in six forms: water quantity and quality, soil water, rainfall and snow, remote sensing, meteorological, and modeling data. At this time, HIS has a collection of WaterOneFlow Web Services, "which enables HIS users to programmatically access data, tools and models residing on different servers. [David's slide 20] Data arrive from or through federal agencies, Unidata, and universities. (Motherlode is accessed through THREDDS.) Upon delivery the data can be analyzed, visualized, and stored in a variety of programs

David mentioned several surprises and revelations as he has worked with netCDF and Motherlode, so it seems that collaboration between Unidata and CUAHSI to develop a consistent vocabulary is a worthy goal for both. CUAHSI would like to be able to ingest real-time weather information as fields and to define services that represent time-sensitive variables, precipitation, evaporation, and surface temps among them. Defining services should be part of the collaborative effort.

Discussion

  • In furtherance of the collaboration between HIS and Unidata, the THREDDS Data Server is being used to permit CUAHSI web services to extract data subsets in standard netCDF form.
  • It was pointed out that NcML is a way to represent netCDF data and metadata in XML.
  • Using one of several commercially available software extension packages (NetCDF Toolbox for MATLAB-5, NCX, MexEPS among others) MATLAB and netCDF combine to make a useful tool for the collaboration.

Strategic Planning (Continuation of day one discussion)

UCAR's strategic plan will be available sometime later in the calendar year. It's projected to be relatively short, and its format will be less formal than in previous years. Unidata is urged to consider a similar format. The Colorado School of Mines Plan was also cited as a good prototype; in particular its Goals area which is one page. In addition, COMET has adopted a shorter, glossier, more colorful format for its strategic plan.

Unidata should consider the plan's, purpose (or message), and its intended audience: is it for staff? our sponsor? or others?. The plan must have verbiage that is consistent with that of NSF-GEO and cyberinfrastructure plans. Other considerations:

  • Is this a marketing tool or is it written for proposal reviewers
  • Will the proposal be based on the strategic plan
  • Is it to justify Unidata's existence
  • Should it be less text intense than in the past
  • What is Unidata's role in facilitating mesoscale prediction

The how and the what need to be defined first. Unidata's role as an enabler and a facilitator should be emphasized.

Other considerations mentioned:

  • Relationship to NASA EOS
  • The data from the Assessment will inform the strategic plan
  • The emphasis on interdisciplinary activity should be highlighted
  • The goals should be stretched to include considerations about diversity
  • Transformational issues must inform it

While there was agreement that the vision should be one or, at most, two sentences, the mission statement should also be concise and relatively short.

Developing Sister University Contacts (Wetzel)

This concept was offered as a new framework for interaction among Unidata participants. There are many long-term benefits that include data sharing and advancing the use of Unidata software. It would also provide a means for integrating the international component for proposals.

Committee members were asked to consider strategies to promote the collaborations for a discussion at the fall meeting. (See Action Item 4.)

Discussion

  • Similar collaborations are mentioned in COMET's strategic plan
  • Data exchange is important and is only one piece of the overall intellectual exchange
  • UNESCO and the foreign agricultural service may be useful in structuring collaborations

Adjourn

The meeting adjourned at 11:30.

Action Items:

1. The Policy Committee will review the planned agreement for the USPLN lightning dataset and make recommendations with respect to the Unidata Users agreement and Unidata community needs by 15 May. [NOTE: active discussion of this issue is underway and ongoing.]

2. The Policy Committee suggests that the UPC consider offering an advanced IDV workshop during the 2007 training workshop sessions that would include training in IDV software development.

3. The Policy Committee requests that a full half day be devoted to Unidata Strategic plan development to include input from committee members at the fall meeting.

4. M. Wetzel will work with the UPC to follow up on the meeting discussion on possible strategies to promote sister-university type collaborations that advance the use of Unidata software and datastreams. The concept was introduced as a "Unidata Network of International Teams (UNIT)", through which Unidata could help current active participants to identify procedures and benefits for establishing academic and research partnerships with universities in other countries for the purpose of sharing meteorological data and related teaching activities. A Unidata e-letter item and committee feedback by e-mail will be used during the next few months to extend this discussion, and a summary will be presented at the fall Polcom meeting.