Unidata Users Committee Meeting Summary

October 4-5, 2007
Boulder , Colorado

 

Members	UPC Staff
Gary Lackmann, North Carolina State Univ, Chair Sean Arms, Univ of Oklahoma -remote Elen Cutrim, Western Michigan Univ Bill Gallus, Iowa State Univ Chris Herbster, Embry Riddle Aeronautical Univ Leigh Orf, Central Michigan Univ Scott Rochette, Suny College at Brockport Tom Whittaker, SSEC - Univ. of Wisconsin-Madison	Tom Baltzer Tina Campbell Steve Chiswell Ethan Davis Ben Domenico Steve Emmerson Ginger Emery Jo Hansen Yuan Ho Linda Miller Terry Mitchell Jeff McWhirter Don Murray Jennifer Oxelson Mohan Ramamurthy Russ Rew Jeff Weber Anne Wilson Tom Yoksas

Please Note:  Proposed Spring Meeting-10-11 April 2008

Meeting Agenda http://www.unidata.ucar.edu/committees/usercom/2007OctMtg/agenda.10.07.html

Action Items from 17-18 May 2007 Meeting  http://www.unidata.ucar.edu/committees/usercom/action.0507.html

Thursday, 4 October 2007

Administrative Items:

• Welcome to Bill Gallus, Iowa State, new member
• Sean Arms joined us remotely via conference call

Outstanding action items were covered with the following results:

• The DeSouza Award will be presented at the April meeting.  The awardee will be asked to provide a brief seminar which will be Webcast to the community on activities being done using Unidata tools and data.
• Two THREDDS focal points were appointed – Leigh Orf (the the assistance of Marty Baxter) and Sean Arms

ACTION 1 : an E-letter item describing THREDDS Data Servers (TDS) and THREDDS Data Repository (TDR) will be written by Sean Arms, as soon as he is able to populate the TDS and has the chance to learn more about the TDR.

Another regional workshop is being planned at Plymouth State, May 2008.  This workshop was an element of an equipment award-winning proposal from Brendon Hoch.  The regional workshop will focus on IDV training.  The question of broadening the focus of the workshops to include other Unidata tools and software was discussed.  It is reasonable to ask if regional workshop hosts would like other tools included, but the hosting site should make the final decision.

Policy Committee Report - Lackmann

The next Policy Committee meeting will take place 29-30 October 2007.  AWIPS2 was discussed with focus on the incompatibility between AWIPS and the NCEP centers.  AWIPS2 is supposed to represent a unified package of open source software to be used at the national centers, as well as forecast offices. The Unidata Proposal to the NSF is due during the Feb-Mar 2008 timeframe. Some discussion during the Polcomm meeting was a Windows based LDM lite, along with integration of climate data. Break out sessions were convened focusing on the six strategic plan areas. The Unidata COSMIC data survey was an action item from the Policy Committee. The discussion is summarized below.

Discussion:

Comments about the use of AWIPS in the classrooms rendered the following comments:

• students benefit by exposure to various software
• some students are interns at NWS so the students would like to use AWIPS
• the use of AWIPS would provide a competitive edge for jobs at NWS
• training students for NWS coudl be a disservice if not exposed to AWIPS2 software
• AWIPS could provide a natural progression moving research to operations
• training on use of AWIPS would be very important
• the question of the number of jobs available at NWS was raised

Director's Report – Ramamurthy

Some highlights taken from the report:

The data flow statistics now indicate that Unidata is serving 450 hosts in 238 unique domains running LDM.  TIGGE data volume is 12-13 GB/hr with bursts to 23 GB/hr—10 operational centers worldwide will be contributing to the TIGGE archive.

Unidata has been working to make COSMIC data available to the community using LDM technology.  The survey results were positive toward distributing the data via LDM

Mohan and Linda attended meetings about a NOAAPORT backup.   The NWS requested a white paper that was submitted, and a response was recently received from the NWS. Additional discussions need to take place.  The CRAFT/Level II project could be a poster child for working together to make other datasets available.  CONDUIT is a perfect example of collaboration that continues to work. 

Unidata will coordinate a meeting with NWS and NCEP at the AMS Annual Meeting in New Orleans to discuss CONDUIT and data distribution.

IDV is experiencing steady growth (approximately 100 sites who have used the IDV during the last month)  with new capabilities (charting, data saved into bundle, experimental remote level II data chooser).  The IDV for developers workshop in August attracted 14 attendees.

GEMPAK continues to grow with 1735 unique users.  NWS/NCEP is migrating NAWIPS  to AWIPS2 with a Service Oriented Architecture.  Unidata is working with NCEP and NWS to meet the challenges of GEMPAK and AWIPS2. 

McIDAS-X usage is decreasing, i.e., 2004 there were 104 downloads, 60 downloads in 2005, and 50 downloads in 2006.

NetCDF4 provides full backward compatibility and allows users to read and write HDF files with the NetCDF API.—38 people participated in the netCDF training workshop.  Shanna Shaye Forbes, SOARS student, created a new C++ API.

Training workshops-75 people attended which included several international participants.

A Regional Workshop was held at University of Oklahoma with 25 participants with the focus of IDV and additional presentations on THREDDS and LEAD.

Community Equipment Awards - Funding of six equipment awards were granted with the focus of broadening participation and entraining users in IDV.  Per Users Committee recommendation, we followed up on the idea of expanding the budget base to other NSF Directorates to cover the equipment awards funding.  This will take some time, but we will continue working through ATM to expand to other Directorates.

The Open Geospatial Consortium (OGC) Interoperability Workshop in September, organized by Ben Domenico, was well attended by 63 people.  It was one day out of a five-day workshop that included GALEON (Geo-interface for Air, Land, Earth, Oceans Netcdf)  and GEOSS sessions.   

AGU and EGU - Earth and Space Science Informatics (ESSI) programs are now sanctioned at AGU and EGU. 

LEAD - 10 universities used the LEAD system to make WRF model runs available in preparing forecasts for the Weather Challenge and NOAA Hazardous Weather Test Bed. LEAD PIs met with NSF principals to discuss LEAD's future beyond the current 5-year proposal period.

Status Reports

Community Services
Equipment Awards
IDV Development
GEMPAK
LDM-IDD
Next Generation Case Studies
Decoders
International Activities
LEAD
McIDAS
NetCDF
OPeNDAP
Web Development
NEXRAD-Hydrology ITR Project
Support
Support by Project
THREDDS
GALEON

Discussion points from Status Reports:

• IDV – new member of the IDV steering committee-Marty Baxter, Central Michigan. There is no student representative yet.
• The Users Committee and the equipment awards panel should begin thinking in broader terms to try to embrace diversity and under represented universities for the equipment awards when possible.
• IDV-GEMPAK - some people are not using IDV due to file format conversion issue.
  Grids might be done, but THREDDS middleware is missing for support of other data.
• The entire community would benefit by converters and decoders.
  GEMPAK file - users can write a C program to take it from GEMPAK and save to netCDF
• Case studies - grids are available through NCDC, but there is no support in the THREDDS catalog for the grids at this time.
• Question of NNTP for LDM development-the new LDM development will borrow concepts from the research done on NNTP.
• The question of resources being used internationally was discussed briefly. The Unidata community is benefiting from international collaborations through sharing of data sets. The MADIS database is augmented by datasets from Brazil. TIGGE will be providing additional data as the project progresses.
• Not a lot of new updates to McIDAS – remember that ADDE servers are providing data to IDV. There is a lot of interest in McIDAS from the international community.

Student Survey and Site Contact Reports - Sean Arms conducted a student survey with a total 127 respondents. The issue of how to deal with the input was raised. Sean will be considering that for the next meeting. There is a need to evaluate the results and determine what’s next. It was suggested that we establish a Student Section in E-letter to provide information on how students are using Unidata technologies. This could be done in the form of setting up a forum for students or a student email list. Online training for freshman and sophomores was discussed, but Unidata developers already provide tutorials on line for training workshop. Perhaps a "Tips of the Week" for each package could be considered. Another idea was selecting a " region of interest" to learn by example. No definite action item was determined.

Site Contacts -Some committee members did not make their contacts, or are still waiting to gather input. Each member will use the Google.docs to add information gleaned from the personal site contact.

Workshop Forum discussion - how can we reinvigorate the Forum discussions from the 2006 workshop?
Should we try to get BAMS to publish the data? Perhaps thinking beyond the scope of the theme of the workshop?

ACTION 2 : Committee members need to mention the Workshop Forum when making site contacts. This will help to faciliate interaction and discussion among Forum participants.
ACTION 3: Gary Lackmann will contact John Snow regarding the point-counterpoint article for BAMS.
ACTION 4: Please provide a graphic, description, etc for the summer 2006 workshop publication in the Forum.
ACTION 5: Committee members divided the names of workshop presenters to get presentations and data used during the 2006 summer workshop. A thumbnail picture to go along with the data would be beneficial. Addition of tutorials and/or lesson plans would be useful.

Discussion:

• Post your lessons - could this lead to fast track publication opportunity?
• A new topic for consideration on the forum—what has happened 18 months later?

FaceBook - Sean Arms

Sean provided a remote presentation about FaceBook. It is a thought provoking framework that is flexible and easy to use.  Some comments included:

• What’s in it for us?
• How do we bring the community along to embrace the new methods
• We need to figure out what the community would want

ACTION 6:  Sean can provide a demo using FaceBook as a scientific demo, and UserComm should get an account.

Regional Workshop Discussion - all

The workshop was hosted by the University of Oklahoma with 25 participants. The workshop focused on IDV training, with a short presentation by Don Murray of THREDDS, and a Unidata community seminar provided by Mohan. Should there be some promotion of other tools, i.e. LDM, GEMPAK, THREDDS? The regional workshops have appeal because the students do not have to pay to come to Boulder for the Users training workshop. Suggestion of entraining the workshop participants to present what they have used and what they are doing with Unidata tools for future workshops. Brendon Hoch, Plymouth State, is going to host a spring regional workshop.

Strategic Plan and input to Proposal - all

• Six focus areas have been identified
  • Community
  • Data Services
  • Tools
  • Communication and Support
  • Cyberinfrastructure Leadership
  • Diversity

Discussion:

• Unidata started with meteorology and is moving organically into broader areas.
• There are only five UCAR university departments that have "meteorology" in their name - a sign of the times
• Unidata needs to evolve as the community evolves
• Geoscience-geography, oceanography, GIS-integration
  • Question of staffing expertise in other disciplines
• Unidata welcomes collaborations and connections with other disciplines
• Question of being spread too thin—increased support demands
• Additional platform of Windows will be useful to the broader community
• Question of Field Project support (field projects use Windows)
• NSF Facilities-cyberinfrastructure and funding
• LEAD will remain an independent proposal.

LEAD - Ramamurthy, Baltzer and Wilson

• Would be most useful in edu context if it could be utilized for an NWP course. 
• Use of portal to run the model. 
• Educational value could be packaged with a course on how the model works
• Some feel that there is no point in having all students doing model runs at the same time
• Conversely, Rich Clark defends LEAD runs—students do not have to know the nuts and bolts, but they can make intelligent use of the components
• Should there be an undergraduate course that goes with the use of LEAD, but some felt that they do not have the faculty or course time to provide such a course
• Democratizing aspect of LEAD for use in the classroom
• Tom B is deploying LEAD to the community - Unidata would be the “conduit” to bring LEAD to the community.

Questions of problems with LEAD runs.  Previously the runs were about 50% successful. There’s a strong partnership with TeraGrid, and TeraGrid is going to make its part more robust.

Proposal Development-cont

There are several challenges that should be addressed in the proposal:

• Need to stay apprised of dual-polarization and of new developments
• Tera and petascale datasets will be used more in the future-need to consider this for the proposal
• Specific challenges for the proposal-global community model-3km grid - Livermore and PNNL PhDs are working on this
• Highlight the work being done via LEAD with WRF in the proposal
• Climate is an important aspect to highlight for the proposal
  • Users are currently using GrADS, NCAR graphics, IDL
• Visualization of large data sets and the climate aspect for Unidata’s proposal are very important
• Need to consider what kind of functionality we need to add to enable climate science - a suggestion of a letter of support from a university recommending Unidata broaden to include climate.
  • NetCDF has been adopted for model runs by IPCC
  • Standards are being developed and Unidata is in the midst of the activity.
• Changing data access methods-going from 32 to 64 bit
• Multi-dimensional chunking
• Field project support and CI support should be added
• Next generation case studies is very important
  • THREDDS repository and IDV
• Next generation LDM
• Consider GIS and meteorology - bridge the gap between GIS and meteorology for geography
  • GIS world-millions in the user base-work out specific partnerships
• Promoting diversity-Anton suggested having specific actions
• Moving toward Windows and Mac is a good start, due to expanding and broadening to community colleges, etc. 
• Diversity-American Indians outreach involvement-be proactive-designate some of the equipment award money specifically targeted for these institutions.- Elen suggested giving seminars, scholarships, with the need to be very specific
• Work with university sites to facilitate diversity via community colleges-leveraging connections
• Gary suggested Users Committee members from the broader disciplines, because governance committees provide an important avenue for impacting the future agenda
• Perhaps Unidata should hold a workshop targeted toward community colleges and other minority institutions
• Leveraging other efforts that Unidata is involved in—SOARS protégé every summer.  Unidata is already reaching out to the SOARS students. There are other opportunities for Unidata to mentor under science and software. Jeff Weber has given presentations about Unidata to various groups.
• Chris Herbster pointed out that there are 1195 community colleges in the U.S. Unidata can't do everything.
• Tom Y pointed out the impact on staff support time - need to write in the proposal indicating the impact with a request for additional funds
• A concern remains that Unidata continue to support their core constituency.

AWIPS2

This development needs to be monitored closely by the UPC. Depending on schedules, AWIPS2 will impact us in some way during the next proposal period. We may need to make course corrections during the period. The question of the type of open source license by Raytheon remains unanswered. We must track this carefully. We need to leverage off of NWS changes for any transition to AWIPS2. Questions such as porting GEMPAK to Java need to be answered. There was a discussion of a potential letter to NWS regarding the open source license.

ACTION 7: Find out if a decision has been made regarding the open source license. If not, draft a letter to NOAA/NWS legal counsel requesting the LGPL or similar licensing be provided for AWIPS2.

Graphical Representation of Data and Tools - Jeff Weber

Jeff presented the new graphical representation of data and tools available on the Unidata web site. Kudos were given to Jeff for providing this information in such a user friendly way. Several committee members indicated that it would be very useful for explaining the data and tools to students.

Case Study Demo and Discussion - Jeff Weber and Mohan Ramamurthy

Jeff and Mohan led the discussion of the next generation case study project. This project will deliver a new generation case study that incorporates learning modules from COMET and other external sources, and a full suite of atmospheric data delivered via THREDDS catalogs utilizing OPeNDAP and ADDE server technologies and visualized in the Unidata IDV. The project was funded through the UCAR/UOP STORM funds. It combined the efforts of collaborators at UNC-Charlotte. The committee was enthusiastic about the demo, and indicated that it has great potential. The Users Committee can be beta testers by late October or early November. They recommended that it become one of the goals for the five-year proposal.

Miscellaneous

Access Data Workshop - Domenico - April 2008 – if people on the UserComm would like to be involved, the meeting will be in Portland. It is a follow on to DLESE Data Services Workshops. Unidata is hosting the 2008 workshop with the focus on undergraduate education. Contact is Ben-there is a planning meeting in October.

A catalog which would point to other catalogs or other resources for real time data and other discovery metadata will be beneficial. Data Discovery is important, and digital libraries are supposed to solve the problem. The question of what Unidata can realistically do still remains an unknown.

Model data catalog-clearing house for current model runs-ADDE/MotherLode

Suggestion to Users Comittee-Provide a list of suggested reviewers for the Unidata proposal, especially in the Climate area- Mark Chandler and Ruby DeLeung would be good, because of their summer workshop involvement.

NWS-NOAAPORT issue- CONDUIT— send an email to the CONDUIT community

ACTION 8:  Sean will create datapedia entry for WikiPedia

ACTION 9: Move forward toward distribution of COSMIC data via LDM.

ACTION Items from this meeting (October 4-5, 2007).

Linda Miller
Community Services - Unidata
University Corporation for Atmospheric Research
P.O. Box 3000
Boulder, CO 80307-3000
303 497-8646 fax: 303 497-8690