Earlier this year, the Unidata Users Committee asked members of the Unidata community to participate in a survey regarding their use of scientific software packages, software training, and community services, and to favor us with their insights into possible future directions for the program. You can read an overview of the survey results in 2016 Community Survey Results.
While Unidata's governing committees and the Unidata Program Center staff will continue to analyze the survey comments in the process of crafting Unidata's next Strategic Plan, individual Program Center development groups are also using the survey as input into their own development plans. This series of articles provides responses from different development groups to comments or concerns raised by the survey. Under review in this article: Python activities at Unidata.
Background: Unidata's Python Activities
The Python programming language has exploded in popularity among geoscientists since the previous survey (which did not address it specifically). More than 60 percent of those answering questions about Python said they use the language in their scientific work, although only a small percentage had tried Unidata's Python packages MetPy or Siphon. An overwhelming majority of respondents (99 percent) encouraged Unidata to continue creating Python resources geared toward the geoscience community.
Unidata's most recent funding proposal to the National Science Foundation (awarded in 2013) identifies improving Python support across the Unidata software suite as an activity critical to Unidata's core effort. As a result, Program Center development staff have been working to augment and improve Unidata's Python software, training, and support resources.
Comments from the Survey
Survey respondents were almost uniformly supportive of Unidata's efforts to support and encourage the use of Python in the atmospheric sciences, with more than 99% saying they thought Unidata should continue its Python activities. Comments about community use of Python and Unidata's role fell into the following broad categories:
Need for additional training resources
Numerous respondents noted that while they were interested in using Python in their scientific work, replacing existing workflows requires spending time learning how to use new tools. Many suggested that in-person training, online training, and software documentation tailored to their needs would be helpful in making the transition from older technologies to workflows based on Python.
Need for Python packages tailored to the needs of atmospheric scientists
Many respondents commented that while Python is generally useful for scientific programming tasks, existing tools such as GEMPAK, AWIPS II, or the IDV provide ready-made features useful to atmospheric scientists. Providing Python solutions for common atmospheric science tasks (such as creating a skew-t plot, for example) emerged as a very common request, with many respondents specifically looking for Python implementations of GEMPAK functionality.
Interestingly, while nearly half the respondents had heard of Unidata's two homegrown Python modules aimed at meeting this need (MetPy and Siphon), very few had tried either package.
Unidata's Plans
The efforts of Unidata's Python development group fall into two main categories: Training/Education and Software Development.
Training/Education
Unidata software developers have been offering a multi-day session on the use of Python with other Unidata technologies as part of the Program Center's annual software training workshops since 2013. In addition to covering specifics of using Python to data in netCDF format or communicate with THREDDS Data Servers, the Python workshops provide a basic understanding of the Python scientific ecosystem, Python coding fundamentals, and code development best practices. The workshops have been well-attended and those attending have given favorable reviews. For information about this year's workshop, see 2016 Unidata Training Workshop.
In addition to the annual software training workshops, in 2016 Unidata received funding from the National Science Foundation for a pilot project to create online training resources focused on Python. This project is in its beginning stages, but plans are to make the first of these resources available by mid-summer 2016.
Software Development
Unidata software developers are spearheading or contributing to development of several Python projects.
- MetPy is a collection of tools in Python for reading, visualizing, and performing calculations with weather data. While MetPy is in its infancy, it is envisioned as eventually comprising much of the functionality of GEMPAK.
- Siphon is a collection of Python utilities for downloading data from Unidata data technologies, including the THREDDS Data Server.
- The AWIPS II Python Data Access Framework is a project aimed at providing access to an AWIPS II Environmental Data EXchange (EDEX) server directly from Python code.
- Unidata Program Center staff assist author Jeff Whitaker in maintaining the netcdf4-python module, which is a Python interface to the netCDF C library. The module can read and write files in both the netCDF 4 and netCDF 3 formats, and can create files that are readable by HDF5 clients.
All of the Python projects Unidata Program Center staff contribute to are open source and available on Github. If you have questions or would like to contribute to any of these development projects, please contact support-python@unidata.ucar.edu or open a Github issue.
Unidata also maintains an e-mail list for those interested in Unidata's python technologies. To join the list, visit Unidata's Mailing Lists page and subscribe to the python-users list.
