In the summer of 2015, from June 1st to July 15th, a fleet of scientists set out each evening in the Midwest. Their goal? To gain insight on the nocturnal mesoscale convective storms (MCS's) that plague America's heartland. While MCS's are necessary for irrigation and the replenishing of aquifers, often these storms can become vicious; especially at night when they are least detected.
With the implementation of the field experiment PECAN (Plains Elevated Convection at Night) scientists from eight research laboratories and fourteen Universities including Millersville University of Pennsylvania hope to gain insight to better predict these nocturnal storms. After the data were collected, the Unidata Program Center (UPC) worked closely with Millersville students and academic staff to help standardize their PECAN study data.
Some of the data Millersville collected was produced using the rawisode data logging system. This data logging system consists of a Vaisala MW41 data acquisition system using a RS41-SGP radiosonde, attached to a 200 gram totex weather balloon.
The rawinsonde system logs an ASCII text file, which then quickly runs through a short amount of python code within the data logger to create a skew-T profile of the lower to middle atmosphere. The rawinsonde system measures parameters such as temperature, wind direction and velocity, relative humidity, location, and pressure.
Overall, the PECAN study collected data from over 100 instruments. The instrumentation included PECAN Integrated Sounding Arrays (PISA's), which consist of four mobile and two stationary data collecting sites, flux towers, and three aircraft.
In addition to being able to collect a large amount of data quickly, the size of the PECAN project allowed for a large amount of student involvement.
“Through the 1.5 month long journey of PECAN, I was constantly reminded that science is awesome. Nothing I have learned in the classroom evoked the same feelings in me that I experienced while participating in research,” says Millersville undergraduate student Natalie Midzak. “Discovering rewards hidden within the daily struggles associated with fieldwork was gratifying and unforgettable.”
Her classmate Megan McAuliffe reflects a similar feeling, saying “The most rewarding thing from the PECAN project was the opportunity to be part of a huge collaborative effort to study meteorology … I personally grew in terms of my leadership skills and knowledge of instrumentation that Millersville had at our Fixed PISA site in Ellis, Kansas.”
Much of the data that was collected by Millersville students was written in a simple text-based data file, bundled with a metadata READme text file. These files were sent to NCAR's Earth Observing Laboratory (EOL). Once at EOL the text files were formatted for consistency as well as transformed into a CF-compliant (Climate Forecast metadata convention) netCDF file format. UPC staff were able to contribute to this effort by augmenting the Rosetta data transformation tool to work better with the PECAN data.
The Rosetta tool, created by UPC software developer Sean Arms, can transform ASCII-format data files into standard CF-compliant netCDF file format. Storing data in CF-compliant netCDF files makes the data more largely accessible because it can be eaisly served to remote users by a variety of data servers such as the THREDDS Data Server. NetCDF files are also more readily usable with a wide array of analytical software programs such as Matlab, Python, or Unidata's Integrated Data Viewer (IDV). For Millersville's PECAN data, Rosetta was enhanced to standardize flight trajectory data collected from weather balloon launches at a PISA site known as FP3, located in Ellis, Kansas.
Millersville PECAN PI Dr. Richard Clark has also been working with UPC staff to document Millersville's data management workflow as a case study in the Unidata Data Management Resource Center (DMRC). One of the goals of the DMRC is to showcase a variety of data management workflows, in hopes that the experiences of participating research teams can serve as guides to others.
Millersville's PECAN data is stored in EOL's PECAN archive, but it is also backed up on their own local network. To make this local data easier to access and use, the UPC's Jeff Weber worked with Millersville to install and configure a RAMADDA server to locally host their PECAN WRF runs and their derived output data.
The case study for Millersville University's PECAN data management workflow is posted on the DMRC. We hope this case study gives universities grappling with similar data sizes and file types insight into a data management workflow that worked well for Millersville and might work for them.
Thanks to the students and researchers who worked on the PECAN project, we are steps closer to having a clear understanding on how MCS's unfold. With increased insight on the evolution of these storms, many hope better prediction techniques will follow. A link to the official PECAN study webpage can be found here.
