Hi,
I think it's OK to have 10,000+ separate netcdf files if they are searchable
using metadata.
We've created 85,302 CF-compliant netCDF-4/HDF5 files with 2D lat/lon for Terra
Fusion project that spans over 15 years.
See slide 15 of [1] for the details.
[1]
http://www.nationaldataservice.org/get_involved/events/DataScienceTools/pdfs/zhao_terra.pdf
From: netcdfgroup <netcdfgroup-bounces@xxxxxxxxxxxxxxxx> On Behalf Of Daniel
Wright via netcdfgroup
Sent: Friday, July 08, 2022 12:18 PM
To: netcdfgroup@xxxxxxxxxxxxxxxx
Subject: [netcdfgroup] Issue with time variable for Monte Carlo "scenarios"
Hey everyone! New poster here.
I am the creator of a software, which, among other things, generates really
large numbers of gridded precipitation "storm scenarios" in netcdf format,
which can then be used for things like monte carlo simulation to estimate
100-year floods. I have a question I am hoping someone could weigh in on. The
question has real-world significance, because I am working with the US Army
Corps of Engineers Hydrologic Engineering Center to figure out how to get
output from my RainyDay software into their flood simulation software.
Anyway, here's the issue: Because the software generates large numbers (think
10,000+) of rainstorm scenarios, each one has a distinct time (e.g. 72 hours of
hourly precipitation), as well as a 2D (lat/lon) precipitation field for each
time period. Of course it isn't ideal to create 10,000+ separate netcdf files,
one for each scenario. So instead I can create one or several files, which
contain multiple storms. Consider this example: Say a file contains N=500
storms, and each storm is comprised of 72 hours of hourly gridded data, but
each storm has a different 72 hour period. My software currently writes the a
2-dimensional 'time' variable, specifically of size 500x72. This can be done,
but as far as I can tell this violates CF standards.
So the question is basically is there an alternative way that is compliant with
standards like CF, but that can contain a large number of gridded "scenarios"
each with a different time?
Thanks,
Dan
--
Daniel B. Wright
Associate Professor
Civil and Environmental Engineering
University of Wisconsin-Madison
Hydroclimate Extremes Research Group (https://her.cee.wisc.edu/)
