Hello
I'm working on a 1D water column model using the F90 NetCDF interface.
Recently I've introduced a namelist option to specify when nf90_sync()
is called. We used to call nf90_sync() everytime data was saved.
Now we can specify every n'th saving will also do syncing. Setting
sync_out=0 will leave everything to the operating system. From an annual
run with with 110 vertical levels the result is (data are saved once pr
day):
sync_out = 0
kb@orca /data/kb/gotm-cases/nns_annual $ time gotm
:
real 0m1.045s
user 0m0.972s
sys 0m0.028s
sync_out = 1
kb@orca /data/kb/gotm-cases/nns_annual $ time gotm
:
real 0m12.567s
user 0m1.288s
sys 0m0.052s
For a single run this might not be very important - but when doing
ensemble and/or parameter estimations with possible thousends of
simulations is does make a difference.
The only downside I see is if the model blows up the latest saved
results will not be on disk making debugging a bit harder.
So unless you you have an external program reading the .nc file my
suggestion is to leave it to the operating system.
Karsten
On Wed, Feb 20, 2013 at 11:00:59 -0600, Leon Maurer wrote:
> I've got a -- possibly ill posed -- question about the NF90_SYNC command:
> does calling it append new data to the already existing file (in the same
> way that a few more 'write(1,*)'s would) and modify a few existing things,
> or does it rewrite the file from scratch? Does it do any/much reading in
> the process?
>
> I'm using a netCDF-4/HDF5 file with compression.
>
> I'm using HTCondor <http://research.cs.wisc.edu/htcondor/>'s standard
> universe, which effectively captures all file input/output and redirects it
> over a network. Although my code only produces ~100MB of new data per day,
> I'm told that the resulting network usage is dangerously high. The program
> calls NF90_SYNC every 5 minutes, so I'm wondering if that could be
> the culprit. (i.e. if it's doing a lot of reading/writing in the process.)
>
> (Also, I just want to confirm that the NF90_SHARE flag doesn't work for
> netCDF-4/HDF5 files. The NF90_SYNC
> page<http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-f90/NF90_005fSYNC.html>
> recommends using
> the flag, but doesn't mention netCDF-4/HDF5file incompatibility; mentioning
> the incompatibility is saved for the NF90_CREATE
> page<http://www.unidata.ucar.edu/software/netcdf/docs/netcdf-f90/NF90_005fCREATE.html>
> .)
>
> Thanks.
> -Leon
> _______________________________________________
> netcdfgroup mailing list
> netcdfgroup@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
--
----------------------------------------------------------------------
Karsten Bolding Bolding & Burchard ApS
Strandgyden 25 Phone: +45 64422058
DK-5466 Asperup Fax: +45 64422068
Denmark Email: karsten@xxxxxxxxxxxxxxxxxxxx
http://www.findvej.dk/Strandgyden25,5466,11,3
"Anti-intellectualism has been a constant thread winding its way
through our political and cultural life, nurtured by the false notion
that democracy means that my ignorance is just as good as your
knowledge." -- Isaac Asimov
----------------------------------------------------------------------
