Hi Mike,
On 10/28/2015 10:02 AM, Michael Dross wrote:
Is there one type of filesystem that is more efficient at handling
these types file large volumes of finds/creations/deletions?
ie. EXT4, zfs.
We (Unidata Program Center) have had good luck with ZFS on our
Solaris 10 motherlode-class machines.
It would be most interesting to hear other users experiences!
Cheers,
Tom
On Oct 28, 2015, at 11:56 AM, Tom Yoksas <yoksas@xxxxxxxx> wrote:
All,
re: how to efficiently scour LDM-created data products from disk
If one organizes one's data files into daily directories, the need to
'find' each file that should be scoured can be pared down to finding
which directories+contents need to be scoured. This does not, however,
get around the fact that deleting large numbers of files
(e.g., millions for NEXRAD Level III files kept on disk for long
periods of time) is a _very_ costly endeavor.
Cheers,
Tom
On 10/28/2015 09:47 AM, Gerry Creager - NOAA Affiliate wrote:
How often to you run your scour command? We've exercised scour pretty
hard and were able to watch it keep up, if it were run frequently.
There's a one-time penalty, if you've been running it, say, daily or
less frequently, but if you run it hourly, it should keep up.
gerry
On Wed, Oct 28, 2015 at 10:39 AM, Steve Emmerson <emmerson@xxxxxxxx
<mailto:emmerson@xxxxxxxx>> wrote:
LDMers,
On Wed, Oct 28, 2015 at 9:28 AM, Donna Cote <d-cote@xxxxxxxx
<mailto:d-cote@xxxxxxxx>> wrote:
I would like to ask for a more io-friendly scour utility.
It's on my list (see <https://github.com/Unidata/LDM/issues/27>.
Can't say when I'll get around to it, though.
Regards,
Steve Emmerson
_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx <mailto:ldm-users@xxxxxxxxxxxxxxxx>
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
--
Gerry Creager
NSSL/CIMMS
405.325.6371
++++++++++++++++++++++
“Big whorls have little whorls,
That feed on their velocity;
And little whorls have lesser whorls,
And so on to viscosity.”
Lewis Fry Richardson (1881-1953)
_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
--
+----------------------------------------------------------------------+
* Tom Yoksas UCAR Unidata Program *
* (303) 497-8642 (last resort) P.O. Box 3000 *
* yoksas@xxxxxxxx Boulder, CO 80307 *
* Unidata WWW Service http://www.unidata.ucar.edu/ *
+----------------------------------------------------------------------+
_______________________________________________
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
--
+----------------------------------------------------------------------+
* Tom Yoksas UCAR Unidata Program *
* (303) 497-8642 (last resort) P.O. Box 3000 *
* yoksas@xxxxxxxx Boulder, CO 80307 *
* Unidata WWW Service http://www.unidata.ucar.edu/ *
+----------------------------------------------------------------------+
