[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
Re: Huge Files
- Subject: Re: Huge Files
- Date: Tue, 23 Nov 2004 15:30:28 -0700
Hi Brian,
With some help from others I've tracked the problem down to a problem
with /home/ldm/util/ucnids. Those very large files are filled not with
data but with this:
ldm@ldm:/mnt/ldmdata/NNEXRAD_NCR/KOAX/20041123> tail
KOAX_SDUS53_NCR_200411231638.raw
Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21
Read: 0 21 1
Cpy: 21
which is a ucnids issue. If you execute this:
strings /home/ldm/util/ucnids
you'll see that the above is ucnids output, i.e., output from this
command includes these lines:
Read: %d %d %d
Cpy: %d
which are the lines that are generating the above. Maybe ucnids is
trying to simultaneously read and write the same file? I would
carefully examine the relevant pqact.conf entries to see if you're
having some sort of a race condition that is causing this.
Please note that this has nothing to do with the upgrade. It must have
been occuring before the upgrade as well.
Are you using WXP? We don't support WXP and haven't for some time.
FWIW, our GEMPAK expert here pointed out that if you're using GEMPAK or
MCIDAS you don't need to uncompress the radar data upon receipt as both
those packages will uncompress for you when you ask them to display the
data.
You could try to debug this problem by running ucnids on a good version
of the problematic data. We have the correct data files available on
our archive - you can go to http://motherlode.ucar.edu/ and get the
correct version of the corrupted files and try running ucnids on them,
which might reveal the problem. If ucnids runs properly on the good
data from the command line then the problem is with interaction across
or within pqact.conf entries.
If this is really a problem for you I may be able to help you after the
holiday. Unfortunately, I can't debug this problem for you now. I was
planning on starting my vacation over an hour ago. For now could you
deal with the problem by turning off the decompression?
Anne
Miller, Brian S wrote:
Anne-
I removed all of the huge files from /home, but there are still a few in
/mnt/ldmdata/NNEXRAD_NCR
In that directory there are 6 radars. Look at the data for today in
each radar folder and you will see what I am talking about.
Thanks again.
Brian
-----Original Message-----
From: Anne Wilson [mailto:address@hidden]
Sent: Tuesday, November 23, 2004 3:25 PM
To: Miller, Brian S
Subject: Re: Huge Files
Hi Brian,
Where are these files? Please respond ASAP if you can.
Anne
Miller, Brian S wrote:
Anne-
Since the upgrade there are random HUGE files that are being created
on
our LDM server. Every now and then instead of having a 1MB file there
is a 2GB file in its place.
Can you take a look at what it is doing and let me know if I can do
anything to stop the "bad" data files? It is filling up my storage
fast.
Brian
--
***************************************************
Anne Wilson UCAR Unidata Program
address@hidden P.O. Box 3000
Boulder, CO 80307
----------------------------------------------------
Unidata WWW server http://my.unidata.ucar.edu/
****************************************************