[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20050113: dcmetr... input file length limit?

Mike,

Since the decoders read from STDIN (eg the piped or redirected file input),
they don't see the file on disk, so are not effected by input file size.

However, the decoders such as dcmetr have a maximum bulletin length, which
is the maximum length of a product  between the ZCZC and NNNN delimeters.
For dcmetr, this limit is 100K (eg DCMXBF  = 102400 ). You will need to
create bulletin delimeters which signal to the decoder that the current
product has been read, and can be decoded, after which the decoder will search
for the next product ZCZC/NNNN pair.

Steve Chiswell
Unidata User Support

On Thu, 13 Jan 2005, Mike Hardiman wrote:

> Hi Folks,
>
> We recently acquired the Surface Archives CDs from Weather Graphics
> Tecnhologies down here for use in a local research project.
>
> Since I'd like to create a few billion surface plots, I need to get the
> data into a gempak-friendly format.
>
> So I wrote a script to read an input "laundry list" then dig into the
> archive CDs, pull out the desired data, unzip it, reformat it with the
> proper AFOS headers that the GEMPAK decoders so love, spit it out in
> text files for each day/hour requested, and then create a GEMPAK
> surface file for each day/hour using dcmetr.
>
> However, I noticed it was only producing the GEMPAK surface files for a
> few scattered days and hours.
>
> Lo and behold, there was plenty of data in the text files that weren't
> being decoded with dcmetr.
>
> So I tried it manually... dcmetr fails.  Log just says it did not
> decode any bulletins.
>
> If I manually chomp out a huge chunk of obs from the files, they decode
> just fine.... so this leads me to the following conclusion:  There must
> be a input file length-limit in dcmetr.  Once it reaches this limit, it
> reads no further (and therefore doesn't see the "NNNN" it needs to work
> properly) and therefore fails.
>
> Can anyone confirm my fears and/or suggest a solution?
>
> If you need more info, like sample files, log file output, etc, please
> let me know and I'll post them on the web.
>
> Thanks much!
>
> -Mike Hardiman
> NWS Santa Teresa, NM
>
>
>