[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

RE: NOAAPORT receive system feeding LDM (fwd)


===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================

---------- Forwarded message ----------
Date: Mon, 18 Oct 1999 20:11:40 -0400
From: Dan Vietor <address@hidden>
To: 'Jim Koermer' <address@hidden>,
     'Michael W Dross' <address@hidden>
Subject: RE: NOAAPORT receive system feeding LDM

> From Mike Dross:
>
> We have just installed a NOAAPORT receive system from Unisys
> that is has both
> the NWSTG and GOES EAST channels.  My intention
> is to feed the LDM with the output and make the data
> available over the IDD to a
> university feed site (probably NIU) as a another feed source
> for the IDD community.  We are connected to the internet via a T3 with
> Sprintlink
> My question is what is the best way to feed the LDM the
> NOAAPORT data?  How is
> Unidata doing it?

The Unisys NOAAPORT system uses QNX as the base operating system.  This
is different from the Wisconsin solution which uses Solaris.  QNX is a
real time operating system which Unisys decided was best for making sure
all data is successfully received.  Five years ago, this was an issue on
PCs but is less an issue today with the fast EIDE and SCSI disks and
fast CPUs.

The solution I developed for the Unisys NOAAPORT system is a program
called "ldmfeed" which runs as a TCP socket server on the QNX server and
feeds data directly to the pqing program using its socket interface.
The LDM feed program uses the WXP ingestor as a filter allowing a
specified subset of the data to go out over pqing.  Future plans for the
data server include filtering and socket (server) output thus
eliminating the need for the intermediate WXP ingestor.

>       - The NOAAPORT 1 seems to lock up more frequently than I would
>       like to see. Over the past ten days, I've had to reboot it
>       on about four occasions. Until this stabilizes, this behavior
>       would leave downstream sites with frequent periods of no data.

There have been two subtle changes to the NOAAPORT feed over the past 5
months.  These continued changes have caused problems in terms of
successfully ingesting NOAAPORT.  The first was removed with a software
change in July.  The second change which happened about 6 weeks ago
seems to cause a system lock on the NWSTG/FOS channel.  I've notified
the engineer here and he is following up on the problem.  We are working
on a solution but this could be buried in one of the device drivers
which could be a problem.  Hopefully, we will have a solution soon.

Also, when we started up the pqing program at PSC, we were running into
problems with pqing dieing from a "permission denied" in writing to the
product queue.  I suspect this might be from overfilling the product
queue but I'm not sure.

>       - You would also need to have downstream users be more specific
>       with the pqact.conf specifications or they might become
>       overwhelmed with data. When I turned on the pqing ingest on one
>       of my machines running LDM, about 2 GB of disk space got used up
>       in about 10 hours and I wasn't even ingesting NOAAPORT II (GOES-
>       E) on that machine. Disk storage from the current IDD feed ramps
>       up much less quickly.

The bottom line here is be careful what you ingest.  As the NOAAPORT
feed slowly fills up, it will put more and more of a burden on
downstream IDD sites, not to mention the local LDM server.  You can
ingest satellite imagery as most imagery is about 1-2MB in size (smaller
than the AREA files) but there are more of them.  These images along
with the meso-Eta can fill a product queue rather fast.  The only thing
I don't recommend at this time is ingesting the CONUS visible images.
These are 26 and 22 MB in size for GOES east and west respectively.  I
have written a resample program which pares down the images from 1 to
4km for output to the LDM.  There is no need to use 1km images unless
you absolutely need them.  I'm planning on setting up the resample
program so its output could be fed to pqing.

________________________________________________________
Daniel Vietor               Mail: address@hidden
Unisys Corp                 Title: Engineer/Meteorologist
221 Gale Lane               Phone: 610-444-2407
Kennett Square PA 19348     Fax: 610-444-2420