[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: CONDUIT latencies

David,

Thanks for sending the information, I'll get this tested on one of our
Linux boxes and let you know the results. If it checks out ok, I'll
include it in the release. 

Robb...

  On Mon, 19 Jul 1999, David Wojtowicz wrote:

> Fri, 16 Jul 1999, Robb Kam
bic wrote:
> 
> > David,
> > 
> > There are Linux machines here at UCAR receiving NMC2 feed, Nexrad, etc.
> > We have found out that it's NOT possible to receive the above feeds on one
> > box because of the mmap memory management problems. These machine have
> > gigabyte of RAM and couldn't handle it. My suggestion is too put at least
> > 750 megabytes of RAM on the box receiving the NMC2 feed. In my opinion,
> > RAM is cheap compared to your time. Some day the Linux programmers will
> > fix the problem, until then use more RAM.  
> > 
> > Robb...
> 
> Granted, Linux's mmap is broke....but until its fixed there can
> be workarounds.
> 
> Adding more memory doesn't necessarily cure the problem, it only makes the
> system more usable.  You can still be hit with the wait to flush everything
> out on exit problem. 
> 
> I'm going to step out on a limb and say I've found a software
> fix to this problem.   Defining _MAPRGNS when compiling seems
> to change this troublesome behaviour completely.  pq.c is
> a bit hard to follow so I'm not sure exactly what going on
> but it seems that mapping regions at a time takes care of
> this.  Whenever a region is unmapped its freed to be written
> to disk.  This does not seem to seriously impact performance
> nor do I know of other negative side effects.  I've been 
> running it on both the NMC2 ingest machine and our main
> ingest machine (FOS,MCIDAS,NEXRAD,NLDN, DIFAX) etc for
> a few weeks and have not had trouble.  Now when you type
> "ldmadmin stop" the processes exit instantly.
> 
> 
> Here's the patch to the configure script:
> 
> *** configure   Fri Apr 16 20:39:56 1999
> --- configure.new       Wed Jun  9 21:47:17 1999
> ***************
> *** 1696,1701 ****
> --- 1696,1702 ----
>         fi
>       elif test -w /dev/conslog; then
>         cat >> confdefs.h <<\EOF
> + #define _MAPRGNS 1
>   #define _DEV_CONSLOG 1
>   EOF
> 
> 
> 
> 
> --------------------------------------------------------
>  David Wojtowicz, Research Programmer/Systems Manager
>  Department of Atmospheric Sciences Computer Services
>  University of Illinois at Urbana-Champaign
>  email: address@hidden  phone: (217)333-8390
> --------------------------------------------------------
> 
> 
> 
> 
> 

===============================================================================
Robb Kambic                                Unidata Program Center
Software Engineer III                      Univ. Corp for Atmospheric Research
address@hidden             WWW: http://www.unidata.ucar.edu/
===============================================================================