[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

20040614: large queue support under Linux (cont.)

>From:  Steve Emmerson <address@hidden>
>Organization:  UCAR/Unidata
>Keywords:  200406141508.i5EF8HtK011065 LDM Linux large queue

Steve,

re: large LDM queue under Linux
>This is good, actionable information.  I'll start work on it today.

Super.

>Offhand, I don't know why the "4000000000" specification didn't work.

I believe the problem is in Perl, not the LDM.  However, since
the syntax "4000M" is allowed by pqcreate, the solution is simply
to stop specifying the queue size as a numeric value:

$pq_size = 4000000000;

and start specifying it as a string:

$pq_size = "4000M";

At least for me, this is easier to understand anyway.

Cheers,

Tom

>--Steve
>
>> >From: Unidata User Support <address@hidden>
>> >Organization: Unidata Program Center/UCAR
>> >Keywords:  200406141508.i5EF8HtK011065 LDM Linux large queue
>>
>> 
>> Steve,
>> 
>> While trying to upgrade the queue a Fedora Core 1 Linux machine (it was
>> using a 2 GB queue, and the age of the oldest product in the queue was
>> less than one hour since it is getting everything in the IDD), I learned
>> out that you can set CPPFLAGS so that large file support is include in
>> LDM routines:
>> 
>> cd ~ldm/ldm-6.0.14/src
>> make distclean
>> export CPPFLAGS="-D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE"
>> ./configure && make && make install && sudo make install_setuids
>> 
>> I then edited ~ldm/bin/ldmadmin and set $pq_size to 4 GB:
>> 
>> $pq_size = 4000000000;
>> 
>> and then used ldmadmin to delete and make a new queue:
>> 
>> ldmadmin delqueue
>> ldmadmin mkqueue -f
>> 
>> No matter what numeric value greater than 2 GB I set in ldmadmin, a 2
>> GB queue was all that was created.  I even tried quoting the numeric
>> size value:
>> 
>> $pq_size = "4000000000";
>> 
>> But I still got only a 2 GB queue.
>> 
>> When I tried to set the queue size using a format that is supported by
>> pqcreate, 4000M:
>> 
>> $pq_size = 4000M;
>> 
>> I got an error.
>> 
>> Quoting the queue size in the 'M' format, on the other hand:
>> 
>> $pq_size = "4000M";
>> 
>> did result in a properly sized queue.
>> 
>> So, two recommendations for the next LDM release are:
>> 
>> - quote the pq_size field in ldmadmin and add instructions for
>>   how to specify a queue size greater than 2 GB
>> 
>> - update the web page that talks about building large file support
>>   into the LDM on Linux to tell folks that then not only have to
>>   set CPPFLAGS, but they also have to change the format of
>>   pq_size in ldmadin:
>> 
>> http://my.unidata.ucar.edu/content/software/ldm/ldm-6.0.14/basics/huge-queue
> s.html#Linux/Intel
>> 
>> I just wanted to get this "down on paper" before the meeting Monday.
>> 
>> Cheers,
>> 
>> Tom
>> --
>> +---------------------------------------------------------------------------
> --+
>> * Tom Yoksas                                             UCAR Unidata Progra
> m *
>> * (303) 497-8642 (last resort)                                  P.O. Box 300
> 0 *
>> * address@hidden                                   Boulder, CO 8030
> 7 *
>> * Unidata WWW Service                            http://www.unidata.ucar.edu
> / *
>> +---------------------------------------------------------------------------
> --+
--
+-----------------------------------------------------------------------------+
* Tom Yoksas                                             UCAR Unidata Program *
* (303) 497-8642 (last resort)                                  P.O. Box 3000 *
* address@hidden                                   Boulder, CO 80307 *
* Unidata WWW Service                             http://www.unidata.ucar.edu/*
+-----------------------------------------------------------------------------+