Re: [ldm-users] ldm data dir question

file system is xfs.

/dev/sde5 on /home type xfs (rw,relatime,attr2,inode64,noquota)

df -i /home
Filesystem        Inodes  IUsed     IFree IUse% Mounted on
/dev/sde5      188351872 169213 188182659    1% /home


I have removed about 150K files from /home/.... /data  but letting the

find /home/ldm/var/data -type f -mtime +1 -print | xargs rm -f

command keep running... I'll try and restart ldm when that finishes.

- jack



On 4/24/20 1:41 PM, Evan Breznyik wrote:
What filesystem is this on...vis a vis how are the inodes looking?  I hit a corner case with this a few years ago where we were saving 100K+ files per day and we simply ran out of inodes on ext4...but I am just thinking aloud.



On Fri, Apr 24, 2020 at 11:32 AM Jack Snodgrass <jack@xxxxxxxxxxxxxx <mailto:jack@xxxxxxxxxxxxxx>> wrote:

    having issues with our server ( centos7 ) that runs ldm... locking
    up. It has happened 2 times in the last 3 weeks or so.
    The server is pingable... so it's not totally dead.. but you can't
    get a local or remote console to start. can't figure out if it is
    out of memory or file handles or what.... it's like a ghost of
    itself.

    After rebooting... the  /home/ldm/var/data/ has around 350,000
    files in it.  I am not sure if that is 'ok' or a bit extra.

    We are running a

    ldmadmin scour

    command... via cron but I don't know what that is doing exactly or
    it it's doing much.

    when I try and restart ldm it says:

    Checking the product-queue...
    The writer-counter of the product-queue isn't zero.  Either a process
    has the product-queue open for writing or the queue might be corrupt.
    Terminate the process and recheck or use
        pqcat -l- -s -q /home/ldm/var/queues/ldm.pq && pqcheck -F -q
        /home/ldm/var/queues/ldm.pq
    to validate the queue and set the writer-counter to zero.
    LDM not started


    In the past.... during testing and what not.. I've been able to run:
    pqcat -l- -s -q /home/ldm/var/queues/ldm.pq && pqcheck -F
    -q/home/ldm/var/queues/ldm.pq

    and ldm would start after that. This time.. with the 350K files or
    so.. that pqcat stuff fails.

    I am deleting older ( than a day ) files from the
    /home/ldm/var/data/ direcory... going to see if

    pqcat -l- -s -q /home/ldm/var/queues/ldm.pq && pqcheck -F
    -q/home/ldm/var/queues/ldm.pq


    will work or if I have to rm -rf /home/ldm/var/data/ and start a
    new q.


    If  ldmadmin scour does not let us remove enough files from
    /home/ldm/var/data/ can I use find and rm to remove files or do
    they have to be removed using ldm to keep and queses or indexes 
    in sync?

    - jack

-- *jack* - Southlake Texas - http://mylinuxguy.net
    <http://mylinuxguy.net/> - *817-601-7338*
    _______________________________________________
    NOTE: All exchanges posted to Unidata maintained email lists are
    recorded in the Unidata inquiry tracking system and made publicly
    available through the web.  Users who post to any of the lists we
    maintain are reminded to remove any personal information that they
    do not want to be made public.


    ldm-users mailing list
    ldm-users@xxxxxxxxxxxxxxxx <mailto:ldm-users@xxxxxxxxxxxxxxxx>
    For list information or to unsubscribe,  visit:
    https://www.unidata.ucar.edu/mailing_lists/



--
*jack* - Southlake Texas - http://mylinuxguy.net <http://mylinuxguy.net/> - *817-601-7338*
  • 2020 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: