Re: [ldm-users] Question about duplicate product checking

  • To: Matthew Foster - NOAA Federal <matthew.foster@xxxxxxxx>
  • Subject: Re: [ldm-users] Question about duplicate product checking
  • From: Steven Emmerson <emmerson@xxxxxxxx>
  • Date: Thu, 5 Apr 2018 10:01:35 -0600
Matt,

Let me amend my previous reply. I answered your question correctly, but
from too-narrow a perspective (one of the risks of getting an answer from
the developer I suppose 😉).

If products are being sent over a connection that's in the alternate
transfer mode (as opposed to primary transfer mode) then, initially, only
the metadata of the product is sent. The downstream LDM then calls a
product-queue function that determines whether or not the product is
already in the queue. If it is, then the downstream LDM responds with a
"don't send" message to the upstream LDM and the product isn't sent. If the
product is not in the queue, however, then the downstream LDM responds with
a "send" message. When the product is then received by the downstream LDM,
the downstream LDM again checks to see if the product is already in the
queue (in case another process added it) and rejects the insertion if it
is. The result of all this checking is reduced traffic on the alternate
transfer mode connection.

Is that what you wanted to know?

Regards,
Steve Emmerson

On Thu, Apr 5, 2018 at 9:38 AM, Steven Emmerson <emmerson@xxxxxxxx> wrote:

> The functions in the product-queue component that are responsible for
> inserting a data-product determine whether or not the product already
> exists in the product-queue and reject the insertion if it does.
>
> Regards,
> Steve Emmerson
>
> On Thu, Apr 5, 2018 at 9:25 AM, Matthew Foster - NOAA Federal <
> matthew.foster@xxxxxxxx> wrote:
>
>> I was trying to find the answer to this question in the online
>> documentation, but I couldn't find it clearly stated.
>>
>> When, exactly, does duplicate checking occur? Does the downstream LDM
>> receive the data product *then* do the duplicate check before inserting
>> into the queue, OR does the upstream send the product's checksum and the
>> accept/reject decision is made at that point (i.e. before the product is
>> sent)? I've always assumed it was the latter, but now I'm not 100% sure.
>>
>> Thanks!
>>
>> Matt
>>
>>
>> --
>> Matt Foster, Science & Operations Officer
>> NWS Operations Proving Ground
>> 7220 NW 101st Terrace
>> Kansas City, MO 64153
>> Office: 816-994-3072
>>
>>
>>
>> _______________________________________________
>> NOTE: All exchanges posted to Unidata maintained email lists are
>> recorded in the Unidata inquiry tracking system and made publicly
>> available through the web.  Users who post to any of the lists we
>> maintain are reminded to remove any personal information that they
>> do not want to be made public.
>>
>>
>> ldm-users mailing list
>> ldm-users@xxxxxxxxxxxxxxxx
>> For list information or to unsubscribe,  visit:
>> http://www.unidata.ucar.edu/mailing_lists/
>>
>
>
  • 2018 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the ldm-users archives: