[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Comms

Kelvin,

> Quick question:  is all of this stuff done in serial mode, i.e.,
> does the LDM system allow for overlapping in things like
> compression, transmission, or is everything done in sequence?  I
> realize that the data are collected in sequence, but my point is
> that, if the processing takes longer than the transmission, then a
> bottleneck exists within the software that might be eliminated by
> overlapping various functions.  I'm speaking with zero knowledge of
> how LDM works in this regard, so forgive me if I'm totally out to
> lunch....

You can certainly be compressing one or more products while
transmitting others.  The LDM uses a separate process for sending data
to each downstream LDM, and that process shares the CPU or CPUs with
all the other processes running on the machine, including a process
compressing a data product.  So a compression process can be running
concurrently with a transmission processes.

However, the LDM cannot start sending a partial product before it has
been completely stored in the in-memory product queue.  So you cannot
overlap compression and sending the same data product.  The LDM
protocol requires the size of the product right up front, so you
cannot send a product before its complete size is known, which would
be the case if it was still being compressed.

Similarly, an LDM cannot begin relaying a product from an upstream
site to a downstream site until the product is completely received
from the upstream site and stored in the product queue.  In this case
it's not a matter of not knowing its size, but preventing deadlocks
and other similar problems ...

--Russ