[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[LDM #YZW-103119]: Questions to send to LDM support team



David,

> For the past month we have been working on  implementing an LDM service
> and have a short list of questions we would appreciate your help in
> answering.  Our planning assumptions is that we will have 25 different
> customers using our service with an average of 600 GB comprised of around
> 170 products sitting in an LDM queue at any one time (or a total of 15
> GB comprised of 4,2 50 total product files).
> 
> 1) Do you recommend using one queue for all customers or a single queue
> per customer and why?

An LDM installation is designed to use a single product-queue. We find that 
sufficient to service roughly 160 customers at an aggregate bit-rate of 
approximately 1 GHz.

> 2) Assuming one queue for each customer, do you recommend using the same
> server for running multiple LDMs or a single server for each LDM?

Your mental model of an LDM system might be incorrect. The LDM server process 
forks itself for each incoming connection from a downstream LDM process. The 
child process sends requested data-products from the product-queue to the 
downstream LDM process. The child process terminates when the connection is 
broken.

> 3) If using a single queue for multiple customers, how do you prevent
> one customer whose network becomes really slow from affecting all
> the other clients using that same queue?

Because a separate child process is used for each downstream LDM, only that 
child process (and that connection) will be affected if the connection is slow.

> 4) What guidelines or rules of thumb can you suggest for sizing the
> LDM queue?  Is there any reason we should not make the LDM queue size
> and number of slots infinite to prevent the queue from getting filled
> up if one customer's connection is really slow and is not downloading
> files fast enough to keep up with the rate at which the files are
> being inserted into the queue?

We recommend that the queue size parameters (total amount of data, maximum 
number of products) be sufficient to hold at least an hour's worth of data.

> 6) We would like to track
> 
> A) whether a file arrived successfully or not, and

If the LDM is started in verbose logging mode ("ldmadmin start -v"), then each 
upstream LDM process will log each successfully-sent data-product and each 
downstream LDM process will log each successfully-received data-product.

> B) the data rate at which the file was transferred

That information is not available. We've never had the need for it.

We do monitor the aggregate LDM data rates on our routers, however.

> Is there any way of doing this given that this information is not in
> the standard LDM log?

There's no way to obtain the instantaneous transmission data-rate of a product 
from the LDM.

The log entries contain the product size and are time-stamped. Differentiating 
them would yield an average data-rate, but because data-products are inserted 
into the product-queue episodically (they have temporal gaps between them) that 
information might not be of much use.

> Thank you in advance for your help in answering these questions.

You're most welcome.

Will you be sending someone to the LDM Training Workshop on October 24th and 
25th?

Regards,
Steve Emmerson

Ticket Details
===================
Ticket ID: YZW-103119
Department: Support LDM
Priority: Normal
Status: Closed
===================
NOTE: All email exchanges with Unidata User Support are recorded in the Unidata 
inquiry tracking system and then made publicly available through the web.  If 
you do not want to have your interactions made available in this way, you must 
let us know in each email you send to us.