Here is a patch I made around version 6.11.1 and have maintained through
6.13.5. It addresses the same situation you described. When it is
deduced that a product is from alternate feed, then unless the product
is already pretty late, delay 0.5 seconds before processing it. This
insures the primary wins out (esp. when primary and alternate have
nearly identical delays). I've never seen any ill effects from this and
also have used it for NOAAPORT feeds.
After making this patch, the total bandwidth was back to close to that
of only using one source.
Three files are patched, the patches below are separated by "#######".
#######
--- src/protocol2/down6.c.prepatch 2016-10-06 14:54:17.000000000 -0500
+++ src/protocol2/down6.c 2016-11-23 14:50:02.785443000 -0600
@@ -332,6 +332,7 @@
errCode = DOWN6_UNINITIALIZED;
}
else {
+ struct timeval tnow;
prod_info *infop = argp->infop;
if (_expectBlkdata) {
@@ -343,6 +344,7 @@
}
(void)set_timestamp(&_class->from);
+ tnow=_class->from;
_class->from.tv_sec -= max_latency;
dh_setInfo(_info, infop, _upName);
@@ -379,6 +381,12 @@
else {
pqe_index idx; /* product-queue index */
+ /* delay some if product not very old */
+ if (d_diff_timestamp(&tnow, &_info->arrival) < 6) {
+ struct timeval tv = {0,500000};
+ select(0,NULL,NULL,NULL,&tv);
+ }
+
/*
* Reserve space for the data-product in the product-queue.
*/
#######
--- src/protocol2/down6.h.prepatch 2016-10-06 14:54:17.000000000 -0500
+++ src/protocol2/down6.h 2016-11-23 14:50:02.794447499 -0600
@@ -6,6 +6,8 @@
#include "ldm.h" /* prod_class */
#include "pq.h"
+#define DOWN6_PATCH_PRIALT /* has been patched for better pri/alt
operation */
+
#ifdef __cplusplus
extern "C" {
#endif
#######
--- src/ldmd/ldmd.c.prepatch 2016-10-06 14:54:17.000000000 -0500
+++ src/ldmd/ldmd.c 2016-11-23 14:50:02.710405499 -0600
@@ -979,6 +979,14 @@
log_notice("Starting Up (version: %s; built: %s %s)", PACKAGE_VERSION,
__DATE__, __TIME__);
+#if defined(PQ_PATCH_BACKTIME)
+ log_notice("The patch to accomodate non-monotonic clock has been
applied.\n");
+#endif
+#if defined(DOWN6_PATCH_PRIALT)
+ log_notice("The primary/alternate enhancement patch has been
applied\n.");
+#endif
+
+
/*
* register exit handler
*/
#######
On 01/12/2018 01:43 PM, Greg Trotter wrote:
I have an LDM server that is connected to two upstreams – one is my
NOAAPORT ingester on my local network, and one is a backup site at a
remote location. I have both listed in my ldmd.conf, with the local
server listed first, and the request patterns are identical.
Looking at the queue, it looks like all is well – the origin on all
the products is from our local NOAAPORT ingest server. If I look at
the network traffic on that server, however, I see a lot of data
coming in from the remote server – almost as much as we are bringing
in from the local server. I theorize that the downstream doesn’t see
it locally, and requests it from the remote upstream, but by the time
it arrives, it’s come in from the local upstream? I can’t find any
other reason that we’d be having that much traffic from an upstream
LDM that never appears as an origin for the products in our queue.
Is there something I am doing wrong? Clearly, I’d prefer not to use
that much bandwidth if we don’t have to. Is there a way to have it
only hit the remote upstream if the local is unavailable?
Here’s the relevant part of my ldmd.conf:
REQUEST ANY ".*" <IP of local noaaport ingester>
REQUEST ANY ".*" <IP of remote noaaport ingester>
*Greg Trotter*
System Administrator
Weather Decision Technologies, Inc
201 David L Boren Blvd Suite 270
Norman, Oklahoma 73072
_______________________________________________
NOTE: All exchanges posted to Unidata maintained email lists are
recorded in the Unidata inquiry tracking system and made publicly
available through the web. Users who post to any of the lists we
maintain are reminded to remove any personal information that they
do not want to be made public.
ldm-users mailing list
ldm-users@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
