[
Date Prev][
Date Next][
Thread Prev][
Thread Next][
Date Index][
Thread Index]
Re: 20010222: LDM-METAR problem (fwd) - more sample data
- Subject: Re: 20010222: LDM-METAR problem (fwd) - more sample data
- Date: Fri, 23 Feb 2001 16:20:23 -0700 (MST)
Celia,
Thanks for the sample reports. It seems that report were actually
processed by pqsurf but were output under the metar heading instead of
speci when the PIL tag was present. I fixed the code so they come out
under speci instead. The surf_split.c code is attached, insert into the
pqsurf dir and compile.
Hopefully this solves your problem,
Robb...
On Thu, 22 Feb 2001, Unidata Support wrote:
>
> ------- Forwarded Message
>
> >To: address@hidden
> >cc: address@hidden (Celia Chen),
> >cc: address@hidden
> >From: address@hidden (Celia Chen)
> >Subject: LDM-METAR problem (fwd) - more sample data
> >Organization: UCAR/Unidata
> >Keywords: 200102221957.f1MJv1L29590
>
> I realized that I only sent you the sample metar reports from US sites in my
> last email. Actually none of reports from the Canadian sites made into the
> gdb
> file. Here are some sample reports for you to check:
>
> ^C^A^M^M
> 619 ^M^M
> SPCN48 CWAO 221206^M^M
> SPECI CYNM 221206Z AUTO 14003KT 1 3/8SM CLR M39/M45 A3025 RMK VSBY^M^M
> VRBL 0.6V9.+=^M^M
> ^M^M
> ^M^M
> ^C^A^M^M
> 622 ^M^M
> SPCN45 CWAO 221205^M^M
> SPECI CYTE 221205Z AUTO 03017KT 1 3/8SM SCT026 M25/M30 A2994=^M^M
> ^M^M
> ^C^A^M^M
> 836 ^M^M
> SPCN51 CWAO 221206^M^M
> SPECI CWUW 221206Z AUTO 09018G24KT 5SM CLR M28/=^M^M
> ^M^M
> ^M^M
> ^C^A^M^M
> 837 ^M^M
> SPCN51 CWAO 221202^M^M
> SPECI CWZZ 221202Z AUTO ///// 1/8SM -RA RMK ICG PAST HR=^M^M
> ^M^M
> ^M^M
>
> ^C^A^M^M
> 962 ^M^M
> SPCN51 CWAO 221204^M^M
> SPECI CZCP 221204Z AUTO 31005KT 8SM FEW017 M26/ RMK ICG PAST HR=^M^M
> ^M^M
> ^M^M
>
> ^C^A^M^M
> 760 ^M^M
> SPCN48 CWAO 221209^M^M
> SPECI CYNM 221209Z AUTO 14003KT 1SM CLR M39/M45 A3025 RMK VSBY VRBL^M^M
> 0.6V5.0=^M^M
> ^M^M
> ^M^M
>
> ^C^A^M^M
> 179 ^M^M
> SPCN42 CWAO 221210^M^M
> SPECI CYHE 221210Z AUTO 10007KT 9SM OVC024 04/M02 A3006=^M^M
> ^M^M
> ^M^M
>
> ^C^A^M^M
> 938 ^M^M
> SPCN51 CWAO 221213^M^M
> SPECI CZCP 221213Z AUTO 32004KT 9SM BKN017 M26/ RMK ICG PAST HR=^M^M
> ^M^M
> ^M^M
>
> ^C^A^M^M
> 249 ^M^M
> SPCN48 CWAO 221211^M^M
> SPECI CYQI 221211Z 32019G28KT 2 1/2SM -SHSN BKN025 RMK SC6=^M^M
> ^M^M
>
>
> >
> > I finally have some sample metar reports that didn't make into
> > the gdb file for you. I have also included two reports that did
> > make into the gdb file.
> >
> > Thanks.
> >
> > Celia
> >
> > Forwarded message:
> > > From address@hidden Mon Feb 5 08:47:43 2001
> > > From: address@hidden (Greg Thompson)
> > > Message-Id: <address@hidden>
> > > Subject: LDM-METAR problem
> > > To: address@hidden, address@hidden
> > > Date: Mon, 5 Feb 2001 08:47:39 -0700 (MST)
> > > X-Mailer: ELM [version 2.5 PL0pre8]
> > > MIME-Version: 1.0
> > > Content-Type: text/plain; charset=us-ascii
> > > Content-Transfer-Encoding: 7bit
> > >
> > >
> > > NCAR-RAP is having problems with LDM's METAR data. It appears that
> > > SPECIs are not making it into our 'GDB' files but are into the plain
> > > text METAR files.
> > >
> > > This is the entry in our pqact.conf file:
> > > WMO ^S[APX].* .... ([0-3][0-9])([0-2][0-9]) FILE
> > > /weather/ddp/(\1:yyyy)(\1:mm)\1/(\1:yyyy)(\1:mm)\1\2.METAR
> > >
> > > While this is the entry in our pqsurf.conf file:
> > > WMO ^(metar|speci) (....) ([0-3][0-9])([0-2][0-9]) DBFILE
> > > /weather/ddp/(\3:yyyy)(\3:mm)\3/(\3:yyyy)(\3:mm)\3\4.metar_gdb \2
> > >
> > > Back in December, Celia Chen contacted Robb K and told him about this
> > > issue.
> > > We thought he fixed it and it appeared he did. BUT, when he made this
> > > fix,
> > > hundreds of Canadian sites suddenly stopped being ingested. So we asked
> > > for the fix for Canadian sites. Not sure if these two items have anything
> > > to do with each other, but we didn't re-check to see if SPECIs were
> > > ingesting
> > > after the Canadian fix. In either case, we are not receiving SPECIs
> > > again in our GDB files. It is my belief we are running LDM5.1.2 except
> > > we got a new surf_split.c from Robb for the fix for Canadian sites.
> > >
> > > Please advise. Thanks.
> > >
> > > --
> > > +--------------------------------------------------------------+
> > > | Greg Thompson http://www.rap.ucar.edu/staff/gthompsn/ |
> > > | Research Applications Program |
> > > | (303) 497-2805 National Center for Atmospheric Research |
> > > | (fax) -8401 P.O. Box 3000 Boulder, CO 80307-3000 |
> > > +--------------------------------------------------------------+
> > >
> >
> >
> > --
> > Celia Chen
> > RAP/NCAR
> >
> > "Nothing is as strong as gentleness or as gentle as strength."
> > - Dove Milk Promises
> >
> >
> >
> > --%--multipart-mixed-boundary-1.19760.982695167--%
> > Content-Type: text/plain; charset=us-ascii
> > Content-Transfer-Encoding: 7bit
> > Content-Description: commands text
> > Content-Disposition: attachment; filename="Metar_12"
> >
> > SAMPLE METAR REPROTS:
> >
> > These made it:
> > ==============
> >
> > ^C^A^M^M
> > 786 ^M^M
> > SAUS46 KLOX 201234^M^M
> > MTRCQT^M^M
> > SPECI KCQT 201233Z AUTO 00000KT 2 1/2SM BR CLR 08/08 A3016 RMK AO2 ^M^M
> > TSNO^M^M
> > ^M^M
> > ^C^A^M^M
> >
> > ^C^A^M^M
> > 467 ^M^M
> > SAUS45 KTFX 201206^M^M
> > MTRHVR^M^M
> > SPECI KHVR 201205Z AUTO 06005KT 1 1/4SM HZ BKN001 BKN006 M19/M22 ^M^M
> > A3025 RMK AO2^M^M
> > ^M^M
> > ^C^A^M^M
> >
> > These didn't:
> > =============
> >
> > ^C^A^M^M
> > 705 ^M^M
> > SAUS43 KAPX 201230^M^M
> > MTRHTL^M^M
> > SPECI KHTL 201227Z AUTO 23007KT 5SM BR SCT021 M06/M08 A2995 RMK AO2^M^M
> > ^M^M
> > ^C^A^M^M
> >
> > ^C^A^M^M
> > 179 ^M^M
> > SAUS43 KGID 201224^M^M
> > MTRGRI^M^M
> > SPECI KGRI 201222Z 03011KT 10SM OVC009 M04/M06 A3023 RMK AO2 CIG ^M^M
> > 008V013 $^M^M
> > ^M^M
> > ^C^A^M^M
> >
> > ^C^A^M^M
> > 758 ^M^M
> > SAUS43 KDLH 201242^M^M
> > MTRDLH^M^M
> > SPECI KDLH 201241Z 29012KT 9SM OVC016 M17/M18 A2997 RMK AO2^M^M
> > ^M^M
> > ^C^A^M^M
> >
> >
> > ^C^A^M^M
> > 409 ^M^M
> > SAUS44 KCRP 201209^M^M
> > MTRCRP^M^M
> > SPECI KCRP 201209Z 12003KT 3/4SM BR VV001 14/14 A3010 RMK AO2 $^M^M
> > ^M^M
> > ^C^A^M^M
> >
> > ^C^A^M^M
> > 442 ^M^M
> > SAUS43 KOAX 201217^M^M
> > MTRBFF^M^M
> > SPECI KBFF 201215Z AUTO 06009KT 2SM BR OVC003 M01/M01 A3009 RMK AO2^M^M
> > ^M^M
> > ^C^A^M^M
> >
> > ^C^A^M^M
> > 441 ^M^M
> > SAUS46 KPQR 201221^M^M
> > MTRBNO^M^M
> > SPECI KBNO 201220Z AUTO 21003KT 2SM -SN BR SCT004 OVC011 00/00 A2992 ^M^M
> > RMK AO2 P0000 TSNO^M^M
> > ^M^M
> > ^C^A^M^M
> >
> > ^C^A^M^M
> > 748 ^M^M
> > SAUS45 KBOI 201226^M^M
> > MTRBOI^M^M
> > SPECI KBOI 201225Z 29003KT 1 3/4SM -RA BR OVC018 03/03 A2995 RMK AO2 ^M^M
> > P0000^M^M
> > ^M^M
> > ^C^A^M^M
> >
> > ^C^A^M^M
> > 954 ^M^M
> > SAUS44 KLCH 201226^M^M
> > MTRBPT^M^M
> > SPECI KBPT 201225Z 10003KT 3/4SM BR VV005 14/14 A3017 RMK AO2 TWR ^M^M
> > VIS 2 1/2^M^M
> > ^M^M
> > ^C^A^M^M
> >
> > --%--multipart-mixed-boundary-1.19760.982695167--%--
> >
>
>
> --
> Celia Chen
> RAP/NCAR
>
> "Nothing is as strong as gentleness or as gentle as strength."
> - Dove Milk Promises
>
>
>
> ------- End of Forwarded Message
>
>
===============================================================================
Robb Kambic Unidata Program Center
Software Engineer III Univ. Corp for Atmospheric Research
address@hidden WWW: http://www.unidata.ucar.edu/
===============================================================================
/*
* Copyright 1993, University Corporation for Atmospheric Research
* See ../COPYRIGHT file for copying and redistribution conditions.
*/
/* $Id: surf_split.c,v 1.34 2001/02/23 22:56:38 rkambic Exp $ */
#include <ldmconfig.h>
#include <stdio.h>
#include <ctype.h>
#include <string.h>
#include "ldm.h"
#include "ulog.h"
#include "wmo_header.h"
#include "tokens.h"
#include "xbuf.h"
#include "surface.h" /* wind_units_t, CALL_SIGN_LEN */
#include "md5.h"
static double md5ctx[16]; /* 88 would be big enough */
static MD5_CTX *md5ctxp = (MD5_CTX *)md5ctx;
extern int usePil; /* 1/0 flag to signal use of AFOS like pil identifier */
static int
get_yygg(xbuf *buf, dtime *time)
{
int status;
int YY = -1;
int GG = -1;
if((status = dget_wnum(buf, &YY, 2)) < 0) return status;
if((status = dget_num(buf, &GG, 2)) < 0) return status;
set_dtime(time, YY, GG, 0);
return status;
}
/* For METAR, check if the HHMMZ time string is present */
static int
whas_yyggZ(xbuf *buf)
{
int ch;
/* skip white space */
do{
ch = nextc(buf);
}while((isascii(ch) && !isgraph(ch)));
unnextc(buf,ch);
if(buf->cnt < 4) {
return 0; /* not enough characters */
} else if(buf->get[4] == 'Z' || buf->get[6] == 'Z' ) {
return 1;
} else if( isdigit(buf->get[3])
&& isdigit(buf->get[2])
&& isdigit(buf->get[1])
&& isdigit(buf->get[0])) {
return 1;
} else if( isdigit(buf->get[5])
&& isdigit(buf->get[4])
&& isdigit(buf->get[3])
&& isdigit(buf->get[2])
&& isdigit(buf->get[1])
&& isdigit(buf->get[0])) {
return 1;
}
return 0; /* failed */
}
/* For METAR, check if "NIL" */
static int
has_NIL(xbuf *buf)
{
char nilstr[] = "NIL";
char *np = (char *)&buf->base[buf->bufsiz - 1 - (sizeof(nilstr) -1 -1)];
if(strncmp(np, nilstr, sizeof(nilstr) -1) == 0)
return 1;
return 0;
}
/* For METAR, get the bulletin time, if possible */
static void
get_wyyggZ(xbuf *buf, dtime *time)
{
int ch;
if(!whas_yyggZ(buf))
return;
(void)get_yygg(buf, time);
ch = nextc(buf); /* eat the 'Z' */
return;
}
/*
* Takes a WMO format product which is a
* SAO, SYNOP, SHIP, METAR, or SPECI message, splits it into
* individual observations. The observations are each encapsulated in a
* new product which inherits most of its description from the
* original product.
* The new product pkey is derived from the observation type
* and has the following form:
*
* SAO - "sao tt ccc ddhhmm"
* where:
* tt is SA, SP or RS
* ccc is the station ID like SFO, LXV, etc
* ddhhmm is the time stamp.
*
* SYNOP - "aaxx nnnnn ddhhmm"
* where:
* nnnnn is the WMO station id (5 digit number)
*
* SHIP - "bbxx c* ddhhmm"
* where:
* c* is the call sign
*
* METAR - "metar cccc ddhhmm"
* where:
* cccc is the call sign
*
* SPECI - "speci cccc ddhhmm"
*
* The new product sequence number is original sequence number times 1000
* plus the sequence of the individual observation within the product.
*
* 'doit' is called on each of the new products. It is presumed
* this function return zero upon success.
*
* Returns the number of successful calls to 'doit', eg, the
* number of splits. Returns -1 on error.
*/
int
surf_split(const prod_info *infop, const void *datap,
int (*doit)(const prod_info *, const void *))
{
int action = -1;
wmo_header_t hdr;
message_type_t mtype;
dtime dt;
xbuf buf[1];
unsigned char dbuf[8192]; /* TODO */
char header[20];
int nsplit = 0;
enum {
SURFACE_BOGUS ,
AAXX,
US_AAXX,
BBXX,
SAO,
sMETAR,
sSPECI
} subtype = SURFACE_BOGUS;
hdr.time = &dt;
if(infop->sz > sizeof(dbuf))
return -1; /* TODO: too big */
memcpy(dbuf, datap, infop->sz);
if( cbuftoxbuf(buf, (unsigned char *)dbuf,
infop->sz) == NULL)
return -1;
skipline(buf, 4); /* SOH */
skipline(buf, 12); /* start */
memcpy( header, datap, 15 );
header[15] = '\0';
if( strstr( header, "SACN") ) {
usePil = 0;
}
if( get_wmo_header(buf, &hdr) == NULL)
{
return -1;
}
usePil = 1;
#if DEBUG
fputs("\t", stderr);
fprint_wmo_header(stderr, &hdr);
fputs("\n", stderr);
#endif
mtype = decode_type(hdr.TT,hdr.AA,hdr.PIL);
/* #### */
{
char cbuf[8];
int digit;
dtime time;
wind_units_t wind_units = WIND_UNAVAIL;
time = *hdr.time; /* default the ob time to the time in the header */
/* delve into section 0 */
switch(mtype) {
case SYNOP :
if(get_wstr(buf, cbuf, 1) < 0 ) return -1;
if(cbuf[0] == 'A')
{
subtype = AAXX;
if(get_str(buf, &cbuf[1], 3) < 0 ) return -1;
if( cbuf[3] != 'X' )
{
/* punt */
uerror("surface_split: Unknown type: %s\n",
cbuf);
return 0;
}
if(get_yygg(buf, &time) < 0 ) return -1; /* YYGG */
if(dget_num(buf, &digit, 1) < 0 ) return -1; /* isubw */
if(digit >= 0 && digit <= 4) wind_units =
(wind_units_t)digit;
}
else if(isascii(cbuf[0]) && isdigit(cbuf[0])) /* US Stations
7NNNN */
{
unnextc(buf,cbuf[0]);
subtype = US_AAXX;
/*
* Some US reports leave off AAXX YYGGisubw, so we use
the
* time from the wmo header.
*/
wind_units = KNOTS;
}
else
{
unnextc(buf,cbuf[0]);
return 0; /* ?? */
}
break;
case SHIP :
if(get_wstr(buf, cbuf, 4) < 0 ) return -1;
if(cbuf[0] == 'B')
{
if( cbuf[3] != 'X' )
{
/* punt */
uerror("surface_split: Unknown type: %s\n",
cbuf);
return 0;
}
subtype = BBXX;
/* get time below */
}
else
{
unnextc(buf,cbuf[0]);
return 0;
}
break;
case METAR :
if( hdr.PIL[0] != NULL &&
(! strstr(hdr.PIL, "METAR") ||
!strstr(hdr.PIL, "MTR"))){
subtype = sMETAR;
} else if(whasSTR(buf, "METAR"))
{
subtype = sMETAR;
get_wyyggZ(buf, &time);
}
else if(whasSTR(buf, "SPECI"))
{
subtype = sSPECI;
get_wyyggZ(buf, &time);
}
else {
subtype = SAO; /* may actually be a METAR, check below
*/
}
break;
case SPECI :
if(whasSTR(buf, "SPECI"))
{
subtype = sSPECI;
get_wyyggZ(buf, &time);
}
break;
default :
uerror("surface_split: Can't handle %s",
sMessage_type(mtype) );
uerror("HDR + PIL: %s%s %s", hdr.TT, hdr.AA, hdr.PIL ) ;
return -1;
}
{ /* while block */
static char newkey[KEYSIZE];
xbuf subbuf[1];
prod_info newinfo = *infop;
#define MAX_SURF_LEN 511
#undef MIN
#define MIN(a,b) ((a) <= (b) ? (a) : (b))
char pbuf[MAX_SURF_LEN + 1];
int l1, l2;
static char ident[CALL_SIGN_LEN+1];
static char type[4];
char *stype;
u_int subseq = infop->seqno * 1000;
unsigned char *pp;
while( get_weqxbuf(buf, subbuf) > 0 )
{
(void)memset(newkey,0,KEYSIZE);
(void)memset(pbuf,0,MAX_SURF_LEN + 1);
(void)memset(ident,0,CALL_SIGN_LEN+1);
pp = subbuf->base;
switch(subtype) {
case AAXX :
case US_AAXX :
strcpy(newkey, "aaxx ");
strcpy(pbuf, "AAXX");
sprintf(&pbuf[strlen(pbuf)], " %02d%02d%1d\r\r\n",
time.mday, time.hour, (int)wind_units);
/* WMO station no. */
if(get_wstr(subbuf, ident, 5) < 0)
continue;
strcat(newkey, ident);
break;
case BBXX :
strcpy(newkey, "bbxx ");
strcpy(pbuf, "BBXX\r\r\n");
/* call sign */
if(get_wstr(subbuf, ident, CALL_SIGN_LEN) < 0)
continue;
strcat(newkey, ident);
if(get_yygg(subbuf, &time) < 0) continue; /* YYGG */
break;
case sSPECI :
/* call sign */
if(get_wstr(subbuf, ident, CALL_SIGN_LEN) < 0)
continue;
stype = "SPECI";
if(strcmp(ident, "METAR") == 0 ||
strcmp(ident, "SPECI") == 0)
{
if( strcmp(ident, "METAR") == 0) {
stype = "METAR";
}
/* They package each ob with a tag */
pp = (subbuf->get +1);
if(get_wstr(subbuf, ident, CALL_SIGN_LEN) < 0)
continue;
}
if(!whas_yyggZ(subbuf))
{
/* Have to insert the date */
sprintf(pbuf, "%s\r\r\n%s %02d%02dZ ",
stype, ident, time.hour, time.min);
pp = subbuf->get;
}
else {
strcpy(pbuf, stype);
strcat(pbuf, "\r\r\n");
}
if(strcmp(stype, "METAR") == 0 ) {
strcpy(newkey, "metar ");
} else {
strcpy(newkey, "speci ");
}
strcat(newkey, ident);
break;
case sMETAR :
if(has_NIL(subbuf))
continue;
/* call sign */
if(get_wstr(subbuf, ident, CALL_SIGN_LEN) < 0) {
continue;
}
stype = "METAR";
if(strcmp(ident, "METAR") == 0 ||
strcmp(ident, "SPECI") == 0)
{
if( strcmp(ident, "SPECI") == 0) {
stype = "SPECI";
}
/* They package each ob with a tag */
pp = (subbuf->get +1);
if(get_wstr(subbuf, ident, CALL_SIGN_LEN) < 0)
continue;
}
if(!whas_yyggZ(subbuf))
{
/* Have to insert the date */
sprintf(pbuf, "%s\r\r\n%s %02d%02dZ ",
stype, ident, time.hour, time.min);
pp = subbuf->get;
}
else {
strcpy(pbuf, stype);
strcat(pbuf, "\r\r\n");
}
if(strcmp(stype, "METAR") == 0 ) {
strcpy(newkey, "metar ");
} else {
strcpy(newkey, "speci ");
}
strcat(newkey, ident);
break;
case SAO :
/* call sign */
if(get_wstr(subbuf, ident, CALL_SIGN_LEN) < 0)
continue;
if(hdr.AA[0] == 'U' && hdr.AA[1] == 'S'
&& strlen(ident) == 6)
{
/* skip 6 char US "AFOS code" */
if(get_wstr(subbuf, ident, CALL_SIGN_LEN) < 0)
continue;
}
/* SA, SP, RS, USP or XP */
if(get_wstr(subbuf, type, 3) < 0)
continue;
if((type[0] == 'S'
&& (type[1] == 'A' || type[1] == 'P'))
|| (type[0] == 'R' && type[1] == 'S')
|| (type[0] == 'U' && type[1] == 'S'
&& type[2] == 'P')
|| (type[0] == 'X' && type[1] == 'P')
|| (type[0] == 'T' &&
(type[1] == 'A' || type[1] == 'S'))
)
{
strcpy(newkey, "sao ");
strcat(newkey, type);
strcat(newkey, " ");
strcat(newkey, ident);
}
else if(isdigit(type[0]) && isdigit(type[1]))
{
/* it is a METAR really */
subtype = sMETAR;
strcpy(newkey, "metar ");
strcat(newkey, ident);
strcpy(pbuf, "METAR\r\r\n");
}
else
continue; /* don't know what it is, "NIL=" */
break;
}
/* safety net */
if(strlen(ident) == 0)
{
continue;
}
/* else */
sprintf(&newkey[strlen(newkey)], " %02d%02d%02d",
time.mday, time.hour, time.min);
if(hdr.retransmit != ORIGINAL)
sprintf(&newkey[strlen(newkey)], " %s",
sRetransmit(&hdr));
newinfo.ident = newkey;
newinfo.seqno = ++subseq;
l1 = strlen(pbuf);
l2 = MIN(MAX_SURF_LEN - l1 - 4, subbuf->bufsiz - (pp -
subbuf->base));
/* N.B.: silent truncation */
strncat(pbuf, (char *)pp, l2 );
strcat(pbuf,"=\r\r\n");
newinfo.sz = l1 + l2 + 4;
#if DEBUG
fprintf(stderr,"\t\t%s\n", newinfo.ident);
#endif
#if PRINT
{
char *cp = pbuf;
char *end = &cp[newinfo.sz];
while(cp < end)
{
putc(*cp, stderr);
cp++;
}
}
putc('\n', stderr);
#endif
MD5Init(md5ctxp);
MD5Update(md5ctxp, (const unsigned char *)pbuf, newinfo.sz);
MD5Final(newinfo.signature, md5ctxp);
/*
* process the single ob in the requested fashion
*/
if((*doit)(&newinfo, pbuf) == 0)
nsplit++;
} /* end while */
#if PRINT
putc('\n', stderr);
#endif
} /* end while block */
} /* end #### block */
return nsplit;
}
