Re: 20050922:DODS/THREDDS confusion

• To: David Wojtowicz <davidw@xxxxxxxx>
• Subject: Re: 20050922:DODS/THREDDS confusion
• From: John Caron <caron@xxxxxxxxxxxxxxxx>
• Date: Mon, 27 Feb 2006 18:25:43 -0700

David Wojtowicz wrote:

I've pinned the problems down somewhat.

 The actual netCDF file that contains the data on the TDS, has string data
that looks like this (from dncdump on the local file):

---------------------------------
dimensions:
   station = 4818;
   id_len = 4;

variables: char station_id(station, id_len) ;

      station_id:long_name = "Station id" ;
      station_id:reference = "sfmetar_sa.tbl" ;
---------------------------------



However, the TDS represents it via DAP as (pertinent bits of DDS and DAS):


---------------------------------
Dataset {
    String station_id[station = 4818];
}
Attributes {
    station_id {
        String long_name "Station id";
        String reference "sfmetar_sa.tbl";
        DODS {
            Int32 strlen 4;
            String dimName id_len;
        }
    }
}
---------------------------------

So, the two dimensional character array has been converted to a single
dimensional array of strings, whose length is stored in the nested attribute
"DODS".

When you do dncdump on the dods URL, you get:

---------------------------------
dimensions:
        station = 4818 ;

variables: char station_id(station) ;

                station_id:long_name = "Station id" ;
                station_id:reference = "sfmetar_sa.tbl" ;
                station_id:DODS
dncdump: Attribute not found  (program terminates)
----------------------------------


Dncdump aborts when it gets to the nested attribute, "DODS" because a nested
attribute is represented as a container and the Attr_container type is
something that the code in Dattr.cc can't deal with.

Now, even if it could deal with nested attributes, there'd still be the
problem of figuring out what to do with the strings.

I temporarily hacked around the Attr_container problem and had it pretend it
was an empty character array....i.e.  station_ID:DODS = "" ;   essentially
making it harmless to the rest of the netCDF API.

dncdump ran through then without aborting, but if I asked it to dump
"station_id"  (-v station_id option) it'd get the strings all wrong because
the second dimension, id_len, isn't there.

So, there are two problems:

 1) Can't deal with nested attributes (i.e. containers)
 2) Can't deal with DAP strings


Ideally the output from the DAP network version should look just like the
output from the local file version... the differences become transparent.

This would involve fixing the attribute handling code to not pass through
the nested attribute containers to the netCDF API (like the Java version

does).

It would also involve converting the string arrays back to two-dimensional

character arrays as far as the netCDF API is concerned.

I've already been hacking around in the code and thinking about this for a
while.   If you'd like I could take a first shot at it.

So are you just fixing dncdump or the dods C++ client library or ??

BTW, in tracking this, i discovered bugs in both the TDS and the nj22 client 
library, around arrys of Strings, so if you are using those, get the latest 
version! Also theres an intermittent bug in the TDS for metar station files. Im 
waiting for it to happpen again so i can get a stacktrace, so pound away!