NOTE: The decoders mailing list is no longer active. The list archives are made available for historical reasons.
On Fri, 22 Oct 2004, Dan Swank wrote:
Robb, One final brief question, are the cdl files time independant? (Can we make one and use it as a template for every other narr file)
dan, yes, once made they can be used as long as they are valid for the datasets. gribtocdl is just a utility to assist in making cdls. robb...
Or does a cdl file need to be created for every file we want to convert? -Dan Robb Kambic wrote: >On Thu, 21 Oct 2004, Dan Swank wrote: > > > >>Almost have it. >>The latest version indeed corrects the gribtocdl bug. >>I was able to create the narr-b .nc file going through both steps. >>Next i tried to convert the narr-a files, which have more then 400 grib >>records each. >>gribtocdl wrote this record: >> >> float 03tot_atm(record,y,x) ; >> 03tot_atm:long_name = "Total ozone at entire atmosphere >>considered as a single layer" ; >> 03tot_atm:GRIB_parameter_number = 237 ; >> 03tot_atm:GRIB_level_flag = 200 ; >> 03tot_atm:units = "Kg/m2" ; >> 03tot_atm:_FillValue = -9999.f ; >> 03tot_atm:navigation = "nav" ; >> >>Gribtonc doesn't seem to like the variable name starting with a number >>zero, and throws a syntax error. >> >> > >dan, > >in netcdf no var can start with a number, guess that parameter hasn't been >used before. the problem has been fixed and tested on the file you gave >as a test. sorry but you have to download the latest release: > >decoders-3.0.6.tar.Z > >robb... > >ps the size of netcdf output file grew to ~625mg from ~56 mg input file. >that;s about right > > > > >>This is also strange becuase the NARR doesn' tcontain any ozone fields. >> >>Suggestions? A sample narr-a file to try: >>http://nomads.ncdc.noaa.gov/tmp/narr-a_221_20010101_0000_000.grb >> >>-Dan >> >> >> >>Robb Kambic wrote: >> >> >> >>>On Thu, 21 Oct 2004, Dan Swank wrote: >>> >>> >>> >>> >>> >>>>Robb, >>>> >>>>No luck. I've recompiled the decoders package again and gribtocdl does >>>>the same thing. >>>>We have: >>>>netcdf-3.5.1 (built from source) >>>>udunits-1.11.7 ( Used binary installation since our system had >>>>trouble compiling this from source) >>>>decoders-2.3.8n ( built from source) >>>> >>>> >>>> >>>> >>>> >>>Dan, >>> >>>that's a really old distribution of the decoders package. Your problem >>>sounds like a bug that was fixed back at that time. the latest decoders >>>package is decoders-3.0.5.tar.Z hopefully the newest distribution will fix >>>your problem. >>> >>>robb... >>> >>> >>> >>> >>> >>> >>>>And the output from the compilation can be found here >>>>http://nomads.ncdc.noaa.gov/tmp/decoders-2.3.8n.logs.tar >>>> >>>>OS: >>>>Red Hat Linux release 7.3 (Valhalla) >>>> >>>>We also have a >>>>Red Hat Linux release 8.0 (Psyche) >>>>We can try this if you think it will really make much of a difference. >>>>Avbl but we dont' have root user on either of these so installing >>>>tpackages can be a bit tricky. >>>> >>>>If you need anymore information let me know. >>>> >>>>-Dan >>>> >>>> >>>> >>>>Robb Kambic wrote: >>>> >>>> >>>> >>>> >>>> >>>>>no need to include the others now... >>>>> >>>>> >>>>>On Wed, 20 Oct 2004, Dan Swank wrote: >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>>Getting a little closer... >>>>>> >>>>>>Set the Env var and now udunits works as expected... however gribtocdl >>>>>>still gets stuck in an infinate loop and don't seem to work right. >>>>>>when i ran it on the narr-b GRIB it got to the following section >>>>>> >>>>>> >>>>>>// Navigation >>>>>>nav_model = "GRIB1" ; >>>>>>grid_type_code = 3 ; >>>>>>grid_type = "Lambert conformal projection" ; >>>>>>grid_name = " " ; >>>>>>grid_center = 7 ; >>>>>>grid_number = 221 ; >>>>>>i_dim = "lon" ; >>>>>>j_dim = "lat" ; >>>>>>Nx = 349 ; >>>>>>Ny = 277 ; >>>>>>La1 = 1.000000 ; >>>>>>Lo1 = -145.500000 ; >>>>>>Lov = -107.000000 ; >>>>>>Dx = 32463.000000 ; >>>>>>Dy = 32463.000000 ; >>>>>>// Pole in proj. plane = North ; >>>>>>centers = 1 ; >>>>>>Latin1 = 50.000000 ; >>>>>>Latin2 = 50.000000 ; >>>>>>Splat = 0.000000 ; >>>>>>SPlon = 0.000000 ; >>>>>>ResCompFlag = 0 ; >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>if you notice the cdl i made didn't have lon and lats. there must be >>>>>something wrong with your gribtocdl build. for a test, i build the >>>>>decoders package on a linux box fedora 9 and gribtocdl created the cdl >>>>>fine. remake the distribution, if you can try it on another box. whats >>>>>your os? >>>>> >>>>>robb... >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>>lon = -145.50,-146.50,-147.50,-148.50,-149.50,-150.50,-151.50,-152.50, >>>>>> -153.50,-154.50,-155.50,-156.50,-157.50,-158.50,-159.50,-160.50, >>>>>> -161.50,-162.50,-163.50,-164.50,-165.50,-166.50,-167.50,-168.50, >>>>>> >>>>>>Then continues decrementing the longitue values and don't stop. I was >>>>>>forced to CNTL-C it and remove the lon section completely, putting the >>>>>>curly bracket at the end. Know why it might do this? >>>>>>After setting the env var, gribtonc successfully re-created narr.nc from >>>>>>the narr.cdl you supplied. However when I tried my CDL from above i get >>>>>>the following: >>>>>> >>>>>>./narr-b_221_20010101_0000_000.grb >>>>>>ncgen: narr-b2.cdl line 6: dimension length must be positive >>>>>>ncgen: narr-b2.cdl line 7: dimension length must be positive >>>>>>ncgen: narr-b2.cdl line 73: Nx not declared as a variable, fatal error >>>>>> >>>>>>Does gribtocdl require any ENV vars to be set? >>>>>> >>>>>>Appreciate the help. >>>>>>-Dan >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>Robb Kambic wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>On Wed, 20 Oct 2004, Dan Swank wrote: >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>I copied the narr.cdl file from the FTP and tried: >>>>>>>> >>>>>>>>gribtonc -v -l ./log -e ./error ./narr.cdl >>>>>>>>narr-b_221_20010101_0000_000.nc < narr-b_221_20010101_0000_000.grb >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>here's the process >>>>>>> >>>>>>># creates cdl file narr.cdl >>>>>>>% gribtocdl -v narr-b_221_20010101_0000_000.grb > narr.cdl >>>>>>> >>>>>>># creates netcdf file narr.nc in verbose mode, logging to screen >>>>>>>% gribtonc -vl - narr.cdl narr.nc < narr-b_221_20010101_0000_000.grb >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>Result >>>>>>>>Segmentation fault >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>explained below, set UDUNITS_PATH >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>and >>>>>>>>Oct 20 19:31:05 gribtonc[32594]: Starting Up >>>>>>>>in the ./log file. >>>>>>>> >>>>>>>>Any idea whats going on? It is likely gribtonc (or more likely one of >>>>>>>>its dependancies) is not installed correctly on our system (RedHat 7.3) >>>>>>>>Also, what did you use to create this CDL file? The data in cdl seems >>>>>>>>like a translation of the information in the grib PDS, reworked into a >>>>>>>>format that ncgen can understand. >>>>>>>>Any way i can get anymore debug information regarding this? >>>>>>>> >>>>>>>>Only hunch is that it is not happy with the udunits package >>>>>>>> >>>>>>>>-> /usr/local/udunits-1.11.7/udunits-1.11.7/bin/udunits >>>>>>>>udunits(3): Couldn't open units database "/upc/udunits/etc/udunits.dat": >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>ahh, that's the problem. udunits can't find udunits.dat either place >>>>>>>udunits.dat in dir /upc/udunits/etc/ or set environment var >>>>>>> >>>>>>>% setenv UDUNITS_PATH /your/udunits/path/udunits.dat >>>>>>> >>>>>>>replace /your/udunits/path/ with appropriate path >>>>>>> >>>>>>>robb... >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>>>No such file or directory >>>>>>>>Segmentation fault >>>>>>>> >>>>>>>>But, while building the unidata decoders package it only seemed to want >>>>>>>>the .dat .a and .h files within these packages. >>>>>>>>Would this be the source of the problems? >>>>>>>> >>>>>>>>-Dan >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>Robb Kambic wrote: >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>>>Thanks russ for the clarification. i was assuming you were familar with >>>>>>>>>the decoders process. if i can answer any more questions let me know. i'll >>>>>>>>>try to be more descriptive. >>>>>>>>> >>>>>>>>>robb... >>>>>>>>> >>>>>>>>> >>>>>>>>>On Tue, 19 Oct 2004, Dan Swank wrote: >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>>>Russ >>>>>>>>>> >>>>>>>>>>Actually, i was, at first, trying >>>>>>>>>>GRIB -( gribtocdl )-> CDL -( gribtonc )-> NetCDF >>>>>>>>>>Which i now understand is completely wrong, thanks for the help. >>>>>>>>>>As you have noticed, we are completely unfamiliar with these programs. >>>>>>>>>> >>>>>>>>>>Attempting it the correct way now, i'll let you know how it goes. >>>>>>>>>> >>>>>>>>>>-Dan >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>Russ Rew wrote: >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>>>Robb, >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>>>i downloaded the file from the last message, created a cdl, and decoded >>>>>>>>>>>>the grb file. there might be something wrong with Dan's decoders build or >>>>>>>>>>>>it could be a platform issue. this was done on a solaris box 5.9 The >>>>>>>>>>>>narr.cdl file is attached and the files narr.cdl, narr.grb, and narr.nc are >>>>>>>>>>>>in the Unidata's ftp dir at >>>>>>>>>>>> >>>>>>>>>>>>ftp unidata.ucar.edu >>>>>>>>>>>> >>>>>>>>>>>>% cd pub/contrib >>>>>>>>>>>>% mget narr* >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>Thanks Robb. The file sizes are: >>>>>>>>>>> >>>>>>>>>>>-rw-rw-r-- 1 rkambic ustaff 3486764 Oct 19 13:32 narr.nc >>>>>>>>>>>-rw-rw-r-- 1 rkambic ustaff 8560 Oct 19 13:32 narr.cdl >>>>>>>>>>>-rw-rw-r-- 1 rkambic ustaff 1398914 Oct 19 13:32 narr.grb >>>>>>>>>>> >>>>>>>>>>>so the netCDF file is about 2.5 times as big as the GRIB file. >>>>>>>>>>> >>>>>>>>>>>I'm guessing the source of the problem may come from using >>>>>>>>>>> >>>>>>>>>>>GRIB -> (via gribtocdl) -> CDL -> (via ncgen) -> netCDF >>>>>>>>>>> >>>>>>>>>>>(Using gribtocdl to generate a very large CDL file and then using ncgen >>>>>>>>>>>to convert that into a netCDF file.) >>>>>>>>>>> >>>>>>>>>>>I think Robb used the following tools instead: >>>>>>>>>>> >>>>>>>>>>>GRIB -> (via gribtocdl) -> CDL >>>>>>>>>>>GRIB and CDL -> (via gribtonc) -> netCDF >>>>>>>>>>> >>>>>>>>>>>(Using gribtocdl to generate a small CDL file describing structure of >>>>>>>>>>>the desired netCDF file and then using gribtonc to convert the GRIB >>>>>>>>>>>data into the netCDF file.) >>>>>>>>>>> >>>>>>>>>>>--Russ >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>-- >>>>>>>>>>Dan Swank <dan.swank@xxxxxxxx> >>>>>>>>>>NOMADS programmer >>>>>>>>>>STG, Incorporated - Government Contractor >>>>>>>>>>151 Patton Avenue, Room 514 >>>>>>>>>>Asheville, NC 28801 >>>>>>>>>>Phone: 828-271-4007 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>============================================================================== >>>>>>>>>Robb Kambic Unidata Program Center >>>>>>>>>Software Engineer III Univ. Corp for Atmospheric Research >>>>>>>>>rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ >>>>>>>>>============================================================================== >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>-- >>>>>>>>Dan Swank <dan.swank@xxxxxxxx> >>>>>>>>NOMADS programmer >>>>>>>>STG, Incorporated - Government Contractor >>>>>>>>151 Patton Avenue, Room 514 >>>>>>>>Asheville, NC 28801 >>>>>>>>Phone: 828-271-4007 >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>============================================================================== >>>>>>>Robb Kambic Unidata Program Center >>>>>>>Software Engineer III Univ. Corp for Atmospheric Research >>>>>>>rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ >>>>>>>============================================================================== >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>-- >>>>>>Dan Swank <dan.swank@xxxxxxxx> >>>>>>NOMADS programmer >>>>>>STG, Incorporated - Government Contractor >>>>>>151 Patton Avenue, Room 514 >>>>>>Asheville, NC 28801 >>>>>>Phone: 828-271-4007 >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>============================================================================== >>>>>Robb Kambic Unidata Program Center >>>>>Software Engineer III Univ. Corp for Atmospheric Research >>>>>rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ >>>>>============================================================================== >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>-- >>>>Dan Swank <dan.swank@xxxxxxxx> >>>>NOMADS programmer >>>>STG, Incorporated - Government Contractor >>>>151 Patton Avenue, Room 514 >>>>Asheville, NC 28801 >>>>Phone: 828-271-4007 >>>> >>>> >>>> >>>> >>>> >>>> >>>============================================================================== >>>Robb Kambic Unidata Program Center >>>Software Engineer III Univ. Corp for Atmospheric Research >>>rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ >>>============================================================================== >>> >>> >>> >>> >>-- >>Dan Swank <dan.swank@xxxxxxxx> >>NOMADS programmer >>STG, Incorporated - Government Contractor >>151 Patton Avenue, Room 514 >>Asheville, NC 28801 >>Phone: 828-271-4007 >> >> >> >> > >============================================================================== >Robb Kambic Unidata Program Center >Software Engineer III Univ. Corp for Atmospheric Research >rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ >============================================================================== > > -- Dan Swank <dan.swank@xxxxxxxx> NOMADS programmer STG, Incorporated - Government Contractor 151 Patton Avenue, Room 514 Asheville, NC 28801 Phone: 828-271-4007
============================================================================== Robb Kambic Unidata Program Center Software Engineer III Univ. Corp for Atmospheric Research rkambic@xxxxxxxxxxxxxxxx WWW: http://www.unidata.ucar.edu/ ==============================================================================
decoders archives: