[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[netCDF #SIY-529931]: Merging Netcdf files

Hi Olumide,

Since I responded to a separate emailed version of this question, I'm including
that response here, so the question and answer can be found with a search of
our support archives.  Below that, Dennis Shea's detailed response is included.

> I have weather datasets from the TRMM 3-hourly data in Netcdf format. I am
> looking for ways to merge these multiple netcdf files that have same
> variables and are in sequential time periods. Such as 3B42.199801.3hr_V7.nc,
> 3B42.199803.3hr_V7.nc, 3B42.199802.3hr_V7.nc.
> 
> I would like to merge these files together with time component added in the
> file. I would appreciate your suggestions.
> 
> I would also like you to help me CDO scripts to merge these Netcdf files
> together.
> 
> Thanks!

Although CDO scripts might be able to do what you want, the NCO tool named 
ncrcat should work, 
assuming each input file has a time dimension and time is a record dimension.

Dennis Shea, an expert at processing TRMM data, might be able to provide more 
advice about using 
NCL or NCO tools to merge the 3-hourly TRMM files.  I'm Cc:ing him in this 
reply.

--Russ

Dennis Shea's reply:

[1] Assuming that the ' 3B42.199801.3hr_V7.nc ' has an unlimited time
    dimension, the NCO operator 'ncrcat' is the recommended tool to
    concatenate multiple files.  I am not an expert on the CDO but, in
    my opinion, the NCO are better suited for concatenating files than
    the CDO.

     %> ncrcat -O -h   3B42.1998*.3hr_V7.nc    V7_3hr.3B42.1998.nc

    would efficiently create one file with all variables. Note ... a
    file containing all 3hrly data for one year would be *huge*. There
    would be 8*365 time steps on a 400x1440 grid for 6 variables!
    Figure it out for yourself.

[2] One month of 3-hrly TRMM precip contains (say) 248 (8*31) time
    steps and is about 571MB. This includes *only* precipitation and
    no other variables.  NO: 'err', 'source', 'hqp' , 'irp',
    'satobstime' variables.
     
[3] How do I know? I created monthly files containing the
    precipitation variable *only*.  A modeling project I am working on
    required some additional information so I used an NCL script to
    perform some tasks and then invoked an NCO operator from with ine
    NCL script to create the files. A sample dump follows:


    %> ncdump -h 3B42.200301.3hr_V7

netcdf \3B42.200301.3hr_V7 {
dimensions:
        time = UNLIMITED ; // (248 currently)
        lat = 400 ;
        lon = 1440 ;
variables:
        double time(time) ;
                time:calendar = "gregorian" ;
                time:units = "hours since 1997-01-01 00:00:0.0" ;
        double lat(lat) ;
                lat:long_name = "Latitude" ;
                lat:units = "degrees_north" ;
        double lon(lon) ;
                lon:long_name = "Longitude" ;
                lon:units = "degrees_east" ;
        int date(time) ;
                date:units = "yyyymmdd" ;
                date:long_name = "current date" ;
        int datesec(time) ;
                datesec:units = "seconds" ;
                datesec:long_name = "current seconds of current date" ;
        int yyyymmddhh(time) ;
                yyyymmddhh:units = "current date" ;
                yyyymmddhh:long_name = "current date with hh" ;
        float precip(time, lat, lon) ;
                precip:avg_period = "0000-00-00 01:00:00" ;
                precip:delta_t = "0000-00-00 03:00:00" ;
                precip:comments = "Unknown1 variable comment" ;
                precip:long_name = "precipitation" ;
                precip:units = "" ;
                precip:time_statistic = "instantaneous" ;
                precip:missing_value = -9999.9f ;
                precip:_FillValue = -9999.9f ;

// global attributes:
                :creation_date = "Mon Mar  3 09:28:42 MST 2014" ;
                :Conventions = "None" ;
                :time_info = "\n",
                        "NASA:                                                  
                  \n",
                        "The 0Z in the filename is the mid point of a 3 hour    
                  \n",
                        "interval beginning at 22:30Z (1/14/1998) and ending at 
1:30Z (1/15/1998).\n",
                        "The precipitation is the average rain rate for this 
time period.         \n",
                        "" ;
                :source_URL = "\n",
                        
"http://mirador.gsfc.nasa.gov/cgi-bin/mirador/collectionlist.pl?keyword=TRMM\n";,
                        "" ;
                :source_data = "\n",
                        "Download netCDF from NASA MIRADOR: Search for:\n",
                        "TRMM 3-Hourly 0.25 deg. TRMM and Other-GPI Calibration 
Rainfall Data (TRMM_3B42)\n",
                        "" ;
                :title = "TRMM: 3B42: V7" ;
}

[4] gzip'd versions of these files are available via ftp. Each file is
    about 51MB 

ftp ftp.cgd.ucar.edu
anonymous
email
cd /archive/PRECIP
prompt
mget 3B42.*.3hr_V7.nc.gz
quit

If you want only specific years and or months. you can use linux/unix
'wild' notation


There is also a simgle file which contans *monthly mean* precip

TRMM_3B43V7.19980101-20140201.nc.gz

============================================

Russ Rew                                         UCAR Unidata Program
address@hidden                      http://www.unidata.ucar.edu



Ticket Details
===================
Ticket ID: SIY-529931
Department: Support netCDF
Priority: Normal
Status: Closed