Chris,
I created the csv file for a single day and I was able to figure out the
lat, lon dimension declaration as indicated in the following code. But, I
think I am close to obtain the netCDF file.
import numpy as np
import netCDF4
import os
# load the data
path='C:\Users\.spyder2'
os.chdir(path)
# this load the file into a Nx3 array (three columns)
data = np.loadtxt('TRMM_1998_01_0100_newntcl.csv', delimiter=',')
# create a netcdf Data object
with netCDF4.Dataset('TEST_file.nc', mode="w", format='NETCDF4') as ds:
# some file-level meta-data attributes:
ds.Conventions = "CF-1.6"
ds.title = 'Non TC precipitation'
ds.institution = 'USU'
lat = data[:,0] # the first column
lon = data[:,1] # the second column
precip = data[:,2] # the third column
lat = lat.reshape( (161, 321) )
lon = lon.reshape( (161, 321) )
lat = ds.createDimension('latitude', 161)
lon = ds.createDimension('longitude', 321)
I am getting an error when I run following first two lines.
precip = ds.createVariable('precip', np.float32, ('latitude',
'longitude'))
var[:] = data[:,:]
## adds some attributes
var.units = 'mm'
var.long_name = 'Precipitation'
Error:
Traceback (most recent call last):
File "<ipython-input-67-cf31aeae7dd7>", line 1, in <module>
precip = ds.createVariable('precip', np.float32, ('latitude',
'longitude'))
File "netCDF4.pyx", line 1721, in netCDF4.Dataset.createVariable
(netCDF4.c:22247)
File "netCDF4.pyx", line 2313, in netCDF4.Variable.__init__
(netCDF4.c:28523)
RuntimeError: NetCDF: Not a valid ID
I appreciate if you can clarify here a bit or provide me some clues of
what's happening to correct it.
Thanks in advance,
Dumindu.
On Wed, Mar 11, 2015 at 3:19 PM, Chris Barker <chris.barker@xxxxxxxx> wrote:
> Sorry, I really dont have time to do it for you, but a couple hints:
>
> IF you're lucky, you can "reshape" the input arrays in one step:
>
> data = np.loadtxt('TEST_file.csv', delimiter=',')
>
> lat = data[:,0] # the first column -- if that is latitude
>
> lat = lat.reshape( (num_times, num_lats, num_lons) )
>
> That _might put it all in the right order, depending on hoe it's written
> to the file. If that doesn't work, this might:
>
> lat = lat.reshape( (num_times, num_lats, num_lons), order='F')
>
> If that doesn't work, then you may have to loop through all of by hand,
> in the right order:
>
> new_data = np.zeros( (num_times, num_lats, num_lons) )
> i = 0
> for t in range(num_times):
> for lat in range(num_lats):
> for lon in range(num_lons):
> new_data[t, lat, lon] = old_data[i]
> i += 1
>
> You may need to change the order of those loops to match your data.
>
> Also,m I see three columns, not four in your sample file -- is each time
> step in a different file? That wold require you to loop through all the
> files to load each time...
>
> Take a look at python and numpy tutorials online to learn a bit more about
> all this.
>
> -Chris
>
>
>
>
> On Wed, Mar 11, 2015 at 12:03 PM, Dumindu Jayasekera <
> d.jayasekera@xxxxxxxxxxxxxxxxx> wrote:
>
>> Chris,
>>
>> Thanks again. I am unable to create the 3D array since I have limited
>> knowledge in python.
>>
>> I have attached the csv files, python scripts and sample netCDF file.
>>
>> Any help is appreciated.
>>
>>
>>
>>
>>
>> On Wed, Mar 11, 2015 at 8:45 AM, Chris Barker <chris.barker@xxxxxxxx>
>> wrote:
>>
>>> On Tue, Mar 10, 2015 at 5:50 PM, Dumindu Jayasekera <
>>> d.jayasekera@xxxxxxxxxxxxxxxxx> wrote:
>>>
>>>> Thanks again Chris.
>>>>
>>>> I think I dont need to core metadata. I was able to produce the .nc
>>>> file using the code below (as you suggested).
>>>>
>>>>
>>> As Rich suggests, you may be better off using onf the libraries
>>> suggested, Iris, in particular will get all the complex CF metadaat stuff
>>> right for you.
>>>
>>> But you're this close, so...
>>>
>>>
>>>> But, how can I modify to rename the var1 = lat, var2 = lon, var3 =
>>>> precipitation in the code below.?
>>>>
>>>
>>> From the nc file you sent, it looks like you want the precipitiaon to
>>> be a 3-d array: (time X ltitude X longitude), so:
>>>
>>>
>>> # this load the file into a Nx3 array (three columns)
>>>> data = np.loadtxt('TEST_file.csv', delimiter=',')
>>>>
>>>
>>> note -- you dont have a 3-d array here, you have a 2-d array, which is
>>> really three vectors -- you will need to re-shuffle these to get the 3-d
>>> array you want, and the time, lat, and long vectors.... but assumign you've
>>> done that:
>>>
>>>
>>> you'll need three dimensions:
>>> time = ds.createDimension('time', num_times)
>>> lat = ds.createDimension('latitude', num_latitude)
>>> lon = ds.createDimension('longitude', num_longitude)
>>>
>>> Then you want your 3-d precip variable -- I"d call it something more
>>> readable than "r", but maybe you need that...
>>>
>>> r = ds.createVariable('r', np.float32, ('time', 'latitude', 'longitude'))
>>> var[:] = data[:,:,:]
>>> ## adds some attributes
>>> var.units = 'mm'
>>> var.long_name = 'Precipitation'
>>> ... and the others that you need.
>>>
>>> HTH,
>>>
>>> -Chris
>>>
>>> --
>>>
>>> Christopher Barker, Ph.D.
>>> Oceanographer
>>>
>>> Emergency Response Division
>>> NOAA/NOS/OR&R (206) 526-6959 voice
>>> 7600 Sand Point Way NE (206) 526-6329 fax
>>> Seattle, WA 98115 (206) 526-6317 main reception
>>>
>>> Chris.Barker@xxxxxxxx
>>>
>>
>>
>
>
> --
>
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R (206) 526-6959 voice
> 7600 Sand Point Way NE (206) 526-6329 fax
> Seattle, WA 98115 (206) 526-6317 main reception
>
> Chris.Barker@xxxxxxxx
>
