On Mon, Mar 16, 2015 at 12:25 PM, Dumindu Jayasekera <
d.jayasekera@xxxxxxxxxxxxxxxxx> wrote:
This would be a better question for stack overflow, or some other forum,
but:
> with netCDF4.Dataset('TEST_file.nc', mode="w", format='NETCDF4') as ds:
> # some file-level meta-data attributes:
> ds.Conventions = "CF-1.6"
> ds.title = 'Non TC precipitation'
> ds.institution = 'USU'
>
> lat = data[:,0] # the first column
> lon = data[:,1] # the second column
> precip = data[:,2] # the third column
>
> lat = lat.reshape( (161, 321) )
> lon = lon.reshape( (161, 321) )
>
> lat = ds.createDimension('latitude', 161)
> lon = ds.createDimension('longitude', 321)
>
>
> I am getting an error when I run following first two lines.
>
> precip = ds.createVariable('precip', np.float32, ('latitude',
> 'longitude'))
> var[:] = data[:,:]
> ## adds some attributes
> var.units = 'mm'
> var.long_name = 'Precipitation'
>
> Error:
> Traceback (most recent call last):
>
> File "<ipython-input-67-cf31aeae7dd7>", line 1, in <module>
> precip = ds.createVariable('precip', np.float32, ('latitude',
> 'longitude'))
>
> This makes it look like that is the first line in the file -- it won't
work to run it in that order, the nc DataSet object needs to have the
dimensions created before you can use them.
>
> RuntimeError: NetCDF: Not a valid ID
>
>
I think that's what this means -- it'll looking for a dimension names
"latitude", but you haven't created it yet.
the dimension creating code needs to be run first.
HTH,
-Chris
> I appreciate if you can clarify here a bit or provide me some clues of
> what's happening to correct it.
>
> Thanks in advance,
>
> Dumindu.
>
>
> On Wed, Mar 11, 2015 at 3:19 PM, Chris Barker <chris.barker@xxxxxxxx>
> wrote:
>
>> Sorry, I really dont have time to do it for you, but a couple hints:
>>
>> IF you're lucky, you can "reshape" the input arrays in one step:
>>
>> data = np.loadtxt('TEST_file.csv', delimiter=',')
>>
>> lat = data[:,0] # the first column -- if that is latitude
>>
>> lat = lat.reshape( (num_times, num_lats, num_lons) )
>>
>> That _might put it all in the right order, depending on hoe it's written
>> to the file. If that doesn't work, this might:
>>
>> lat = lat.reshape( (num_times, num_lats, num_lons), order='F')
>>
>> If that doesn't work, then you may have to loop through all of by hand,
>> in the right order:
>>
>> new_data = np.zeros( (num_times, num_lats, num_lons) )
>> i = 0
>> for t in range(num_times):
>> for lat in range(num_lats):
>> for lon in range(num_lons):
>> new_data[t, lat, lon] = old_data[i]
>> i += 1
>>
>> You may need to change the order of those loops to match your data.
>>
>> Also,m I see three columns, not four in your sample file -- is each time
>> step in a different file? That wold require you to loop through all the
>> files to load each time...
>>
>> Take a look at python and numpy tutorials online to learn a bit more
>> about all this.
>>
>> -Chris
>>
>>
>>
>>
>> On Wed, Mar 11, 2015 at 12:03 PM, Dumindu Jayasekera <
>> d.jayasekera@xxxxxxxxxxxxxxxxx> wrote:
>>
>>> Chris,
>>>
>>> Thanks again. I am unable to create the 3D array since I have limited
>>> knowledge in python.
>>>
>>> I have attached the csv files, python scripts and sample netCDF file.
>>>
>>> Any help is appreciated.
>>>
>>>
>>>
>>>
>>>
>>> On Wed, Mar 11, 2015 at 8:45 AM, Chris Barker <chris.barker@xxxxxxxx>
>>> wrote:
>>>
>>>> On Tue, Mar 10, 2015 at 5:50 PM, Dumindu Jayasekera <
>>>> d.jayasekera@xxxxxxxxxxxxxxxxx> wrote:
>>>>
>>>>> Thanks again Chris.
>>>>>
>>>>> I think I dont need to core metadata. I was able to produce the .nc
>>>>> file using the code below (as you suggested).
>>>>>
>>>>>
>>>> As Rich suggests, you may be better off using onf the libraries
>>>> suggested, Iris, in particular will get all the complex CF metadaat stuff
>>>> right for you.
>>>>
>>>> But you're this close, so...
>>>>
>>>>
>>>>> But, how can I modify to rename the var1 = lat, var2 = lon, var3 =
>>>>> precipitation in the code below.?
>>>>>
>>>>
>>>> From the nc file you sent, it looks like you want the precipitiaon to
>>>> be a 3-d array: (time X ltitude X longitude), so:
>>>>
>>>>
>>>> # this load the file into a Nx3 array (three columns)
>>>>> data = np.loadtxt('TEST_file.csv', delimiter=',')
>>>>>
>>>>
>>>> note -- you dont have a 3-d array here, you have a 2-d array, which is
>>>> really three vectors -- you will need to re-shuffle these to get the 3-d
>>>> array you want, and the time, lat, and long vectors.... but assumign you've
>>>> done that:
>>>>
>>>>
>>>> you'll need three dimensions:
>>>> time = ds.createDimension('time', num_times)
>>>> lat = ds.createDimension('latitude', num_latitude)
>>>> lon = ds.createDimension('longitude', num_longitude)
>>>>
>>>> Then you want your 3-d precip variable -- I"d call it something more
>>>> readable than "r", but maybe you need that...
>>>>
>>>> r = ds.createVariable('r', np.float32, ('time', 'latitude',
>>>> 'longitude'))
>>>> var[:] = data[:,:,:]
>>>> ## adds some attributes
>>>> var.units = 'mm'
>>>> var.long_name = 'Precipitation'
>>>> ... and the others that you need.
>>>>
>>>> HTH,
>>>>
>>>> -Chris
>>>>
>>>> --
>>>>
>>>> Christopher Barker, Ph.D.
>>>> Oceanographer
>>>>
>>>> Emergency Response Division
>>>> NOAA/NOS/OR&R (206) 526-6959 voice
>>>> 7600 Sand Point Way NE (206) 526-6329 fax
>>>> Seattle, WA 98115 (206) 526-6317 main reception
>>>>
>>>> Chris.Barker@xxxxxxxx
>>>>
>>>
>>>
>>
>>
>> --
>>
>> Christopher Barker, Ph.D.
>> Oceanographer
>>
>> Emergency Response Division
>> NOAA/NOS/OR&R (206) 526-6959 voice
>> 7600 Sand Point Way NE (206) 526-6329 fax
>> Seattle, WA 98115 (206) 526-6317 main reception
>>
>> Chris.Barker@xxxxxxxx
>>
>
>
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception
Chris.Barker@xxxxxxxx
