Dumindu,
You can paste a file name together in a few loops like so (check for mistakes!):
yb=1998; ye=2000; % for the years yb through to ye
mb=1; me=12; %% month begin, month end
%% day numbers in the year for times of data
md=[31;28;31;30;31;30;31;31;30;31;30;31]; % days per month
yd=1; for i=2:12; yd(i)=yd(i-1)+md(i-1); end % doy of month start
mc=['jan';'feb';'mar';'apr';'may';'jun';'jul';'aug';'sep';'oct';'nov';'dec']
for iy=yb:ye % for the years
cy=sprintf('%4d',iy) %formated print to the string
for im=mb:me % for the months
if(mod(iy,4)==0 && im==2 ) mtyd=mtyd+1; end % account for leap years
cm1=sprintf('%3d',100+im);
cm=cm1(2:3);
for id=1:md(im) %
for ic=1:8 % eight files per day?
cd1=sprintf('%3d',10000+id*100+ic*3);
cd=cd1(2:5);
fname=[‘TRMM_’,cy(3:4),’_’,cm,’_’,cd,'_newntcl.csv']
% Open the file and assign it a file ID
fid = fopen(fname,'r');
…
End all loops
If that does not work, then pipe all fine names into a file using ls and read
name by name of that file in a loop over all file names. This has the advantage
that you will not try to read missing files and it does not matter whether
there is a pattern to the file name or not.
Best wishes,
Joerg
From: netcdfgroup-bounces@xxxxxxxxxxxxxxxx
[mailto:netcdfgroup-bounces@xxxxxxxxxxxxxxxx] On Behalf Of Dumindu Jayasekera
Sent: 12 March 2015 01:11
To: Chris Barker
Cc: netcdfgroup@xxxxxxxxxxxxxxxx
Subject: Re: [netcdfgroup] Need help to convert .csv files to netCDF files
Chris,
Each time step is a different file. Original final name of the "TEST_file.csv
is TRMM_1998_01_0100_newntcl.csv. So the next file for next time step is
TRMM_1998_01_0103_newntcl.csv, TRMM_1998_01_0106_newntcl.csv,
TRMM_1998_01_0109_newntcl.csv and so on. I have the time steps until
TRMM_1998_02_1512_newntcl.csv.
Do you know how to loop through file names and store in arrays?
Thanks again.
On Wed, Mar 11, 2015 at 3:19 PM, Chris Barker
<chris.barker@xxxxxxxx<mailto:chris.barker@xxxxxxxx>> wrote:
Sorry, I really dont have time to do it for you, but a couple hints:
IF you're lucky, you can "reshape" the input arrays in one step:
data = np.loadtxt('TEST_file.csv', delimiter=',')
lat = data[:,0] # the first column -- if that is latitude
lat = lat.reshape( (num_times, num_lats, num_lons) )
That _might put it all in the right order, depending on hoe it's written to the
file. If that doesn't work, this might:
lat = lat.reshape( (num_times, num_lats, num_lons), order='F')
If that doesn't work, then you may have to loop through all of by hand, in the
right order:
new_data = np.zeros( (num_times, num_lats, num_lons) )
i = 0
for t in range(num_times):
for lat in range(num_lats):
for lon in range(num_lons):
new_data[t, lat, lon] = old_data[i]
i += 1
You may need to change the order of those loops to match your data.
Also,m I see three columns, not four in your sample file -- is each time step
in a different file? That wold require you to loop through all the files to
load each time...
Take a look at python and numpy tutorials online to learn a bit more about all
this.
-Chris
On Wed, Mar 11, 2015 at 12:03 PM, Dumindu Jayasekera
<d.jayasekera@xxxxxxxxxxxxxxxxx<mailto:d.jayasekera@xxxxxxxxxxxxxxxxx>> wrote:
Chris,
Thanks again. I am unable to create the 3D array since I have limited knowledge
in python.
I have attached the csv files, python scripts and sample netCDF file.
Any help is appreciated.
On Wed, Mar 11, 2015 at 8:45 AM, Chris Barker
<chris.barker@xxxxxxxx<mailto:chris.barker@xxxxxxxx>> wrote:
On Tue, Mar 10, 2015 at 5:50 PM, Dumindu Jayasekera
<d.jayasekera@xxxxxxxxxxxxxxxxx<mailto:d.jayasekera@xxxxxxxxxxxxxxxxx>> wrote:
Thanks again Chris.
I think I dont need to core metadata. I was able to produce the .nc file using
the code below (as you suggested).
As Rich suggests, you may be better off using onf the libraries suggested,
Iris, in particular will get all the complex CF metadaat stuff right for you.
But you're this close, so...
But, how can I modify to rename the var1 = lat, var2 = lon, var3 =
precipitation in the code below.?
From the nc file you sent, it looks like you want the precipitiaon to be a 3-d
array: (time X ltitude X longitude), so:
# this load the file into a Nx3 array (three columns)
data = np.loadtxt('TEST_file.csv', delimiter=',')
note -- you dont have a 3-d array here, you have a 2-d array, which is really
three vectors -- you will need to re-shuffle these to get the 3-d array you
want, and the time, lat, and long vectors.... but assumign you've done that:
you'll need three dimensions:
time = ds.createDimension('time', num_times)
lat = ds.createDimension('latitude', num_latitude)
lon = ds.createDimension('longitude', num_longitude)
Then you want your 3-d precip variable -- I"d call it something more readable
than "r", but maybe you need that...
r = ds.createVariable('r', np.float32, ('time', 'latitude', 'longitude'))
var[:] = data[:,:,:]
## adds some attributes
var.units = 'mm'
var.long_name = 'Precipitation'
... and the others that you need.
HTH,
-Chris
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959<tel:%28206%29%20526-6959> voice
7600 Sand Point Way NE (206) 526-6329<tel:%28206%29%20526-6329> fax
Seattle, WA 98115 (206) 526-6317<tel:%28206%29%20526-6317> main
reception
Chris.Barker@xxxxxxxx<mailto:Chris.Barker@xxxxxxxx>
--
Christopher Barker, Ph.D.
Oceanographer
Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959<tel:%28206%29%20526-6959> voice
7600 Sand Point Way NE (206) 526-6329<tel:%28206%29%20526-6329> fax
Seattle, WA 98115 (206) 526-6317<tel:%28206%29%20526-6317> main
reception
Chris.Barker@xxxxxxxx<mailto:Chris.Barker@xxxxxxxx>
