Dear Ryan May
Good afternoon.
Thank you very much for your help, Ryan.
Thanks to your help, we can test several things about the availability of
THREDDS Data Server and AWS to our rainfall processing system.
However, while testing them, other questions arose to me.
Now, our rainfall processing algorithms which are developed fully with c
language have used the LEVEL II data obtained from NCDC, and
read and process them based on NASA's rsl c library. Due to this, LEVEL II data
structure and conventions in our programs are the same as those defined in rsl
library.
Since I realized the advantages of TDS, I have tried to find out a way to
access LEVEL II data through TDS and process them with our algorithms written
in C without downloads of the data.
However, the data structures offered by TDS (OpenDAP and CdmRemote) look
different from LEVEL II data from NCDC; I mean it is not readable using rsl
library.
I think the data offered by HTTPSever have the same structure as those from
NCDC, but it can only be downloaded, not accessed without downloads. Am I right?
So, my question is,
1. Is there any way to read and process LEVEL II data using rls c library
in c program through TDS without downloads of the data?
As I know, if we use OpenDAP c API, it is possible to access LEVEL II data on
OpendDAP server through c program.
However, in this case, as I said, data structure looks different from raw LEVEL
II data, so we need to modify our algorithms a lot since our algorithms are
developed based on rsl c library.
The most ideal case for us is that we access raw LEVEL II data through TDS,
read the data using rsl c library, process them using our c programs, and
download the final result file.
I am wondering whether or not this method is possible. Could you share your
expertise about this problem?
2. If this way is impossible, another possible way may be to use python
to read LEVEL II data through TDS, and connect them to c programs.
This way is similar to the way to use OpenDAP c API, but in this case, all
necessary programs are already developed.
Also, as I know, there is a way to deliver python objects to c program. Am I
right?
However, I question the execution speed of this method.
I think simply downloading through TDS and processing LEVEL II data with our c
programs could be faster, but I cannot assure it. Could you tell me how you
think of this way?
(For this issue, I tested the reading speed using pyart packages. When I access
LEVEL II data through TDS and read them through pyart.io.read_nexrad_cdm,
It takes more time than simply downloading)
3. Finally, I would like to know when I download LEVEL II data through
TDS into AWS EC2 instance, is there any cost for downloading data regardless of
Availability Zone and Region?
I wonder the difference between using TDS and AWS S3 for downloads of the data.
In the case AWS S3, if my EC2 instance is located in different Zone or Region,
some cost is charged.
However, I am not sure about TDS.
I am sorry to list too many things, but if you give me your advice about my
questions, it should be really helpful for me.
Thank you for reading this email and for considering the contents.
Thank you.
Best regards,
Munsung Keem
IIHR-Hydroscience & Engineering
Department of Civil & Environmental Engineering
The University of Iowa
