But for a useful service, the form and syntax of the URL should be independent
of the mechanism that does the server side calculations (which rules out
SWAMP). So for example, both Grads an F-TDS use the same format in the URL to
say that "this is an expression", but the expressions themselves are platform
specific. That is not the way to get overarching services.
Instead, we need agreement on how in the URL request we signal a server-side
function, the syntax of that function (independent of the engine underneath)
and a few simple functions to start (say simple data transformations,
differencing and averaging on a dimension(s)). Then the server back-end can
parse the request and use Ferret or Grads or NCO or Python or whatever is
desired, and like with any good service, the back end could change without
having any affect on the URL or the user.
I know Matthew Arrott at least used to like the approach in Chapter 12 of
"Python Scripting for Computation Science". But a lot of that is the engine
underneath. I am more interested in the form of the URL. Get some agreement
on that, and some real implementation could proceed.
-Roy
On Jun 18, 2012, at 2:37 PM, Russ Rew wrote:
> Jeff,
>
>> However, we have to keep in mind performance ramifications. It still takes
>> a long time to move gigabytes of data across a network. This brings up the
>> importance of moving the computation to the data, instead of moving the
>> data to the computation. For some data sets and many use cases remote
>> access to data works very well so things like brokering are tractable.
>> However, for *big* data sets (e.g., climate model output) we need to come
>> up with richer mechanisms (like the NCO on local data) to bring computation
>> to the data.
>
> See Daniel Wang's SWAMP (the Script Workflow Analysis for
> MultiProcessing), built on top of NCO:
>
> https://code.google.com/p/swamp/
>
> --Russ
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
**********************
"The contents of this message do not reflect any position of the U.S.
Government or NOAA."
**********************
Roy Mendelssohn
Supervisory Operations Research Analyst
NOAA/NMFS
Environmental Research Division
Southwest Fisheries Science Center
1352 Lighthouse Avenue
Pacific Grove, CA 93950-2097
e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address)
voice: (831)-648-9029
fax: (831)-648-8440
www: http://www.pfeg.noaa.gov/
"Old age and treachery will overcome youth and skill."
"From those who have been given much, much will be expected"
"the arc of the moral universe is long, but it bends toward justice" -MLK Jr.
