On Jun 18, 2012, at 4:19 PM, Doug Lindholm wrote:
> Hi,
>
> I've been implementing "filters" for our LaTiS server
> (http://lasp.colorado.edu/lisird/tss.html) using the OPeNDAP (DAP2) URL
> "function" syntax. For example,
>
> http://lasp.colorado.edu/lisird/tss/american_sunspot_number.csv?&thin(10)&format_time(yyyy-MM-dd)
>
> It's not ideal, but it's a start. DAP2 says very little about functions.
> Maybe DAP4 could say more.
>
> I'd also strongly suggest that people think about Functional Programming
> paradigms. I suspect that such formalisms will lend themselves better than an
> OO biased approach.
I've been thinking about just this topic for some time. I am pretty certain
that a functional language is the way to go. While the DAP2 specification say
little about server functions, Hyrax does come with a few and the parser &
evaluator does support composing functions. In addition there are two groups
now working on server functions that perform some significant operations:
unstructured grid subsetting and satellite data reprojection (satellite corrds
to mercator, ...).
I'd certainly be interested in talking about this stuff more.
James
>
> Doug
>
> On 6/18/12 3:56 PM, Roy Mendelssohn wrote:
>> But for a useful service, the form and syntax of the URL should be
>> independent of the mechanism that does the server side calculations
>> (which rules out SWAMP). So for example, both Grads an F-TDS use the
>> same format in the URL to say that "this is an expression", but the
>> expressions themselves are platform specific. That is not the way to
>> get overarching services.
>>
>> Instead, we need agreement on how in the URL request we signal a
>> server-side function, the syntax of that function (independent of the
>> engine underneath) and a few simple functions to start (say simple
>> data transformations, differencing and averaging on a dimension(s)).
>> Then the server back-end can parse the request and use Ferret or
>> Grads or NCO or Python or whatever is desired, and like with any good
>> service, the back end could change without having any affect on the
>> URL or the user.
>>
>> I know Matthew Arrott at least used to like the approach in Chapter
>> 12 of "Python Scripting for Computation Science". But a lot of that
>> is the engine underneath. I am more interested in the form of the
>> URL. Get some agreement on that, and some real implementation could
>> proceed.
>>
>> -Roy
>>
>>
>>
>>
>>
>> On Jun 18, 2012, at 2:37 PM, Russ Rew wrote:
>>
>>> Jeff,
>>>
>>>> However, we have to keep in mind performance ramifications. It
>>>> still takes a long time to move gigabytes of data across a
>>>> network. This brings up the importance of moving the computation
>>>> to the data, instead of moving the data to the computation. For
>>>> some data sets and many use cases remote access to data works
>>>> very well so things like brokering are tractable. However, for
>>>> *big* data sets (e.g., climate model output) we need to come up
>>>> with richer mechanisms (like the NCO on local data) to bring
>>>> computation to the data.
>>>
>>> See Daniel Wang's SWAMP (the Script Workflow Analysis for
>>> MultiProcessing), built on top of NCO:
>>>
>>> https://code.google.com/p/swamp/
>>>
>>> --Russ
>>>
>>> _______________________________________________ thredds mailing
>>> list thredds@xxxxxxxxxxxxxxxx For list information or to
>>> unsubscribe, visit: http://www.unidata.ucar.edu/mailing_lists/
>>
>> ********************** "The contents of this message do not reflect
>> any position of the U.S. Government or NOAA." **********************
>> Roy Mendelssohn Supervisory Operations Research Analyst NOAA/NMFS
>> Environmental Research Division Southwest Fisheries Science Center
>> 1352 Lighthouse Avenue Pacific Grove, CA 93950-2097
>>
>> e-mail: Roy.Mendelssohn@xxxxxxxx (Note new e-mail address) voice:
>> (831)-648-9029 fax: (831)-648-8440 www: http://www.pfeg.noaa.gov/
>>
>> "Old age and treachery will overcome youth and skill." "From those
>> who have been given much, much will be expected" "the arc of the
>> moral universe is long, but it bends toward justice" -MLK Jr.
>>
>> _______________________________________________ thredds mailing list
>> thredds@xxxxxxxxxxxxxxxx For list information or to unsubscribe,
>> visit: http://www.unidata.ucar.edu/mailing_lists/
>
> _______________________________________________
> thredds mailing list
> thredds@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
--
James Gallagher
jgallagher at opendap.org
406.723.8663
