[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [Fwd: NSF Weather Station Obs]

Hi Dave,
This proposal is really interesting! As Cliff Jacobs suggested, it seems appropriate for you to discuss at the Unidata Users Committee level. The next Users Committee takes place in Boulder on 14-15 March. Would you be available to attend that meeting? Information on the Unidata Users Committee can be found at: 

<http://www.unidata.ucar.edu/community/committees/users/users.index.html>
I've cc'd Rich Clark, Users Committee Chair, and several others on this email to solicit the right information for the committee. The more information available, the easier it is for committee discussion. As you know, one of the main considerations Unidata has is the issue of Support. There are so many exciting opportunities, but Support can be a major sink of time. Conversely, Unidata wants to assist, where feasible, in providing services for our university community to facilitate their roles in the community and nation. 

Some things that come to mind are:

        - how do you see the data delivered, e.g., netCDF
        - frequency of data delivery, e.g., near real time, once every 24 hrs
- the NSF Cliffometer and related information is not considered to be "operational" - would all CWOP data be available to Unidata university sites, if they are interested
There are many other things to discuss, Dave, so perhaps we can continue the discussion during one of the breaks at the Unidata Policy Committee next Monday and Tuesday in Arlington. 

CWOP Background Page (Joe Schmidt)

<http://www.fiu.edu/orgs/w4ehw/CWOP-Main.html> is not functional
If others on this email wish to pose questions or issues, or further distribution of the email is warranted, please forward it (unless Dave Helms objects). 

Thank again!

Linda
--On Tuesday, February 19, 2002 9:51 AM -0500 David Helms <address@hidden> wrote: 


Linda,

I noticed Cliff Jacobs had an automated weather station at the NSF in
Alexandria, VA. and asked if he would consider adding the data to the
Civilian Weather Observer Program (CWOP) collected at Forecast Systems
Lab in Boulder, CO (Patty Miller is my contact at FSL).    After an
interaction or two, Cliff suggested I make a presentation to the UNIDATA
folks to establish a means of adding the 160 NSF university observing
systems to CWOP as well as other issues I outlined.

What do you think?

Dave Helms


_____________________________

from: Cliff Jacobs..

Dave,
Thank you for your very informative reply.  And, yes, we did meet at the
last PC meeting, but I did not connect your email with Unidata.  From time
to time I get emails about the WX station at NSF -- they are usually
complaining that the station is not up and that I should be shot at sunrise
for not providing 100% coverage.
The information in your email would be of interest to Unidata.  Linda Miller
would be the best judge of that.  I suggest that you send your message to
Linda with the note that we corresponded and I suggested that a presentation
to the Unidata User's committee would be useful.
I will see you next week at the PC.
Cliff

-----Original Message-----
From: Sandy and David Helms [mailto:address@hidden]
Sent: Monday, February 18, 2002 9:56 PM
To: Jacobs, Clifford A.
Cc: David Helms; David Helms
Subject: RE: NSF Weather Station Obs

Cliff,

Thanks for responding.  I am the NWS representative to UNIDATA and have
assisted Linda Miller in getting CONDUIT on the Next Generation Internet
(NGI) with the highest resolution NCEP grids.  I'm also assisting Linda in
getting the UNIDATA universities access to WSR-88D base data (archive level
2) by collaborating with Kelvin Drogemeier and the CRAFT folks.  Didn't I
meet you at the last UNIDATA in Boulder?  I'll be at this month's UNIDATA
exec meeting in Arlington and hope to see you again.

The people in CWOP don't support the program full-time, it just a hobby for
the almost 500 members collectively disseminating over 20,000 observations /
day (not bad for volunteer e-observers).  Adding the 160 university UNIDATA
observations would be very helpful in testing the sensitivity of local
numerical and diagnostic models to dense mesonets.  The other issue to test
in these heterogeneous mesonets is the quality control of non-standard data
many of which we will not know the precise metadata except for maybe a
LAT/LON.  Are these data useless or can assimilation schemes be developed to
maximize the benefits of these data while minimizing error introduced to the
model from them?

The NWS and FAA have an ASOS airport network containing about 900 sensors;
we also have about 200 CMANs and BUOYs adding to the total.  We are trying
to increase the density of observations by modernizing the Cooperative
Observer (COOP) Network with what we hope will eventually be 8,000 hourly
reporting stations (about one every 30Km) by about 2008.  Even if we are
fully successful, I don't believe we will have nearly enough surface
observations to calibrate satellite and radar data and characterize small
scale meteorological phenomena.  The solution I feel will be necessary is to
leverage the rapidly growing non-NWS mesonets.  Along these lines, we are
engaging the State Departments of Transportation (DOT) to access their Road
Weather Information Systems (RWIS), 2,000 stations and growing. We are also
negotiating with the Automated Weather System (AWS) "school net" people to
gain access to their >8,000 stations.  In this light, the NSF/UNIDATA
observations would fill a significant piece in the network because of the
expertise NSF and its universities bring to the table.  Its the combination
of calibrated NWS sensors and ad hoc sensors that will allow us to reach
30,000 stations in just a few years.

Here's my pitch on CWOP:

Its cheap and easy to contribute data to FSL (Russ Chadwick and Patty Miller
are my buddies there).  The software is free (Weather Display Lite); you
need an open COM port on a windows PC (some use LINUX and MAC) and an
Internet connection.  It will be a great way bridging the academic gap
between the universities and the public.  There is so much ignorance in the
general public about the atmospheric sciences, this would be a great way for
the NSF universities to spread some wisdom into the community.  This is why
I want the NWS to collaborate with the AWS School Net people so we can help
the 8,000 schools that participate in their program.  NASA Goddard is doing
exactly this with the NYC through its "METNET" activities,
http://muspin.gsfc.nasa.gov/nrts/ccny/ccny.html .

Here's the CWOP URLs:
CWOP Background Page (Joe Schmidt)
http://www.fiu.edu/orgs/w4ehw/CWOP-Main.html
APRSWXNET Server - CWOP Data Collection and Routing (Steve Demise)
http://www.findu.com/citizenweather/
FSL Mesonet Page (Patty Miller), select the "ASPRWXNET" data source
http://www-frd.fsl.noaa.gov/mesonet/
CWOP Quality Control Page (Russ Chadwick)
http://www.peakpeak.com/~kb0tvj/


Regards,

Dave Helms


-----Original Message-----
From: Jacobs, Clifford A. [mailto:address@hidden]
Sent: Monday, February 18, 2002 12:40 PM
To: 'HELMS '; Jacobs, Clifford A.
Subject: RE: NSF Weather Station Obs


Dave,

Thank you for your interest in the NSF WX station.  This station was set up
with the help of National Center for Atmospheric Research (NCAR) which is
sponsored by the NSF.  It was set up as an informal station that provides a
well thought out interface readily accessable over the Internet.  It is used
mostly internally at NSF and we do not have the staff to do anyting beyound
what you see. Although I know the folks at FSL, I have not made them aware
of the NSF station.  However, NCAR colleagues probably have.

All the data available on the http://weather.geo.nsf.gov/ is distributed
over the Internet via technology developed by the NSF sponsored Unidata
program.  NSF does not maintain stations other than the one on the roof of
NSF headquarters.  However, additional observations are taken at many of the
160 university departments participating in the Unidata program.  These
observations are distributed over the Internet in near real time to other
Unidata sites.  We are present working on making observations from
Antarctica available to the Unidata community.

Regards,

Cliff

-----Original Message-----
From: HELMS
To: address@hidden
Sent: 2/17/02 9:19 PM
Subject: NSF Weather Station Obs

Cliff,

I visited your NSF weather station in Alexanderia, VA
(http://atm.geo.nsf.gov/cgi-bin/nsfweatherE).  Ever try and
disseminate the data to NOAA? Forecast Systems Lab and the
Ham Radio folks help disseminate data through the Citizen
Weather Observer Program (CWOP). Does NSF maintain any
other surface obs stations (other than Boulder)?  Your info
could be useful!

If intersted, let me know and I'll give you info on how you
can connect to CWOP.

Regards,
Dave Helms
CW0351
http://dhelms.mystarband.net/wx.html
NWS Email:  address@hidden


Linda Miller - address@hidden
External Liaison, Unidata
University Corporation for Atmospheric Research
P.O. Box 3000
Boulder, CO 80307-3000
303 497-8646 fax: 303-497-8690
URL:  http://www.unidata.ucar.edu/staff/lmiller/un.act.html


From address@hidden Tue Feb 19 10:28:07 2002
To: Linda Miller <address@hidden>
CC: Patty Miller <address@hidden>, address@hidden,
  address@hidden, address@hidden, address@hidden,
  address@hidden
Subject: Re: [Fwd: NSF Weather Station Obs]


Linda,

Like the "Cliff-O-Meter", the CWOP data are not guaranteed to be
operational either. Its much like most data on the Internet (and
UNIDATA!), available most of the information will be available most of
the time for a very small amount of money because of the collaborative
nature of the data exchange avoiding the cost of "operationalizing" the
data stream.

Here's some info on the CWOP/MADIS Observations:

   - Data are formatted into ASCII text using LDAD SHEF/hydro format
     (as volume grows, maybe BUFR format will be more appropriate???)
   - Data are sent to the FINDU server at the time interval the owner
     selects (I send mind every 5 minutes); FSL collects whatever data
     are available every 15 minutes
   - Data are not considered to be operational, but collectively can
     simulate a poor man's mesonet
   - Dissemination of CWOP is not restricted; I suggest using  FSL as
     an access point for obtaining the data

To the greatest extent possible, I'd like to see the UNIDATA
universities contributing to their data to the mesonet data collective
as well as consumers of the data.  Focus areas that could be
investigated by distributing these data are:

   - NWP data assimilation strategies and sensitivities using non-NWS mesonets
   - Costs/benefits of very high resolution ad hoc mesonets
   - Outreach to secondary schools and the public to support atmospheric
     science education and awareness

I'll coordinate with FSL on where we go from here before committing to
attend the Policy Meeting.  We all don't need another labor sink that
isn't consistent with our primary roles.  Its my sense that this would
be a voluntary collaboration requiring minimal resources to startup and
maintain (low risk, moderate pay-off).

Regards,
Dave Helms


From address@hidden Wed Feb 20 09:37:39 2002
Subject: Re: NSF Weather Station Obs
To: address@hidden
Date: Wed, 20 Feb 2002 09:37:37 -0700 (MST)
Cc: address@hidden, address@hidden,
  address@hidden, address@hidden, address@hidden



                                       20 Feb 2002


Linda and all,

FSL would be happy to work with you to add data to our CWOP data
stream, and also to provide Unidata with access to our full mesonet
database.

Just to provide you with some additional information:  CWOP is
one of several mesonets that FSL is bringing in as part of our MADIS
project.  The data are ingested, quality controlled, and then
integrated into netCDF files.  We have nearly 4000 stations now that
make up the mesonet database, and we add to it weekly.  Current
mesonets, in addition to CWOP, include MesoWest, RAWS, Iowa DOT,
Minnesota DOT, Florida Automated Weather Network (FAWN), and many
others.  The data is used here for research and development, and we
also share it with other organizations upon request, e.g. NCEP/EMC,
NCAR/RAP, UCAR/COMET, and several WFOs are already getting data from
us.  Although none of the mesonet data is considered "operational", it
is becoming quite popular... several universities have also asked
about the MADIS data, so Unidata would be a good connection for us.

The mesonet data is distributed in netCDF files that are updated every
5 minutes with new observations and QC info, and these files are
already available via LDM - so it would be easy to send them to you.
The files are also directly displayable on AWIPS and AWIPS-like
systems (e.g. FX-Net), and we also provide software support, i.e. we
provide software to read the files.  This software also supports NWS
observations, so you can, for example, read in *all* surface obs
(mesonets, METARs, SAOs, maritime data) with one subroutine call, or
just as easily pick out one or more subproviders, e.g. Minnesota DOT
and ASOS.  In the near future, we plan on hooking up the MADIS data
stream (both surface and upper-air obs) and the MADIS software to
WRF, in order to promote research with the WRF data assimilation system
- another thing Unidata might be interested in.
Let me know if I can answer any questions. For a list of current mesonet stations, see www-sdd.fsl.noaa.gov/MADIS/mesonet_providers.html. 
See www-frd.fsl.noaa.gov/mesonet for real-time MADIS surface observations,
and www-sdd.fsl.noaa.gov for general MADIS information.

Thanks,
Patty 
Date: Tue, 19 Feb 2002 12:28:05 -0500
From: "David Helms" <address@hidden>
Organization: Science Plans Branch (W/ST12)
X-Mailer: Mozilla 4.77 [en] (WinNT; U)
X-Accept-Language: en
MIME-Version: 1.0
To: Linda Miller <address@hidden>
CC: Patty Miller <address@hidden>, address@hidden,
       address@hidden, address@hidden,
       address@hidden, address@hidden
Subject: Re: [Fwd: NSF Weather Station Obs]
References: <address@hidden>
Content-Type: multipart/mixed;
boundary="------------44FF73A4005E34DD14066E4D"
Status: OR

This is a multi-part message in MIME format.
--------------44FF73A4005E34DD14066E4D
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

Linda,


Like the "Cliff-O-Meter", the CWOP data are not guaranteed to be
operational either. Its much like most data on the Internet (and
UNIDATA!), available most of the information will be available most of
the time for a very small amount of money because of the collaborative
nature of the data exchange avoiding the cost of "operationalizing"
the data stream.

Here's some info on the CWOP/MADIS Observations: - Data are formatted
into ASCII text using LDAD SHEF/hydro format (as volume grows, maybe
BUFR format will be more appropriate???)  - Data are sent to the FINDU
server at the time interval the owner selects (I send mind every 5
minutes); FSL collects whatever data are available every 15 minutes -
Data are not considered to be operational, but collectively can
simulate a poor man's mesonet - Dissemination of CWOP is not
restricted; I suggest using FSL as an access point for obtaining the
data

To the greatest extent possible, I'd like to see the UNIDATA
universities contributing to their data to the mesonet data collective
as well as consumers of the data.  Focus areas that could be
investigated by distributing these data are: - NWP data assimilation
strategies and sensitivities using non-NWS mesonets - Costs/benefits
of very high resolution ad hoc mesonets - Outreach to secondary
schools and the public to support atmospheric science education and
awareness

I'll coordinate with FSL on where we go from here before committing to
attend the Policy Meeting.  We all don't need another labor sink that
isn't consistent with our primary roles.  Its my sense that this would
be a voluntary collaboration requiring minimal resources to startup
and maintain (low risk, moderate pay-off).


Regards,

Dave Helms



Linda Miller wrote:


Hi Dave,

This proposal is really interesting!  As Cliff Jacobs suggested, it seems
appropriate for you to discuss at the Unidata Users Committee level.  The next
Users Committee takes place in Boulder on 14-15 March.  Would you be available
to attend that meeting?  Information on the Unidata Users Committee can be
found at:

<http://www.unidata.ucar.edu/community/committees/users/users.index.html>

I've cc'd Rich Clark, Users Committee Chair, and several others on this email
to solicit the right information for the committee.  The more information
available, the easier it is for committee discussion.  As you know, one of the
main considerations Unidata has is the issue of Support.  There are so many
exciting opportunities, but Support can be a major sink of time.  Conversely,
Unidata wants to assist, where feasible, in providing services for our
university community to facilitate their roles in the community and nation.

Some things that come to mind are:

        - how do you see the data delivered, e.g., netCDF
        - frequency of data delivery, e.g., near real time, once every 24 hrs
        - the NSF Cliffometer and related information is not considered to be
        "operational"
        - would all CWOP data be available to Unidata university sites, if they 
        are
interested

There are many other things to discuss, Dave, so perhaps we can continue the
discussion during one of the breaks at the Unidata Policy Committee next Monday
and Tuesday in Arlington.

CWOP Background Page (Joe Schmidt)

<http://www.fiu.edu/orgs/w4ehw/CWOP-Main.html> is not functional

If others on this email wish to pose questions or issues, or further
distribution of the email is warranted, please forward it (unless Dave Helms
objects).

Thank again!

Linda

--On Tuesday, February 19, 2002 9:51 AM -0500 David Helms
<address@hidden> wrote:

> Linda,
>
> I noticed Cliff Jacobs had an automated weather station at the NSF in
> Alexandria, VA. and asked if he would consider adding the data to the
> Civilian Weather Observer Program (CWOP) collected at Forecast Systems
> Lab in Boulder, CO (Patty Miller is my contact at FSL).    After an
> interaction or two, Cliff suggested I make a presentation to the UNIDATA
> folks to establish a means of adding the 160 NSF university observing
> systems to CWOP as well as other issues I outlined.
>
> What do you think?
>
> Dave Helms

_____________________________

from: Cliff Jacobs..

Dave,
Thank you for your very informative reply.  And, yes, we did meet at the
last PC meeting, but I did not connect your email with Unidata.  From time
to time I get emails about the WX station at NSF -- they are usually
complaining that the station is not up and that I should be shot at sunrise

?> for not providing 100% coverage.

The information in your email would be of interest to Unidata.  Linda Miller
would be the best judge of that.  I suggest that you send your message to
Linda with the note that we corresponded and I suggested that a presentation
to the Unidata User's committee would be useful.
I will see you next week at the PC.
Cliff

-----Original Message-----
From: Sandy and David Helms [mailto:address@hidden]
Sent: Monday, February 18, 2002 9:56 PM
To: Jacobs, Clifford A.
Cc: David Helms; David Helms
Subject: RE: NSF Weather Station Obs

Cliff,

Thanks for responding.  I am the NWS representative to UNIDATA and have
assisted Linda Miller in getting CONDUIT on the Next Generation Internet
(NGI) with the highest resolution NCEP grids.  I'm also assisting Linda in
getting the UNIDATA universities access to WSR-88D base data (archive level
2) by collaborating with Kelvin Drogemeier and the CRAFT folks.  Didn't I
meet you at the last UNIDATA in Boulder?  I'll be at this month's UNIDATA
exec meeting in Arlington and hope to see you again.

The people in CWOP don't support the program full-time, it just a hobby for
the almost 500 members collectively disseminating over 20,000 observations /
day (not bad for volunteer e-observers).  Adding the 160 university UNIDATA
observations would be very helpful in testing the sensitivity of local
numerical and diagnostic models to dense mesonets.  The other issue to test
in these heterogeneous mesonets is the quality control of non-standard data
many of which we will not know the precise metadata except for maybe a
LAT/LON.  Are these data useless or can assimilation schemes be developed to
maximize the benefits of these data while minimizing error introduced to the
model from them?

The NWS and FAA have an ASOS airport network containing about 900 sensors;
we also have about 200 CMANs and BUOYs adding to the total.  We are trying
to increase the density of observations by modernizing the Cooperative
Observer (COOP) Network with what we hope will eventually be 8,000 hourly
reporting stations (about one every 30Km) by about 2008.  Even if we are
fully successful, I don't believe we will have nearly enough surface
observations to calibrate satellite and radar data and characterize small
scale meteorological phenomena.  The solution I feel will be necessary is to
leverage the rapidly growing non-NWS mesonets.  Along these lines, we are
engaging the State Departments of Transportation (DOT) to access their Road
Weather Information Systems (RWIS), 2,000 stations and growing. We are also
negotiating with the Automated Weather System (AWS) "school net" people to
gain access to their >8,000 stations.  In this light, the NSF/UNIDATA
observations would fill a significant piece in the network because of the
expertise NSF and its universities bring to the table.  Its the combination
of calibrated NWS sensors and ad hoc sensors that will allow us to reach
30,000 stations in just a few years.

Here's my pitch on CWOP:

Its cheap and easy to contribute data to FSL (Russ Chadwick and Patty Miller
are my buddies there).  The software is free (Weather Display Lite); you
need an open COM port on a windows PC (some use LINUX and MAC) and an
Internet connection.  It will be a great way bridging the academic gap
between the universities and the public.  There is so much ignorance in the
general public about the atmospheric sciences, this would be a great way for
the NSF universities to spread some wisdom into the community.  This is why
I want the NWS to collaborate with the AWS School Net people so we can help
the 8,000 schools that participate in their program.  NASA Goddard is doing
exactly this with the NYC through its "METNET" activities,
http://muspin.gsfc.nasa.gov/nrts/ccny/ccny.html .

Here's the CWOP URLs:
CWOP Background Page (Joe Schmidt)
http://www.fiu.edu/orgs/w4ehw/CWOP-Main.html
APRSWXNET Server - CWOP Data Collection and Routing (Steve Demise)
http://www.findu.com/citizenweather/
FSL Mesonet Page (Patty Miller), select the "ASPRWXNET" data source
http://www-frd.fsl.noaa.gov/mesonet/
CWOP Quality Control Page (Russ Chadwick)

?> http://www.peakpeak.com/~kb0tvj/


Regards,

Dave Helms

-----Original Message-----
From: Jacobs, Clifford A. [mailto:address@hidden]
Sent: Monday, February 18, 2002 12:40 PM
To: 'HELMS '; Jacobs, Clifford A.
Subject: RE: NSF Weather Station Obs

Dave,

Thank you for your interest in the NSF WX station.  This station was set up
with the help of National Center for Atmospheric Research (NCAR) which is
sponsored by the NSF.  It was set up as an informal station that provides a
well thought out interface readily accessable over the Internet.  It is used
mostly internally at NSF and we do not have the staff to do anyting beyound
what you see. Although I know the folks at FSL, I have not made them aware
of the NSF station.  However, NCAR colleagues probably have.

All the data available on the http://weather.geo.nsf.gov/ is distributed
over the Internet via technology developed by the NSF sponsored Unidata
program.  NSF does not maintain stations other than the one on the roof of
NSF headquarters.  However, additional observations are taken at many of the
160 university departments participating in the Unidata program.  These
observations are distributed over the Internet in near real time to other
Unidata sites.  We are present working on making observations from
Antarctica available to the Unidata community.

Regards,

Cliff

-----Original Message-----
From: HELMS
To: address@hidden
Sent: 2/17/02 9:19 PM
Subject: NSF Weather Station Obs

Cliff,

I visited your NSF weather station in Alexanderia, VA
(http://atm.geo.nsf.gov/cgi-bin/nsfweatherE).  Ever try and
disseminate the data to NOAA? Forecast Systems Lab and the
Ham Radio folks help disseminate data through the Citizen
Weather Observer Program (CWOP). Does NSF maintain any
other surface obs stations (other than Boulder)?  Your info
could be useful!

If intersted, let me know and I'll give you info on how you
can connect to CWOP.

Regards,
Dave Helms
CW0351
http://dhelms.mystarband.net/wx.html
NWS Email:  address@hidden

Linda Miller - address@hidden
External Liaison, Unidata
University Corporation for Atmospheric Research
P.O. Box 3000
Boulder, CO 80307-3000
303 497-8646 fax: 303-497-8690
URL:  http://www.unidata.ucar.edu/staff/lmiller/un.act.html



From address@hidden Thu Feb 21 10:23:16 2002
To: Patty Miller <address@hidden>
cc: address@hidden, address@hidden,
  address@hidden, address@hidden,
  address@hidden, address@hidden
Subject: Re: NSF Weather Station Obs


Hi Patty,
I really appreciate the information you provided on the MADIS project. It comes at a perfect time, because there was an action item from our last Unidata Users Committee meeting for me to check with FSL about the MADIS project on behalf of the Unidata community. At the time I received the message from Dave Helms, I didn't make the connection, but finally it dawned on me ;-) We had a Users Committee member make that request. It looks like a great resource!
We have a Unidata Policy Committee coming up next week at the NSF, but I will be contacting you to discuss this indepth, prior to the Users Committee meeting on 14-15 March. 

Thanks so much, Patty!

Linda