[ldm-users] Processing METAR into a database

• To: "ldm-users@xxxxxxxxxxxxxxxx" <ldm-users@xxxxxxxxxxxxxxxx>
• Subject: [ldm-users] Processing METAR into a database
• From: "Herbster, Christopher G." <herbstec@xxxxxxxx>
• Date: Sat, 14 Dec 2024 17:51:34 +0000
• Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=erau.edu; dmarc=pass action=none header.from=erau.edu; dkim=pass header.d=erau.edu; arc=none
• Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=7655CQfiVvF11W/J4EsLIuxEn5QS5lwgBye+NAs6bgg=; b=aQ6G5oEC0ZilJsKVdU8H+kfdnF4MA9FszhjaNJFv7icJvunijrTgos073eHXCtRM+GSVIGiT8iuphrzkHSBp7tiQd/n90c6c+Y1JKkfrp2gt9OSoeJDiEbZauHPp671TC3R0dR5XsuNGBpRsPgGznbOHjhG1ayMtcq2B1HIyKS0tfTEQ6o8GKsWZu/5t7I3UzfdoLmg82zKnKSHzVTxXM14h0a8ZZgjbiKBTW6KTeKpqdZ1uMjydlyWoBhSVoaHrGTG9tA2GG4TxhNy0LdRAu3OtJof/NbibHs060G/G8A8o0XsMBlp/2Pll9RAEOew7QcZjFeIV+6mzBw/FZziGig==
• Arc-seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=R65Tsl0RDMNhTnRHUyl27UZTxTJG+r6yzgkRb7LxTP7gG4ZrO4OvRffqErZMe4ZNQwIB509JV1MBwUj4ki+T8vhlL8eS5qEYUH4BhwUKvWSpZW+PU0K+4tHAhjTL9FxQ3q9+B5b0NF8IkF7SWf+89jrkZkUUnkHrucWoLnF8vc+Ys/Cc4I4vrm54dQcLwQKd6531ROd+ROdxQ1ADLR30KmMpsJqfToItZiAflWRdzU38SDV2oi6L/Ia69B39pkzbyVg8GraUzO1xjd9KwOcw2o5ODIuT4x+dxK/mtVnqaIP7g1DbfnH/NcKkQVdZKgCMwI787cTgYeg+UAisM0GR7w==
• Authentication-results: dkim=none (message not signed) header.d=none;dmarc=none action=none header.from=erau.edu;
• Msip_labels:

Hi all,

I have a number of desires for having processed METAR data in a database like 
format. We would like to do some forecast verifications for an internal effort, 
as well as possibly create charts of recent/past conditions like: 
https://www.weather.gov/wrh/timeseries?site=KDAB

I suspect that some of you have significant experience doing this, and I would 
really appreciate any sagely advice you can provide.

I was expecting to implement something that would happen as a pqact entry when 
data arrive, though I have not ruled out making hourly files that get processed 
next hour. Data would be either from NOAAPort, IDD or (perhaps?) local 
archives. Does it matter that the Unidata (supported) decoders look like they 
are unchanged for many years? An Internet search showed many GitHub options are 
out there. (Yikes!)

I was also going to create a database entry of the images that flow through our 
system as a way to query/load/delete files that live under our web pages. 
Again, I would certainly benefit from any input folks would like to provide.

I am not yet married to a database on the backend, so input there is welcome as 
well. We do have an Apache Hive system that is a candidate for this, but I have 
tinkered with MariaDB in the past. (I am a rookie at whatever I will use.)

Is this already being done in EDEX? We had, and will have again, and EDEX 
server running internally, so perhaps I can exploit that.

Thanks in advance for any thoughts, suggestions, and/or tools.

Chris


Christopher Herbster, Ph.D.
Associate Professor of Meteorology
Applied Aviation Science
Daytona Beach Campus
College of Aviation Rm. 338

1 Aerospace Boulevard
Daytona Beach, FL 32114
386.226.6444
christopher.herbster@xxxxxxxx<mailto:christopher.herbster@xxxxxxxx>

Embry-Riddle Aeronautical University
Florida | Arizona | Worldwide

[cid:b06a0c23-6872-4749-82a6-87c0509a3c59]

386.226.6446 Weather Center
http://wx.erau.edu/

Schedule at:  http://wx.erau.edu/faculty/herbster/Schedules/

• Follow-Ups:
  • Re: [ldm-users] Processing METAR into a database
    • From: Mike Zuranski