____________________________________________________________________________ | | | ____ _____ | | | _ \ ___ ___ _ _| __/___ ___ _____ ___ ___ | | | |_| / _ \/ __| |/ | |_ / _ \| _ |_ _| |_ _/ _ \ | | | _ | |_|| |__| ' <| _| |_| | / | | _ | | |_| | | | |_| \_\___/\___|_|\_|_| \___/|_|_\ |_| |_| |___\___/ | | | |__________________________________________________________________________|
____ _____ | _ \ ___ ___ _ _| __/___ ___ _____ ___ ___ | |_| / _ \/ __| |/ | |_ / _ \| _ |_ _| |_ _/ _ \ | _ | |_|| |__| ' <| _| |_| | / | | _ | | |_| | |_| \_\___/\___|_|\_|_| \___/|_|_\ |_| |_| |___\___/

HungaroMet ODP project intro ============================ 

First draft: 2024-12-10
Published:   2025-01-06
Last update: 2025-01-08

Table of contents:

Introduction to a Rust development series on making custom visualizations for official Hungarian weather data.

Motivation

Website of the official Hungarian Meteorological Service, or HungaroMet in short, www.met.hu has many great options for visualizing all the data the organization is gathering at their system of weather observation points throughout the country. Many are showing a map and values distributed on it graphically, with an interpolated overlay displayed in nice colors.

What I am missing is a table display where I can look at all the variables from one station, for example the nearest to me in Sopron. Also it would be interesting to be able to produce plots on certain variables of the same station from the previous day or week, to get the general feeling of the current situation or short-term trends. Sincere apologies in case these visualization options are available, and it was just me who could not find them. In any case as with any national organization operating on taxpayer funds, all this data should be public and available to access, and it certainly is since 2021, to my relief.

One other useful purpose of having this data locally is that I would be able to do things such as correlating it with other series. For example, it is obvious that having weather station data is beneficial when analyzing indoor environmental data, and it is not possible to install one's own in certain situations. I will launch another series of articles soon (link to follow) about indoor measurements in my office, which is in the historical city center so it would not be easy to get permissions to install anything on the roof. Also even if I had my own station, it is always nice to have something to compare to as a baseline, it can even improve data quality by outlier detection or using as a substitute for missing records.

About HungaroMet ODP

The service is available at the odp.met.hu address. The abbreviation stands for "Open Data Policy", indicative of actions implemented within the organization to comply with European Union directives. Please excuse if I were to refer to the service just as "platform" casually.

I have an obligation to note the terms of use. There is a "General Terms and Conditions of Use" document available (and also an official English translation), so please read it if you plan to use the data yourself. In summary, HungaroMet owns and have copyright on everything published on their servers, but all measurement data accessible through ODP can be used freely while retaining indication of the source, such as "HungaroMet Nonprofit Zrt." or "Hungarian Meteorological Service". Most of the data can be used without restrictions, but certain sensitive reports must not be changed in any way (e.g. forecasts that contain alerts, flight-related weather data). These permissions only apply to the ODP data and it is not allowed to use anything else from the HungaroMet websites.

The service works basically as a file share, its tree containing various climate and weather data, most being in CSV format archived in ZIP. Actual station data sets can be found in the `weather_reports` directory, where several temporal resolutions are available, like hourly, daily, monthly. Naturally, if I would like to display values as recent as possible, I shall choose the smallest aggregation period available, which is 10 minutes. According to the documentation, raw measurements are gathered in 1 minute interval, but apparently these are not shared through the ODP directly.

Plans

I intend to create at least three applications in this project:

  1. ODP client: As I would like to use a relational database to store the data, I have to create a small command-line tool to download the files periodically, parse the CSV and insert the records. This would then be scheduled by `cron` or something similar to always have up-to-date data to show in front end.
  2. FastCGI demo: I was always fond of CGI scripts for displaying quick one-page info when nothing more sophisticated was necessary. I understand the drawbacks of that particular implementation, so it would be nice this time to try something more robust but still simple, like FastCGI.
  3. Web application with Rocket: Displaying the plots, and the possibility to select different weather stations would require much more effort than a one-pager, so this could be an ideal environment for getting to know one of the Rust backend frameworks, Rocket.

I also have one more idea but nothing solid: It would be interesting to try and replicate the map-type display every weather report has. I would like to make it as procedurally generated SVG, based on OpenStreetMap paths for the country borders and rivers. It sounds simple enough to include it in the Rocket application, although I can only hope that I will still have the dedication in the end to implement this as well.

Current status

Project is right now in preparation for stage 1, development of the client application.

Other blog posts published in this series: