Scraping PM data - help needed

I’ve been collecting the PM data from my sensors in a local influxdb and -with great effort- managed to hack a python script to scrape PM data of the official PM stations from a website. Now they have changed the website and I don’t have the energy to start all over again.

Is somebody with more experience & knowledge willing to help me?

Cheers,
Torsten

Why don’t you scrap from the SC archive ?
Tell me exactly what you need, I know a few method.

Apologies, I was not very precise - I mean the official stations operated by the administration.

In my case the website is https://luftdaten.brandenburg.de/ and I managed to read data from the table. However, this table is not updated during nighttime between 21:00 and 4:00. I would need a script which simulates a few clicks to download a csv-file which has the full data.

UPDATE: I’ve just discovered that you already seem to collect the official data :slight_smile: (https://sensor.community/en/campaign/sensorAtRefs/)

Sensor@Refs is not exactly a scrapping of the public data.
Look here:

You can find the scrapping code in the src but it is not very straight formward. It is the reason why I stopped.

EEA
Luchtmeetnet
UmweltBundesamt

should still be scrappable.

You can also ask the administrations to publish free and normal API as requested by the EU.

The map (don’t republish it is a test server)
http://pjgueno.epizy.com/SCPublicData/

EEA looks easy enough for me to digest - many thanks for the pointer!

Scraping the EEA website works and yields hourly updates (w/o the gap during nighttime on the owner’s website). BTW, my NPM has risen from the dead. There was quite a lot of pollen in the inlet.

2 Likes