My sensor has stopped uploading data to sensor.community, but it is still uploading ok to other services like madavi.de.
This is a sensor I’ve had since 2019.
I recently moved house, so updated the address in the settings page, and also renewed the SDS011 sensor and upgraded the temperature sensor to a BME280.
It was uploading fine yesterday evening when I first set it up at the new address, but today the problem started happening.
The WiFi quality was around 50% so I thought that was the problem, but placing the WiFi access point out near the sensor to increase the quality to 100% still doesn’t solve the problem.
Chip ID: esp8266-14063112
Debug log:
airRohr: NRZ-2020-133/DE
mounting FS...
opened config file...
parsed json...
output debug text to displays...
Connecting to Bonny Doon
....SNTP synced: Wed Mar 13 18:37:32 2024
twoStageOTAUpdate
.
WiFi connected, IP is: 192.168.178.21
Starting Webserver... 192.168.178.21
ChipId: 14063112
MAC Id: 68c63ad69608
Read SDS...: 23-06-26(ca5e)
Stopping SDS011...
Read BMxE280...
Trying BMx280 sensor on 76 ... found
Send to :
sensor.community
Feinstaub-App
Madavi.de
----
Auto-Update active..PM10: 19.35
PM2.5: 10.43
----
## Sending to sensor.community - SDS011
Temperature (°C): 10.75
Pressure (hPa): 101059.34
Humidity (%): 57.90
----
## Sending to sensor.community - BME280
## Sending to madavi.de:
Succeeded - api-rrd.madavi.de
## Sending to opensensemap:
Succeeded - ingress.opensensemap.org
## Sending to Server FS App:
Succeeded - server.chillibits.com
Time for Sending (ms): 751
Device status:
Parameter Wert
Firmware NRZ-2020-133
ST:1
SDK:2.2.2-dev(38a443e)
Core:2.7.3-3-g2843a5ac=20703003
lwIP:STABLE-2_1_2_RELEASE
glue:1.2-30-g92add50
BearSSL:5c771be
Free Memory 30576
Heap Fragmentation 11 %
Last OTA 2 min, 56s
NTP Sync 1
NTP Info 192.168.178.1 (?) reachable: Yes
Zeit (UTC) Wed Mar 13 18:40:31 2024
Uptime 2 min, 56s
Reset Reason External System
SDS011 23-06-26(ca5e)
Fehler
WiFi 0/-46/0
OTA Return connection failed
Sensor.Community 2
Data Send Return connection failed
SDS011 0
Anzahl Messungen 1
Dauer Messübertragung 751 ms
Same here. Three of my sensors stopped uploading data to https://api.sensor.community at 03:21 UTC+1 last night (all are from nettigo with firmware NAMF-2020-46rc1 or NAMF-2020-45).
Disabling HTTPS for api.sensor.community fixes it. But this should be no permanent solution.
There must have been some changes to the SSL config or the certificate.
No problems with Madavi.de, aqi.eco and OpenSenseMap.
Removing the BEARSSL_SSL_BASIC definition makes the firmware aware of all available ciphers and the sensor now uses the EC certificate from Sun, 28 Jan 2024 22:40:10 UTC.
Some more background on this issue (and why it popped up now):
On Thursday, Feb 8th, 2024, we stopped providing the cross-sign by default in requests made to our /acme/certificate API endpoint. For most Subscribers, this means that your ACME client will configure a chain which terminates at ISRG Root X1, and your webserver will begin providing this shorter chain in all TLS handshakes. The longer chain, terminating at the soon-to-expire cross-sign, will still be available as an alternate chain which you can configure your client to request.
The Firmware (airrohr and nettigo) uses this old X3 certificate, so the certificate chain is broken with all certificates from Let’s Encrypt issued after Feb 8th, 2024 resulting in broken HTTPS connects:
This means the API from madavi and other sites using certificates from LE (mostly all of them do that) will be unreachable via HTTPS after their next certificate renewal in the next days or weeks.
At this moment it looks like api.sensor.community, api-rrd.madavi.de and firmware.sensor.community still/again use a cross-signed certificate chain. So until the next renewals all should be fine.
I’ve been running the SC firmware with that patch for almost a year now and HTTPS uploads and downloads to/from the various servers work and should also work when the cross-signed intermediates expire or are removed.
is there any update on this issue? Esp. for airrohr-users? I disabled SSL as a workaround, which sends data, but - obviously - OTA is broken (https), so I think a manual update will be necessary, but the latest airrohr firmware I find is the 2020-133 which I have installed already…
This PR will fix the firmware but for OTA upgrades and uploads to actually work with the installed base the certificates on the various SC-servers most likely will need to be ‘downgraded’ to RSA keys. Currently the server certificates are using “Public key type EC/secp384r1” . The deployed firmware can’t cope with that as you already noticed. Changing (server side) to RSA should make it work again.
I don’t know what mechanism is used on the SC-servers to renew certificates but when using certbot that would be something like:
certbot --force-renewal --preferred-chain 'DST Root CA X3' --key-type rsa
This also will keep the ‘old’ chain active to keep serving the installed base.
As a first step it would be even sufficient to go back to the previous certificate, which should be still valid for more than 20 days.
But only if a project member would merge your PR and build the firmware asap.
At this moment all SC-servers do have a certificate with the old chain which should work for the next few weeks so I can only assume that they are running the previous certificates.
There are however still connection failures for OTA upgrade checks and possibly also uploads which might be related to other recent changes in the server-side SSL-config. key-type is my first guess but without actually knowing what changed that’s just a guess.
Issuer R3 (Let's Encrypt from US)
Certificate Validity (UTC) 89 >= 30 days (2024-03-16 17:21 --> 2024-06-14 17:21)
Most sensors seem to be back at https://maps.sensor.community since the early evening.
About 11.400 right now vs. about 9.200 sensors before the certificate change.
At the end of February I saw about 11.900.
Good news indeed. And with a forced renewal a few days before Thursday, June 6th, 2024 will extend the time left before the definite expiration of the cross-signed root with another 90 days.
OTA checks still give a connection failure. It looks like firmware.sensor.community and the other SC-servers serve certificates based on used protocol/capabilities and firmware.sensor.community uses a non-cross-signed certificate when connecting with an ‘old client’ similar to the sensor firmware:
This is still an issue for any devices sending data to services using newer SSL certs.
I’m using opensensemap, which is hardcoded to HTTPS in the firmware (I’m unsure is opensensemap support unencrypted connections). OpenSensemap curerrently have a message up about the issue on their site.
@Phaze-III ’s PR updates the root cert on the device firmware to allow it to resolve/validate newer certs successfully.
Any chances of getting it merged/built/deployed OTA?
Otherwise I’m going to have to try to build locally (haven’t had an environment set up for a few years) go retrieving devices and re-flashing them manually, which is going to take a while (I was supposed to be using some for a baseline for a project involving closing streets near schools to measure the impact).
We are working on this. As a reminder we are a volunteered project, doing all this in our spare time (while OpenSenseMap got millions of EU fundings …)
The problem is that we need to support 2 root certificates for some time until all devices are updated. And that the OTA update needs some more testing as we will lose even more devices (possibly all) when they can’t start anymore after the update.
