Support for IPv6

Hello

I have a sensor up and running since May 1st and I am impressed. But there is one thing that disappoints me: it does not support IPv6!

IPv6 has been around everywhere for well over a decade now and IMNSHO it can no longer be ignored. Is there any chance it will be implemented in the foreseeable future? By a firmware upgrade?

Hello,

every new feature needs more memory. But the ESP8266 has very limited resources. You could check the amount of additional memory by modifying our firmware, it’s open source. Enabling IPv6 should be only changing a compile switch in the file platform.io. Search for IPv6, that should find a comment where you can also read why we haven’t implemented this until now even as we had the needed memory.

As for the additional memory usage when enabling IPv6:

With IPv6:

RAM:   [====      ]  43.9% (used 35944 bytes from 81920 bytes)
Flash: [=======   ]  69.6% (used 726907 bytes from 1044464 bytes)
Reported free memory: ~ 26K

Without IPv6:

RAM:   [====      ]  41.9% (used 34300 bytes from 81920 bytes)
Flash: [=======   ]  67.2% (used 702159 bytes from 1044464 bytes)
Reported free memory: ~ 28K

There is also a github issue IPv6 Support · Issue #605 · opendata-stuttgart/sensors-software · GitHub with some background on the main reason it is currently not enabled.

IPv6-enabled IoT-devices with limited resources heavily depend on security policies on the perimeter (the router). Most consumer-grade IPv6-enabled routers nowadays should however by default block incoming IPv6 connections but the risk of exposing the WebUI to the public Internet is still a bit higher with IPv6.

Besides that there are also other issues. Just enabling IPv6 in platformio.ini by default won’t do much. You will be able to connect to the WebUI over IPv6 and you can configure IPv6-only API-servers. But most traffic will just be over IPv4 unless you use a compile time switch to prefer IPv6 DNS-resolving (AAAA records) over IPv4 (A records). That however will break the firmware on IPv4 only networks.

Also the IPv6-enabled firmware won’t work in an IPv6-only network. The code expects an IPv4 address and will start AP-mode without that (amongst other issues).

So without additional code modifications the benefits of just enabling IPv6 are minor.

If you have a working dual-stack network you can try the code at GitHub - Phaze-III/sensors-software at feature/enable-ipv6 . Firmware binaries can be downloaded from the actions page at Workflow runs · Phaze-III/sensors-software · GitHub Select the most recent run and download the ‘artifacts’.

That one prefers IPv6 connections (but as said won’t work on IPv4-only nor IPv6-only networks) and will upload data over IPv6 to Madavi and Sensor.Community. It even does OTA-upgrades over IPv6 :slight_smile:

@ricki-z
I found the compile switch in platformio.ini. Although I have done my fair share of programming for stand alone devices, that was over 30 years ago when such things were written in assembler and burned into EPROM. I have been out of the loop for too long and I no longer have the resources and the knowledge to compile and build a new firmware myself.

I didn’t realise that the ESP8266 was so spartanic on memory that adding IPv6 might be a problem. It reminds me of my advertures with the Science of Cambridge MK14 when an extra 256 bytes of memory was a gift from heaven. :wink:

@Phaze-III
Tnx for your response. It took me a couple of days to let it sink in…

  1. I am pleased to see that IPv6 has not been ignored and that a lot of it is already potentially there.

  2. Regarding memory: Looking at the numbers you gave, I do not think it is serious problem. With IPv6 enabled there is still room for other new features I’d say…

  3. Regarding te UI being accesable from outside with IPv6 anabled: I do not think that is a problem either. It may have been an issue in the early days of IPv6, but not any more. I have been playing around with IPv6 for more than 15 years and I have never had a router in my hand that does not have a firewall that does not block all unsollicited incoming IPv6 packets by default.

  4. Of course IPv6 should be prefered when making connections, with fall back to IPv4 when an IPv6 connection fails. You mention that setting that compile switch will break the firmware in IPv4 only networks. What exactly will it break and does the break only occur in client mode or also in AP mode?

  5. A pity that with full IPv6 enabled the device can not function in Single Stack networks. I have had Full Dual Stack for some 15 years so for me it is not a problem. But I am already experimenting with IPv6 only in my lab. The device - like any other network - device should eventually be able to function in an IPv6 only environment but I accept that for the moment this is a bridge too far. And of course - while IPv4 is still around - it should also work in an IPv4 only environmemt.
    My understanding is that it goes into AP mode when it has not been served an IPv4 addrese within two mintutes after boot up. Is that the only problem with IPv6 only?

  6. I’d love to test the experimental IPv6 version for which you provided a link. As soon as I have figured out how to load that into my ESP8266. And how to recover when something goes wrong. As I mantioned I have not done such things in a long time. Can you give me some pointers on how to flash alternate firmware into the device?

  7. Just something that popped up in my mind: For the future: How about instead of one or more compile switches, making it a user switch in the configuration. In the “advanced, only know what you are doing” section of course. A switch that is off by default. And to save the user from painting him/herself into a corner have it only apply to client mode and leave the AP mode IPv4 only for the time being?

Re 3.: The switch used to prefer IPv6 DNS lookups will just do that, when connecting to a server the firmware will lookup the IPv6 address (DNS AAAA record) of the hostname. If there is no AAAA record it will look for an A record (IPv4 address). It will then connect to the first found address. When, for whatever reason, that connection fails there is no fall back or retry.

Without additional code modifications it will also do that on an IPv4 only network. Which means that connections to hosts with an AAAA record will always fail.

This is in client mode. For AP mode it doesn’t really matter since in AP-mode there is no connected network and the node runs with a predefined IPv4 address. There won’t be outgoing connections in AP-mode.

Note that this option might not be the best way to do things, I didn’t do a deep dive in the LWIP code to look for other/better methods. The main goal was to get the firmware to use IPv6 without major modifications.

Re 4.: Going to AP-mode is the main problem. Other potential issues are NTP and mDNS and I’m not sure how the used framework/library handles the various IPv6 autoconfiguration setups.

Re 5.: The provided link only shows a downloadable file with firmware builds for registered github users. I’ve also created a ‘release’ where you can get the firmware builds without github-account: Release NRZ-2024-136-B1 with IPv6 enabled (preliminary test builds) · Phaze-III/sensors-software · GitHub

For flashing the firmware I use esptool (esptool.py or esptool.exe) from the command line. esptool can be found at Releases · espressif/esptool · GitHub . On Linux/Mac esptool(.py) is also available in most package repositories.

From the command line you can use something like the following to flash the firmware (Windows):

esptool --port COM3 --baud 115200 write_flash 0x0 latest_nl.bin

or (on Linux/Mac)

esptool.py --port /dev/<yourdevice> --baud 115200 write_flash 0x0 latest_en.bin

Existing settings on the sensor are preserved and if it breaks you can always go back to the official firmware with the airrohr-flasher (which I assume you used already).

If you’re not comfortable with the command line you can (Windows only) use a small graphical utility from Releases · BattloXX/ESPEasyFlasher · GitHub which basically provides a simple GUI to call esptool with the above mentioned parameters.

Re 6.: Making IPv6 configurable certainly would be an option although ideally it should not be needed. However it needs to be coded, just as the modifications needed to fix the above mentioned issues. For me that might be a bridge too far at the moment.

@Phaze-III
Thank you for your elaborate reply. Once again it took me a while to let it sinki in.

Re 3: I agree the the present way of choosing between IPv4 and IPv6 is not optimal. OTOH, Happey Eyeballs" may be a bit of an overkill for this application. I think it will do for now, even for a public release.

Re 4: Maybe we should just forget about IPv6 only networks for now and leave that for the next version. If it works on dual stack networks that is better than no iPv6 at all.

Re 5: I have not used the airrohr flasher yet. This is a project from the municipality (Utrechtse Heuvelrug). With some 20 participants we assembled it from a kit. The firmware was preloaded, we only needed to enter the SSID and password of the WiFi to make it go on-line.

But I found a link to a desription (in Dutch) on how to use the flasher, so I will probably all right when it comes ti that, Or I could try to find the guy who flashed the firmware into the units in the kit.

I am familiar with the command line. The concept of “USB to serial” is new for me however. :smile: I think I can make it work when I get a round toit.

Re 6: I agree that in the end IPv6 should work without having to activate it as an option. It may however be a work around for the moment. I had hoped that we could have a public release with IPv6 working. So without the need to install special beta versions. If making it an option would make that possible without extensive modifications to the code, it would be nice. Especially it it would arrive as a reular OTA update.

BTW, I have firmware version NRZ-2024-135/EN (Apr 9 2024) So pretty recent.

@Phaze-III

I had to go through a steep learning fase, but I finally managed to flash a NodeMCU with your beta IPv6 version. I did not connect any sensors yet, just testing the IPv6 for now.

First impressions:

  1. In AP mode it identifies itself as NRZ-2024-136-B1+IPv6/EN (May 6 2024). In client mode as NRZ-2024-136-B1/EN (Apr 9 2024)

  2. It is assigned a global IPv6 address by the router in my home network. That address does not ping (host unreachable) and the web interface does not respond to that address. (time out) Assuming, it is a SLAAC address, I tried the corresponding link local address. No response either.

  3. Presently I have no way to check for any other IPv6 activity.

  4. I have enabled “Load beta firmware” in the configuration. Will that automatically include OTA beta firmware updates of the IPv6 version?

That’s it for now. I will continue testing…

@PA0MMV Good to hear that you can test the firmware. However since these binaries are not published on the regular firmware-servers you should disable Auto-Update before flashing.

Without that the node will automatically ‘update’ to one of the released firmware versions. That is what causes the different client mode version and the fact that you can’t connect over IPv6. Basically the node went back to the IPv4-only firmware. It did however do that over IPv6 :wink:

So if you disable auto-update and flash the beta IPv6 firmware again you should be able to connect to the IPv6 address. The assigned address is printed in the regular debug output and can be found on the ‘Debug level’ page just after reboot.

Aha. Another snake comes out of the grass. :smile: That probably explains why it took a while before it went from AP mode to client mode after I entered the WiFi parameters. It was maybe busy processing the update. For this experiment I started out with a virgin NodeMCU so there was no “before flashing” and after the first flash it had the default setting of “auto update on”.

Anyway, I disabled “auto update” and reflashed. This time without problems.

It now pings on the IPv6 address and I can access the web interface via IPv6. Great!

Testing continues. Stay tuned…

@Phaze-III

Update:

The IPv6 beta firmware has now been moved from the lab to the field. It is humming away and sending data to sensor.community, madavi.de and opensensemap.org. Presumably over IPv6 but I can not verify that.

Sensor #87057 on sensor.community.

If there are updates for this version will you let me know so I can test those?

Hi fellow measurers :slight_smile:
After some dusty years sensor 1214 is online again. One of the oldest in the NL.
Happy sniffing!

@PA0MMV Sounds good. One way to verify that IPv6 is used would be to block outgoing IPv4 HTTP and HTTPS (only those, DNS and NTP should be allowed) connections from the IPv4 src address of the sensor in your router. The sensor should still be able to send data to sensor.community and madavi.de. For opensensemap.org (no IPv6 AFAIK) you should see upload errors.

I’ll post here if there are updates.

@Phaze-III

  1. I use the modem/router supplied by my ISP. It serves my purpose but its firewall can not be tuned to that level of detail. So nocando.

  2. In the log I see that the sensor connects to api.sensor.community. While Sensor.Community has an AAAA record, api.sensor.community has not. So the sensor does not send data via IPv6. How about adding an AAAA record to api.sensor.community?

  3. Regarding NTP. x.pool.ntp.org do not have AAAA records. Many other NTP servers do, so perhaps this can be solved by making the NTP servers configurable, These days I use ntp.time.nl, a relatively new service from SIDN. Time services come and go so maybe making it configurable may be a good idea anyway.

@PA0MMV
May I ask which ISP you have? The reason I ask is that for example the standard router/modem of my ISP allows IP/Port filtering (Ziggo Connect Box). I’m a bit surprised there are ISP’s not offering such basic functionality.

As for NTP: the IPv6 build has 2.pool.ntp.org as first set of hard-coded NTP-servers. That specific pool explicitly has AAAA-records :wink: I’m not convinced though that the NTP-code actually uses them.

And indeed api.sensor.community has no AAAA-records, I somehow assumed that was the case :frowning: @ricki-z : would it be possible to make api.sensor.community IPv6-connected?

@Phaze-III
It so happens that I am in a transition fase, at the moment I am multi-homed. Ziggos (via coax) and Caiway (via Delta Fiber). So I could test with both. The modemrouter from Caiway (Nokia XS-2426G-B) sucks. Presently the sensor is connected via Ziggo and the modem/router is the Black Sagemcom. With beta firmware, I am a member of the test groep. The Sagemcom has some options to block outgoing traffic but it has limitations. I confess that I may have overlooked the possibility to specifically block some protocols for IPv4 only. But if the older Connectbox (which I had before) can do it then probably the Sagemcom can do it too. I will have another look at it.

I overlooked that 2.pool.ntp.org is IPv6 capable. (Thanks @fredvduin for pointing that out) But it does not look like that is actually used. From “device status”

NTP Sync 7
NTP Info 5.255.99.180 (2.pool.ntp.org) reachable: Yes
Time (UTC) Sat Jun 15 15:34:16 2024

Yes, adding an AAAA record to api.sensor.community would be nice.

@ricki-z
What would also be nice is if it could be visible on the map if a sensor connects via IPv6. One way or another. Maybe by using a different shape for the “tack”. A square instead of a hexagon? Although a square for IPv4 and a hexagon for IPv6 would be more logical. But that may be too much to ask. Then again… maybe the mayority of the sensors already is IPv6 capable I really have no idea. :slight_smile:

Or maybe just add “IPv6 only” on the same place where “indoor sensors” can be selected.

@PA0MMV I’ve added the AAAA record for api.sensor.community.

(I don’t think that “the majority of the sensors already is IPv6 capable”. My provider, the second largest in Germany is not able to acitvate IPv6 for most of its users … :roll_eyes: )

@ricki-z
Thanks for adding the AAAA record. My sensor still connects. So far so good: :smile: )


Sending to sensor.community - SDS011

Succeeded - api.sensor.community
Temperature (°C): 15.22
Pressure (hPa): 100513.13
Humidity (%): 77.81

Sending to sensor.community - BME280

Succeeded - api.sensor.community

Sending to madavi.de:

Succeeded - api-rrd.madavi.de

Sending to opensensemap:

Succeeded - ingress.opensensemap.org
Time for Sending (ms): 2741
PM10: 2.33
PM2.5: 1.35

Shame on your ISP for not yet providing IPv6 to all their customers. But I do not think that is the bottleneck. Germany scores very good on the world map of IPv6. 65%. Better than most. Better than The Netherland (46%) and almost as good as Belgium (68%).

https://stats.labs.apnic.net/ipv6

My guess is that it is the sensors that are the bottleneck. If the majority of the sensors are from the Airrohr project, then the majority of the sensors will be IPv4 only. For now anyway. But we won’t be sure until there is a way to see it on the map.

@Phaze-III
I took another good look at the firewall setting of my modem/router and I found the way to block outgoing IPv4 from the sensor. This is the result

Edit:
Hmm… Forget it.It does not work as I expected. Sorry…

@PA0MMV This is what I have in my white connectbox, with a little luck the UI on the Sagemcom is similar.:

With those two rules I get the results as you mentioned them in the first draft of your post (SC and madavi succeeded, failure -1 for opensensemap)

Another way to deduce how connections to the API are made is by looking at the DNS-requests made by the sensor (I use my own internal resolver and can look at the requests made). This is what happens on a (different) sensor after reboot:

2024-06-16 14:20:58.197762 IP 192.168.178.90.64009 > 192.168.178.2.53: 1+ A? 2.pool.ntp.org. (32)
2024-06-16 14:20:58.596463 IP 192.168.178.2.53 > 192.168.178.90.64009: 1 4/0/0 A 193.123.37.231, A 45.138.55.62, A 185.51.192.62, A 87.238.174.133 (96)
2024-06-16 14:21:13.619219 IP 192.168.178.90.64009 > 192.168.178.2.53: 2+ A? 1.pool.ntp.org. (32)
2024-06-16 14:21:13.655793 IP 192.168.178.2.53 > 192.168.178.90.64009: 2 4/0/0 A 213.109.127.82, A 149.210.142.45, A 95.211.120.4, A 45.83.205.227 (96)

2024-06-16 14:23:22.974743 IP 192.168.178.90.64009 > 192.168.178.2.53: 4+ AAAA? api.sensor.community. (38)
2024-06-16 14:23:23.183742 IP 192.168.178.90.64009 > 192.168.178.2.53: 4+ AAAA? api.sensor.community. (38)
2024-06-16 14:23:23.273392 IP 192.168.178.2.53 > 192.168.178.90.64009: 4 1/0/0 AAAA 2a01:238:4259:b400:1e5d:9714:fa8b:6155 (66)
2024-06-16 14:23:23.273502 IP 192.168.178.2.53 > 192.168.178.90.64009: 4 1/0/0 AAAA 2a01:238:4259:b400:1e5d:9714:fa8b:6155 (66)

2024-06-16 14:23:26.456855 IP 192.168.178.90.64009 > 192.168.178.2.53: 5+ AAAA? api-rrd.madavi.de. (35)
2024-06-16 14:23:26.522158 IP 192.168.178.2.53 > 192.168.178.90.64009: 5 1/0/0 AAAA 2a01:238:434c:fd00:a587:fc06:8d77:20f9 (63)

2024-06-16 14:23:29.348860 IP 192.168.178.90.64009 > 192.168.178.2.53: 6+ AAAA? ingress.opensensemap.org. (42)
2024-06-16 14:23:29.470676 IP 192.168.178.2.53 > 192.168.178.90.64009: 6 0/1/0 (126)
2024-06-16 14:23:29.473601 IP 192.168.178.90.64009 > 192.168.178.2.53: 7+ A? ingress.opensensemap.org. (42)
2024-06-16 14:23:29.511065 IP 192.168.178.2.53 > 192.168.178.90.64009: 7 1/0/0 A 128.176.196.25 (58)

This shows:

  1. Only A records of the configured NTP-servers/pool are requested. So no IPv6 for NTP
  2. For the three configured API-servers first AAAA records are requested. If found no additional requests are made. If not found (as is the case with ingress.opensensemap.org) a request for A records follows.

From this you can conclude that connections to SC and Madavi are done over IPv6 and Opnsensemap over IPv4.

@Phaze-III
Thanks for your comment.

I was too impatient, it apparently takes a couple of minutes for the changes to take effect. Plus that one has to specify the origin port nummer as “all” and just specify the destination port. I should have known because that is how it works for opening a port in IPv6 too. Very unlogical but that is how it is.

For me just blocking HTTPS works to get the error. This is the result now:

Sending to sensor.community - BME280

Succeeded - api.sensor.community

Sending to madavi.de:

Succeeded - api-rrd.madavi.de

Sending to opensensemap:

Request failed with error: -1
Time for Sending (ms): 18682

So, indeed, it works as expected. I can not test whay happens if I block port 53, my router does not allow setting that, But if I block all ports then the sensor.community and madavi.de won’t connect either.

Can I conclude that DNS does not use IPv6 either?

I posted a message (in German) in opensensemap’s forum with a request to add an AAAA record to their api. No reply yet.