# Codes indicating sensor errors **URL:** \ **Category:** FAQ\ **Created:** [January 22, 2023, 10:19am UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835 "2023-01-22T10:19:02Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![RuudvanMunsterZTM](https://forum.sensor.community/user_avatar/forum.sensor.community/ruudvanmunsterztm/32/1277_2.png) [@RuudvanMunsterZTM](https://forum.sensor.community/u/RuudvanMunsterZTM)\ **Post date:** [January 22, 2023, 10:19am UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/1 "2023-01-22T10:19:02Z")
Maybe I overlooked this in the documentation. Since I read my sensors (SDS011, SPS30) locally in Home Assistant (HA, home automation system that I use), I regularly receive sensor data that falls far outside the usual measuring range. I don’t see those outliers in the data sent to the Sensor Community site. They may be filtered out there. That’s what I’m doing myself in HA now. I wonder if these are special data values that indicate a (error) condition. Is there documentation about that? ---
**Author:** ![air-ais](https://forum.sensor.community/letter_avatar_proxy/v4/letter/a/ad7895/32.png) [@air-ais](https://forum.sensor.community/u/air-ais)\ **Post date:** [January 22, 2023, 11:18pm UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/2 "2023-01-22T23:18:49Z")
I doubt that there is any type of filtering. Are you referring to the sensor community or to madavi? If it is the former, there are regular errors for many users when uploading to the server, so momentary spikes would be less likely to appear in this case. ---
**Author:** ![pjg](https://forum.sensor.community/user_avatar/forum.sensor.community/pjg/32/194_2.png) [@pjg](https://forum.sensor.community/u/pjg)\ **Post date:** [January 23, 2023, 6:08am UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/3 "2023-01-23T06:08:20Z")
There is no filter. Please send me the values from archive, the graphics and the Sensor ID ---
**Author:** ![pjg](https://forum.sensor.community/user_avatar/forum.sensor.community/pjg/32/194_2.png) [@pjg](https://forum.sensor.community/u/pjg)\ **Post date:** [January 23, 2023, 6:11am UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/4 "2023-01-23T06:11:35Z")
HA produces maybe byte shifting in the transmission which produces Bad values. ---
**Author:** ![RuudvanMunsterZTM](https://forum.sensor.community/user_avatar/forum.sensor.community/ruudvanmunsterztm/32/1277_2.png) [@RuudvanMunsterZTM](https://forum.sensor.community/u/RuudvanMunsterZTM)\ **Post date:** [January 23, 2023, 6:57pm UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/5 "2023-01-23T18:57:25Z")
I am sending the requested information: - the sensor data from the past 10 days stored in the Home Assistant database; - the graphs as shown by Home Assistant for the past 10 days; - the sensor ID: 14990217 It took some work (and a steep learning curve) to get the data from the HA database. I did this with TablePlus, with which the HA database can be read properly. Note: The time is in UTC. It seems I was able to locate part of the problem. If the outliers occur, the data has been swapped. For example, it seems that the PM2.5 then gets the value of the air pressure. I have done some more tests, the data as HA shows it, corresponds well with the data that the sensor shows as a web server. For the sake of completeness: I use the following template to read in the sensor values: ```auto - platform: command_line name: "Air Quality opzij PM10 (SDS)" command: 'curl http://192.168.178.20/data.json' value_template: "{{ value_json.sensordatavalues[0].value | round(1) }}" unit_of_measurement: "µg/m³" - platform: command_line name: "Air Quality Aside PM2.5 (SDS)" command: 'curl http://192.168.178.20/data.json' value_template: "{{ value_json.sensordatavalues[1].value | round(1) }}" unit_of_measurement: "µg/m³" ``` Outliers are filtered out by: ```auto - platform: template sensors: air_quality_opzij_pm25: friendly_name: Air quality opzij PM 2.5 correctie unit_of_measurement: "µg/m³" value_template: >- {% if states('sensor.luchtkwaliteit_opzij_pm2_5_sds') | float < 10000 %} {{ states('sensor.luchtkwaliteit_opzij_pm2_5_sds') }} {% else %} 0 {% endif %} ``` Data can be found here: > – Link to data removed by admin after issue was solved – ![Opzij_PM2_5](https://forum.sensor.community/uploads/default/original/2X/a/a4d59cfc39208d689a198a8d2ad55c16313b3123.jpeg) ![Opzij_PM2_5_Corrected](https://forum.sensor.community/uploads/default/original/2X/5/54f83972f50f67cd4fb3ec9e9764e067ce94973e.jpeg) ![Opzij_Airpressure](https://forum.sensor.community/uploads/default/original/2X/6/602915d727a26a72348a00fc0573e9b1745a780e.jpeg) ![Opzij_PM10](https://forum.sensor.community/uploads/default/original/2X/2/2f57dced8cc8a9da98063510f5945abec281de91.jpeg) ![Opzij_Temperature](https://forum.sensor.community/uploads/default/original/2X/9/9efed152fc8b498925507b42c46046c0428deb09.jpeg) ---
**Author:** ![RuudvanMunsterZTM](https://forum.sensor.community/user_avatar/forum.sensor.community/ruudvanmunsterztm/32/1277_2.png) [@RuudvanMunsterZTM](https://forum.sensor.community/u/RuudvanMunsterZTM)\ **Post date:** [January 23, 2023, 7:55pm UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/6 "2023-01-23T19:55:23Z")
By the way, import by Home Assistant is based on the approach described in ["Alarm" for values - #4 by RuudvanMunsterZTM](https://forum.sensor.community/t/alarm-for-values/1609/4) ---
**Author:** ![RuudvanMunsterZTM](https://forum.sensor.community/user_avatar/forum.sensor.community/ruudvanmunsterztm/32/1277_2.png) [@RuudvanMunsterZTM](https://forum.sensor.community/u/RuudvanMunsterZTM)\ **Post date:** [February 25, 2023, 6:23pm UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/7 "2023-02-25T18:23:59Z")
Do you mean by “_shifting in the transmission_” that some values are missing in the JSON file? After some investigations I found that it happens regularly that the lines with the SDS sensor data in the file below are missing. ```auto > {"software_version": "NRZ-2020-133", "age":"118", > "sensordatavalues":[ > {"value_type":"SDS_P1","value":"17.35"}, > {"value_type":"SDS_P2","value":"9.60"}, > {"value_type":"BME280_temperature","value":"7.12"}, > {"value_type":"BME280_pressure","value":"103936.44"}, > {"value_type":"BME280_humidity","value":"62.25"}, > {"value_type":"samples","value":"5060693"}, > {"value_type":"min_micro","value":"28"}, > {"value_type":"max_micro","value":"20128"}, > {"value_type":"interval","value":"145000"}, > {"value_type":"signal","value":"-40"} > ]} ``` Because the code I have been using to import the local sensor data into Home Assistant sensor assigns the data to the sensor based upon the position in the JSON data area, the BME280\_temperature is assigned to sensor SDS\_P1 and the BME280\_pressure value is assigned to SDS\_P2. Especially the BME280\_pressure values are seen in the graphs, becaus of their higher values. I solved the problem by changing the following code to obtain the values locally: ```auto - platform: command_line name: "Luchtkwaliteit Opzij PM10 (SDS)" command: 'curl http://192.168.178.20/data.json' value_template: "{{ value_json.sensordatavalues[0].value | round(1) }}" unit_of_measurement: "µg/m³" - platform: command_line name: "Luchtkwaliteit Opzij PM2.5 (SDS)" command: 'curl http://192.168.178.20/data.json' value_template: "{{ value_json.sensordatavalues[1].value | round(1) }}" unit_of_measurement: "µg/m³" ``` by: ```auto - platform: rest name: JSON airquality unique_id: airquality_opzij json_attributes: - software_version - age - sensordatavalues resource: http://192.168.178.20/data.json value_template: "{{ value_json.software_version }}" - platform: template sensors: airquality_opzij_software_version: unique_id: airquality_opzij_software_version friendly_name: "Airquality Opzij Software version" value_template: "{{ state_attr('sensor.airquality', 'software_version') }}" airquality_opzij_age: unique_id: airquality_opzij_data_age friendly_name: "Airquality Opzij Data Age" value_template: "{{ state_attr('sensor.airquality', 'age') }}" - platform: template sensors: air_quality_opzij_pm10_new: unique_id: airquality_opzij_PM10_REST friendly_name: "Air quality opzij PM10 (REST)" unit_of_measurement: "µg/m³" value_template: >- {% if state_attr('sensor.airquality', 'sensordatavalues')[0].value_type == 'SDS_P1' %} {{ state_attr('sensor.airquality', 'sensordatavalues')[0].value | round(1) }} {% else %} 0 {% endif %} - platform: template sensors: air_quality_opzij_pm25_new: unique_id: airquality_opzij_PM25_REST friendly_name: "Air quality opzij PM2.5 (REST)" unit_of_measurement: "µg/m³" value_template: >- {% if state_attr('sensor.airquality', 'sensordatavalues')[1].value_type == 'SDS_P2' %} {{ state_attr('sensor.airquality', 'sensordatavalues')[1].value | round(1) }} {% endif %} ``` This solution checks whether the sensors are at the right position in the JSON file. It compares the value\_type of the sensor data against the expected value\_type. The problem I described above was because some of the data are simply missing in the datafile if there is a sensor error (e.g. if the measurment failed) (I am still searching for the cause of that) The new template ignores the data if it is at the wrong position in the JSON file. You see two variations on this theme: - The first sensor template adds zero’s for the missing data. Which is convenient for testing purposes. - The second sensor simply ignores the wrong data. Which is beter for operational use, I think. There must be a better way to code this, using a list construction which links the data\_value to the right sensor based upon the value\_type. I am still a novice in this kind of programming, in a steep learning curve. So suggestions here are very welcome. The new solution is working now for about a week, together with the former solution, which enables me to check how it works. It looks very stable now. ---
**Author:** ![RuudvanMunsterZTM](https://forum.sensor.community/user_avatar/forum.sensor.community/ruudvanmunsterztm/32/1277_2.png) [@RuudvanMunsterZTM](https://forum.sensor.community/u/RuudvanMunsterZTM)\ **Post date:** [February 25, 2023, 6:34pm UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/8 "2023-02-25T18:34:16Z")
Based upon my experience of the last months, I think we should prefer the following approach: - Have your sensor send the data directly to the Sensor Community site, for sharing data and research; - Use a local data-exchange for your own operational purposes. As far as I see now, that is far more reliable. And it fits better in the (Home Assistant) philosophy to avoid the cloud for real-time operational purposes. As a side-effect there is less load on the Community servers. Their intended use is probably just for sharing data and research not for being “polled” every few minutes by a lot of users. ---
**Author:** ![RuudvanMunsterZTM](https://forum.sensor.community/user_avatar/forum.sensor.community/ruudvanmunsterztm/32/1277_2.png) [@RuudvanMunsterZTM](https://forum.sensor.community/u/RuudvanMunsterZTM)\ **Post date:** [February 25, 2023, 6:44pm UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/9 "2023-02-25T18:44:06Z")
And of course, I am still wondewring why these data are missing. Are there transmission problems? Is the sensor defective or has it to be cleaned up? Now having more control about the incoming data, I can do some investigation on that. ---
**Author:** ![pjg](https://forum.sensor.community/user_avatar/forum.sensor.community/pjg/32/194_2.png) [@pjg](https://forum.sensor.community/u/pjg)\ **Post date:** [February 25, 2023, 7:03pm UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/10 "2023-02-25T19:03:37Z")
I mean byte shifting in the Serial transmission from the sensor to the board. It can produce huge values. ---
**Author:** ![pjg](https://forum.sensor.community/user_avatar/forum.sensor.community/pjg/32/194_2.png) [@pjg](https://forum.sensor.community/u/pjg)\ **Post date:** [February 25, 2023, 7:07pm UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/11 "2023-02-25T19:07:36Z")
Your cables are defective 🙂 Check the connections and put some Kontakt60 everywhere (not a lot, just the right amount…). ---
**Author:** ![RuudvanMunsterZTM](https://forum.sensor.community/user_avatar/forum.sensor.community/ruudvanmunsterztm/32/1277_2.png) [@RuudvanMunsterZTM](https://forum.sensor.community/u/RuudvanMunsterZTM)\ **Post date:** [February 25, 2023, 10:22pm UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/12 "2023-02-25T22:22:07Z")
Thanks! I’ll check that. ---
**Author:** ![Andrix](https://forum.sensor.community/user_avatar/forum.sensor.community/andrix/32/1766_2.png) [@Andrix](https://forum.sensor.community/u/Andrix)\ **Post date:** [March 23, 2024, 1:59pm UTC](https://forum.sensor.community/t/codes-indicating-sensor-errors/1835/13 "2024-03-23T13:59:37Z")
Did you solve it? I’ve just learn by reading this thread the possibility to get data from “http://…local IP…/data.json”… I did similar thing by taking in another manner (someone could consider this “not efficient”, but it was the only one that I was able to do… I’m not so skilled): - Install in Home Assistant the MULTISCRAPE integration from HACS ([GitHub - danieldotnl/ha-multiscrape: Home Assistant custom component for scraping (html, xml or json) multiple values (from a single HTTP request) with a separate sensor/attribute for each value. Support for (login) form-submit functionality.](https://github.com/danieldotnl/ha-multiscrape)), I already was using it for other purpose (get hourly price of energy from an institutional website). - Add to the configuration.yaml the following code (I’m reporting just for the first sensor) ```auto multiscrape: - name: AQ Local resource: http://...local IP.../values scan_interval: 60 sensor: - unique_id: AQ Local PM2.5 name: AQ Local PM2.5 select: 'tr:nth-child(2) > td.r' value_template: '{{ value[:-6] | float }}' unit_of_measurement: 'µg/m³' icon: 'mdi:molecule' ``` I’ll test my way, and in case I’ll try with the “http://…local IP…/data.json”. Thank you