P1 smartmeter (API)

I’ll have a look to make those changes. It sound very logic indeed. Think i’ll split the P1 in two different devices: Electricity and Gas that way there is also room for the low and high channels.

Can you add this to a git repository? Then I’ll link to this in the readme of the app.

I’ve added an issue on GH: https://github.com/koktaildotcom/com.p1.smartmeter/issues/3

Seems a bit odd to start a git repository for just this Q&D script. Feel free to put the script in the example directory as an unsupported way to reproduce the steps some other guy (me :slight_smile: ) took to make use of your beautiful com.p1 app on the homey. You might want to add two lines of comment to make the steps even more clear:

# 1-0:2.8.2 delivered_tariff2, hack string to proper numeric value
# From "1-0:1.8.2(01259.637*kWh)" To: "01259.637"
tdelivered_tariff2=`/bin/grep "1-0:2.8.2" /tmp/smarttelegram.txt | /bin/sed "s/.*(//;s/\*.*//"`
# From: "01259.637" To: "1259.637"
delivered_tariff2=$(/usr/bin/awk "BEGIN {printf \"%.3f\",$tdelivered_tariff2}")

And rereading my statement from yesterday about the ELECTRICITY DELIVERED idea… we would always have a confusion about what delivered energy actually is, to the house or to the grid? On this second thought, I reckon it would be best to avoid DELIVERED altogether and go for ‘ELECTRICITY USED’ and ‘ELECTRICITY PRODUCED’. Produced is technically not correct either (since it is solar production minus electricity used in the house), but I think it is the most intuitive wording of what the value is showing.

If people ask why I use full pathnames in my scripts, I like to know what version of a command I am using and to avoid being ‘hacked’ by a script that says grep and is in my search path and does a grep and make a local user root. I typically use defines in the beginning of my shell scripts, like:
AWK=/usr/bin/awk
and use them in the commands like
delivered_tariff2=$($AWK “BEGIN {printf “%.3f”,$tdelivered_tariff2}”)

In example code that seems a bit odd however, but that is how the script is running here. If for anything, I like the fact that this bash script is providing a more or less readable explanation of what your script part on the raspberry is actually doing. Which might be helpful in debugging.

This looks very promising to integrate my (just installed) smart meter with Homey.
I have a spare Raspberry Pi lying around, but I wouldn’t know where to start to get the readings from the meter to the P1 app on Homey? Could anyone help me looking in the good direction or has a nice guide for me?

Thank you for creating and sharing this!

Can one use a Raspberry Pi Zero W to run this script?

Reason for asking: I noticed that the memory usage increases to around 90% within half an hour after starting the script, while Homebridge running on a Pi 3 Model B+ stays around 5% when running for days/weeks.

While I’m not using the app, I do use a Raspberry Pi to publish smartmeter data to an MQTT Host. Maybe this script can help you out a bit.

P1 smartmeter.py
#!/usr/bin/env python
# Python script om P1 telegram weer te geven in Node-RED
import datetime
import re
import serial
import paho.mqtt.publish as publish
from time import sleep

# Settings
MQTT_HOST = "192.168.1.38"
MQTT_PORT=32772
MQTT_USER="user"
MQTT_PASS="password"
MQTT_TOPIC="meter"
MQTT_AUTH = {
        "username": MQTT_USER,
        "password": MQTT_PASS
}

T_SLEEP = 15
# Seriele poort confguratie
ser = serial.Serial()

# DSMR 4.0/4.2 > 115200 8N1:
ser.baudrate = 115200
ser.bytesize = serial.EIGHTBITS
ser.parity = serial.PARITY_NONE
ser.stopbits = serial.STOPBITS_ONE

ser.xonxoff = 0
ser.rtscts = 0
ser.timeout = 12
ser.port = "/dev/ttyUSB0"
ser.close()

kwhtot = 0
kwhoud = 0
kwhverschil = 0

gastot = 0
gasoud = 0
gasverschil = 0

while True:
        try:
                ser.open()
                checksum_found = False
                gasflag = 0
                kwh1 = 0
                kwh2 = 0
                gas = 0
                watt = 0
                rawdata = []

            while not checksum_found:
                    telegram_line = ser.readline() # Lees een seriele lijn in.
                    telegram_line = telegram_line.decode('ascii').strip() # Strip spaties en blanke regels
                    rawdata.append(telegram_line)

                    if re.match(b'(?=1-0:1.7.0)', telegram_line): #1-0:1.7.0 = Actueel verbruik in kW 1-0:1.7.0(0000.54*kW)
                            kw = telegram_line[10:-4] # Knip het kW gedeelte eruit (0000.54)
                            watt = float(kw) * 1000 # vermengvuldig met 1000 voor conversie naar Watt (540.0)
                            watt = int(watt) # rond float af naar heel getal (540)

                    if re.match(b'(?=1-0:1.8.1)', telegram_line): #1-0:1.8.1 - Hoog tarief / 1-0:1.8.1(13579.595*kWh)
                            kwh1 = telegram_line[10:-5] # Knip het kWh gedeelte eruit (13579.595)

                    if re.match(b'(?=1-0:1.8.2)', telegram_line): #1-0:1.8.2 - Laag tarief / 1-0:1.8.2(14655.223*kWh)
                            kwh2 = telegram_line[10:-5] # Knip het kWh gedeelte eruit (14655.223)

                    if gasflag == 1:
                            gasflag = 0

                    if re.match(b'(?=0-1:24.2.1)', telegram_line): #0-1:24.2.1 - Gasverbruik
                            gasflag = 1
                            if gasoud < 1:gas = float(telegram_line[26:-4])

                    # Check wanneer het uitroepteken ontvangen wordt (einde telegram)
                    if re.match(b'(?=!)', telegram_line):
                            checksum_found = True

            ser.close()

            # MQTT PUBLISH
            publish.single(MQTT_TOPIC + "/electricity", watt, hostname=MQTT_HOST, port=MQTT_PORT, auth=MQTT_AUTH)
            publish.single(MQTT_TOPIC + "/gas", gas, hostname=MQTT_HOST, port=MQTT_PORT, auth=MQTT_AUTH)
            publish.single(MQTT_TOPIC + "/telegram", "\n".join(rawdata), hostname=MQTT_HOST, port=MQTT_PORT, auth=MQTT_AUTH)
    except Exception as err:
		try:
            publish.single(MQTT_TOPIC + "/error", err, hostname=MQTT_HOST, port=MQTT_PORT, auth=MQTT_AUTH)
		except:
			pass
    sleep(T_SLEEP)

The idea I got from this site

For the second night in a row (i.e., every night since I’m experimenting with this) the Pi (Zero) is completely unresponsive in the morning. I have to power it off and on again to be able to SSH into it. :cry:

When I checked this morning the Homey App reported an extremely high current usage of gas (almost equal to the total amount of gas used since the smart meter got installed).

Known issue or new bug?

How can I delete the faulty data sample?

do you use the node-p1-reader example js script for this?

Yes, I use the (fork of the) P1-Reader script that uploads data to the Homey.

Ah okay, I try to use it too… but it uses a lot of memory (>700Mb) and CPU on a Pi 3B … and after a few Hours it crashes and even pm2 is not able to keep it “Alive” but it is a really easy task , read serial, parse and push data… so i’m Not sure what the problem is … is this something you Experience too?

I limited the memory Node.js can use and removed logging to the console.

The script is still using almost all resources, but I have received data for more than a week now.

Maybe someone can help me with the steps, beacause it’s not working for me yet… I know that my P1 cable is connected correctly to my Raspberry Pi because I can get live readings with command “cu -l /dev/ttyUSB0 -s 115200”

I’m not sure what to fill in in the examples file in the port section, because should it be /dev/ttyUSB0? Or should it be /dev/tty-usbserial1

If I leave those lines uncommented, it’s not working…

Also, there’s no possibility for settings in Homey.

Please help!

My P1 has 2 connections. One RJ11 port to the smart meter and one RJ45 port to the network.
If you put the network cable into your RPi, then Homey won’t be able to see it in your network and will not be able to connect to or use it.
Or am i completely wrong?

@Koktail
Is it possible to extend the P1 App so that the smartmeter-obis counter (SmlProtocol,D0Protocol,JsonEfrProtocol) can also be read and these are visible in the Homey App?

smartmeter-obis:

I don’t see any functionality in the smartmeter-obis package for posting the read result to and endpoint (which would be Homey). How did you envision using this?

My guess is that this would require writing a new package which implements the smartmeter-obis package and sends the data to Homey. Basically another implementation of https://github.com/koktaildotcom/node-p1-reader but then implement smartmeter-obis to read the data.

I’m also running on a RBPI ZeroW,
I’ve disabled logging to console and limited memory for node.js using -node --max-old-space-size=“Value”.
I’m unable to find the perfect setting.
Or process runs out of memory, or RBPI running out of memory.

What are the setting’s you are using?

Aside from this, why is memory usage this high? If I understand correctly it should be read --> parse --> post --> forget… Right?

Likely a memory leak.

Ok, is this fixable by changing the code of the script or should I do something else?