uni-meter is a small tool that emulates a Shelly Pro3EM device for the usage with the Hoymiles MS-A2, the Marstek Venus storage and the EZHI hybrid inverter. It is not a full implementation of a Shelly Pro3EM device, currently only the parts that are needed by the storages are implemented.
The real electrical meter data currently can be gathered from the following devices:
- Fronius smart meter
- Generic HTTP (configurable HTTP interface, usable for many devices)
- ioBroker datapoints (via simple API adapter)
- MQTT
- Shelly 3EM
- SHRDZM smartmeter interface module (UDP)
- SMA energy meter / Sunny Home Manager (UDP protocol)
- SMD120 modbus energy meter (via Protos PE11) (SMD630 could be added, I have no test device)
- Tasmota IR read head (via HTTP)
- Tibber Pulse (local API)
- VzLogger webserver
The idea is to further enhance the tool in the future by adding more input and output devices to get a universal converter between different electrical meters, inverters and storage systems.
The release versions can be downloaded from the GitHub releases.
You just have to download the uni-meter-<version>.tgz
archive.
The project can be build using Java 17 and Maven 3.8.5 or later by simply typing
mvn install
within the project's root directory.
Afterward you will find a uni-meter-<version>.tgz
archive in the target
directory which can be used to deploy the tool to the target system which most likely is a Raspberry Pi.
Since some people asked, it is not necessary to build the tool yourself. For a normal user, that does not want to modify the source code, the provided prebuild releases are sufficient.
Not all the steps mentioned in this documentation are necessary for every setup. A lot of configuration options like throttling of the sampling frequency and default power values are optional and not necessary for most of the users. Only for fine-tuning the system, these options may be specified.
For a Marstek Venus storage you just have to enable JSON RPC over UDP port 1010. It is not necessary to change the device id of the virtual Shelly and announce the tool via mDNS using the Avahi daemon. These steps can just be skipped.
For a Hoymiles MS-A2 storage you have to announce the tool via mDNS using the Avahi daemon. You also have to change the device id of the virtual Shelly to a unique value. Otherwise, the Hoymiles app refuses to attach the Shelly to your Hoymiles system. Enabling JSON RPC over UDP is not necessary for the Hoymiles storage.
If you want to use an ESP32 or a similar system which does not support Java 17, there is another project called Energy2Shelly which is written in C++ and can be used as an alternative on such systems.
To install the tool on a Raspberry Pi, you need to have a Java 17 runtime installed. To check if this is the case, type
java -version
If the output shows a version 17 or later, you are good to go. If not, you can install the OpenJDK 17 using the following commands:
sudo apt update
sudo apt install openjdk-17-jre
Afterward, you can copy the uni-meter-<version>.tgz
archive to the Raspberry Pi. In theory, you can extract the archive
to any location you like, but all the scripts and configuration files included assume an installation in the /opt
directory. So preferably you should extract it to the /opt
directory using the following commands:
sudo tar xzvf uni-meter-<version>.tgz -C /opt
sudo ln -s /opt/uni-meter-<version> /opt/uni-meter
To make the tool discoverable by the Hoymiles storage via mDNS, the avahi-daemon
is used. On recent Raspbian versions,
the avahi-daemon
is already installed and running. To check if this is the case, type
sudo systemctl status avahi-daemon
If you see an output like active (running)
, you are good to go. If not, you can install the avahi-daemon
using the
sudo apt install avahi-daemon
and enable it using the following command:
sudo systemctl enable avahi-daemon
sudo systemctl start avahi-daemon
Afterward you can copy the provided service files to the /etc/avahi/services
directory.
sudo cp /opt/uni-meter/config/avahi/*.service /etc/avahi/services/
The daemon will automatically pick up the new service files and announce the tool via mDNS. No restart of the
avahi-daemon
is needed.
The provided service files announce the Shelly Pro3EM emulator running on port 80. Even if I used another port within the service file, the Hoymiles storage does not seem to evaluate the port information and still connects to port 80. Maybe future Hoymiles firmware version also will honor the port information.
In the meantime this has the disadvantage that the uni-meter
tool has to bind to port 80 which requires root privileges.
Using nginx
or using setcap
to run the tool as a non-root user is possible but not covered in this documentation.
The configuration is done using a configuration file in the HOCON format.
The provided start script assumes the configuration file to be located in the /etc
directory. To do so, copy the
provided configuration file to that location using:
sudo cp /opt/uni-meter/config/uni-meter.conf /etc/uni-meter.conf
Then use your favorite editor to adjust the configuration file to your needs as described in the following sections.
It seems as if the Hoymiles storage expects a global unique identifier for the Shelly device. This Shelly device id
is freely configurable and can be set in the /etc/uni-meter.conf
file. Please configure the mac address and the hostname
of your virtual Shelly Pro3EM device. The mac address should be an arbitrary 12 character long hexadecimal string.
The hostname should end with the mac address in lower case.
uni-meter {
# ...
output-devices {
shelly-pro3em {
device {
mac = "B827EB364242"
hostname = "shellypro3em-b827eb364242"
}
}
}
#...
}
Additionally, you have to modify the two service files you copied into the /etc/avahi/services
directory, so that the correct
device name is announced. Please replace the shellypro3em-b827eb364242 hostname with the one you have configured in the
uni-meter.conf
file. Each service file contains the hostname twice. Once in the <name> tag and once in the <txt-record> tag.
<?xml version="1.0" standalone='no'?>
<!DOCTYPE service-group SYSTEM "avahi-service.dtd">
<service-group>
<name replace-wildcards="yes">shellypro3em-b827eb364242</name>
<service protocol="ipv4">
<type>_http._tcp</type>
<port>80</port>
<txt-record>gen=2</txt-record>
<txt-record>id=shellypro3em-b827eb364242</txt-record>
<txt-record>arch=esp8266</txt-record>
<txt-record>fw_id=20241011-114455/1.4.4-g6d2a586</txt-record>
</service>
</service-group>
As a default the JSON RPC over UDP interface of the Shelly Pro3EM emulator is disabled. To enable it, configure the
udp-port
and optionally the udp-interface
in the /etc/uni-meter.conf
file:
uni-meter {
# ...
output-devices {
shelly-pro3em {
#...
udp-port = 1010
udp-interface = "0.0.0.0" # default, can be omitted
#...
}
}
#...
}
In some setups with a higher latency until the real electrical meter readings are available on the output side, it might be necessary to throttle the sampling frequency of the output data. Otherwise, it might be possible that the storage oversteer the power production and consumption values and that they are fluctuating too much around 0 (see the comments and findings to this issue).
To throttle the sampling frequency you can configure a min-sample-period
in the /etc/uni-meter.conf
file. This
configuration value specifies the minimum time until the next output data is delivered to the storage.
uni-meter {
#...
output-devices {
shelly-pro3em {
#...
min-sample-period = 5000ms
#...
}
}
#...
}
In some setups, it might be necessary to add a static offset to the power values. This can be the case if the real electrical meter readings are not 100% accurate to your household's electrical meter readings.
You can either configure a power offset for the single phases or a total power offset. The phase power offsets take precedence over the total power offset. If at least one phase power offset is configured, the total power offset is ignored.
Setting the power offset is done in the /etc/uni-meter.conf
file:
uni-meter {
#...
output-devices {
#...
shelly-pro3em {
#...
power-offset-total =0
power-offset-l1 = 0
power-offset-l2 = 0
power-offset-l3 = 0
}
}
}
If the physical input device is not reachable and no power values are available for a certain time, the uni-meter will fall back to default power values. Without any configuration that happens after one minute and the power will default to 0 watts. If you need a different behaviour, you can configure this using the following configuration below. Single phase power values take precedence over the total power value. If at least one phase power value is configured, the total power value is ignored.
uni-meter {
#...
output-devices {
#...
shelly-pro3em {
#...
# These are the defaults used without any configuration:
forget-interval = 1m
default-power-total = 0
default-power-l1 = 0
default-power-l2 = 0
default-power-l3 = 0
}
}
}
In its default configuration, the uni-meter listens on port 80 for incoming HTTP requests. That port can be changed to
for instance port 4711 by adding the following parts to your /etc/uni-meter.conf
file:
uni-meter {
# ...
http-server {
port = 4711
}
output-devices {
shelly-pro3em {
# ...
port = 4711
}
}
}
A Fronius smart meter can be accessed using the generic-http input source. To access the Fronius smart meter, use the
following configuration in the /etc/uni-meter.conf
file:
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.generic-http"
input-devices {
generic-http {
url = "http://192.168.x.x/solar_api/v1/GetMeterRealtimeData.cgi?Scope=System"
power-phase-mode = "tri-phase"
channels = [{
type = "json"
channel = "power-l1"
json-path = "$.Body.Data.0.PowerReal_P_Phase_1"
},{
type = "json"
channel = "power-l2"
json-path = "$.Body.Data.0.PowerReal_P_Phase_2"
},{
type = "json"
channel = "power-l3"
json-path = "$.Body.Data.0.PowerReal_P_Phase_3"
}]
}
}
}
The generic HTTP input source can be used to gather the electrical meter data from any device providing the data via an HTTP get request as JSON value. You have to configure the complete URL where the data is gathered from, including the entire path and query parameters.
The input source can operate in two modes. Either in a mono-phase
mode, where the power and/or the energy data is
provided as a single value for all three phases, or in a tri-phase
mode, where the power and/or the energy data is
provided as a separate value for each phase.
The input data is gathered through channels. The following channels exist for the different energy and power phase modes:
- Power
mono-phase
power-total
- total current power
- Power
tri-phase
power-l1
- current power phase 1power-l2
- current power phase 2power-l3
- current power phase 3
- Energy
mono-phase
energy-consumption-total
- total energy consumptionenergy-production-total
- total energy production
- Energy
tri-phase
energy-consumption-l1
- energy consumption phase 1energy-consumption-l2
- energy consumption phase 2energy-consumption-l3
- energy consumption phase 3energy-production-l1
- energy production phase 1energy-production-l2
- energy production phase 2energy-production-l3
- energy production phase 3
For each channel to be read, you have to configure where the data is gathered from and what type it is. Currently only
the json
type is supported, but in the future, other types might be added.
For channels in JSON format, an additional JSON path has to be provided which specifies which part of the JSON data contains the actual value.
Additionally, each channel has a scale
property which can be used to scale the data. The default scale is 1.0 and can
be omitted.
So a /etc/uni-meter.conf
file reading the data from a VzLogger webserver could look like this:
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.generic-http"
input-devices {
generic-http {
url = "http://vzlogger-server:8088"
#username = "username"
#password = "password"
power-phase-mode = "mono-phase"
energy-phase-mode = "mono-phase"
channels = [{
type = "json"
channel = "energy-consumption-total"
json-path = "$.data[0].tuples[0][1]"
scale = 0.001
},{
type = "json"
channel = "energy-production-total"
json-path = "$.data[1].tuples[0][1]"
scale = 0.001
},{
type = "json"
channel = "power-total"
json-path = "$.data[2].tuples[0][1]"
}]
}
}
}
Reading ioBroker datapoints as input source can be done using the generic http interface on uni-meter side and using
the simpleAPI adapter on ioBroker side. When the simpleAPI adapter
is installed and configured on the ioBroker, you can use the following configuration in the /etc/uni-meter.conf
file:
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.generic-http"
input-devices {
generic-http {
# Adjust the IP address of the ioBroker and the datapoints to read according to your needs
url = "http://192.168.x.x:8082/getBulk/smartmeter.0.1-0:1_8_0__255.value,smartmeter.0.1-0:2_8_0__255.value,smartmeter.0.1-0:16_7_0__255.value/?json"
# sample ioBroker output: [
# {"id":"smartmeter.0.1-0:1_8_0__255.value","val":16464.7379,"ts":1740054549023,"ack":true},
# {"id":"smartmeter.0.1-0:2_8_0__255.value","val":16808.0592,"ts":1740054549029,"ack":true},
# {"id":"smartmeter.0.1-0:16_7_0__255.value","val":4.9,"ts":1740054549072,"ack":true}
# ]
power-phase-mode = "mono-phase"
energy-phase-mode = "mono-phase"
channels = [{
type = "json"
channel = "energy-consumption-total"
json-path = "$[0].val"
scale = 0.001
},{
type = "json"
channel = "energy-production-total"
json-path = "$[1].val"
scale = 0.001
},{
type = "json"
channel = "power-total"
json-path = "$[2].val"
}]
}
}
}
The MQTT input source can operate in two modes. Either in a mono-phase
mode, where the power and/or the energy data is
provided as a single value for all three phases, or in a tri-phase
mode, where the power and/or the energy data is
provided as a separate value for each phase.
The input data is gathered through channels. Each channel has a unique identifier and a topic where the data is gathered from. The following channels exist for the different energy and power phase modes:
- Power
mono-phase
power-total
- total current power
- Power
tri-phase
power-l1
- current power phase 1power-l2
- current power phase 2power-l3
- current power phase 3
- Energy
mono-phase
energy-consumption-total
- total energy consumptionenergy-production-total
- total energy production
- Energy
tri-phase
energy-consumption-l1
- energy consumption phase 1energy-consumption-l2
- energy consumption phase 2energy-consumption-l3
- energy consumption phase 3energy-production-l1
- energy production phase 1energy-production-l2
- energy production phase 2energy-production-l3
- energy production phase 3
Each channel is linked to a topic where the data is gathered from and has a type which specifies how the data is
stored within the MQTT topic. Currently, two types are supported: value
and json
. Use the value
type for data
stored as a number string within the topic. Use the json
type for data stored as JSON within the topic. For
channels in JSON format, an additional JSON path has to be provided which specifies which part of the JSON data
contains the actual value.
Additionally, each channel has a scale
property which can be used to scale the data. The default scale is 1.0 and can
be omitted.
So a /etc/uni-meter.conf
file for a MQTT input source could look like this:
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.mqtt"
input-devices {
mqtt {
url = "tcp://127.0.0.1:1883"
#username = "username"
#password = "password"
power-phase-mode = "mono-phase"
energy-phase-mode = "mono-phase"
channels = [{
type = "json"
topic = "tele/smlreader/SENSOR"
channel = "power-total"
json-path = "$..power"
scale = 1.0 # default, can be omitted
},{
type = "json"
topic = "tele/smlreader/SENSOR"
channel = "energy-consumption-total"
json-path = "$..counter_pos"
},{
type = "json"
topic = "tele/smlreader/SENSOR"
channel = "energy-production-total"
json-path = "$..counter_neg"
}]
}
}
}
To use a Shelly 3EM as an input source, set up the /etc/uni-meter.conf
file as follows
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.shelly-3em"
input-devices {
shelly-3em {
url = "<shelly-3em-url>"
}
}
}
Replace the <shelly-3em-url>
placeholder with the actual URL of your Shelly 3EM device.
This input device is currently totally untested, as I have no test device. If you have a Shelly 3EM device, please provide feedback if it works or not via the GitHub issues.
To use a SHRDZM smartmeter interface providing the smart meter readings via UDP, set up the /etc/uni-meter.conf
file
as follows
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.shrdzm"
input-devices {
shrdzm {
port = 9522
interface = "0.0.0.0"
}
}
}
The above configuration shows the default values for the ShrDzm device which are used, if nothing is provided. If you want to use a different port or interface, you have to adjust the values accordingly.
To use a SMA energy meter or a Sunny Home Manager as an input source, set up the /etc/uni-meter.conf
file as follows
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.sma-energy-meter"
input-devices {
sma-energy-meter {
port = 9522
group = "239.12.255.254"
//susy-id = 270
//serial-number = 1234567
network-interfaces =[
"eth0"
"wlan0"
// "192.168.178.222"
]
}
}
}
The above configuration shows the default values which are used, if nothing is provided. If your port
and group
are
different, you have to adjust the values accordingly.
If no susy-id
and serial-number
are provided, the first detected device will be used. Otherwise, provide the values
of the device you want to use.
The network interfaces to use are provided as a list of strings. Either specify the names or the IP addresses of the interfaces you want to use.
To use a SMD120 modbus energy meter via a Protos PE11 as an input source, set up the /etc/uni-meter.conf
file as
follows:
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.smd120"
input-devices {
smd120 {
port = 8899
}
}
}
To use a Tasmota IR read head as an input source, set up the /etc/uni-meter.conf
file as follows:
uni-meter {
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.tasmota"
input-devices {
tasmota {
url = "http://<tasmota-ir-read-head-ip>"
# username=""
# password=""
power-json-path = "$..curr_w"
power-scale = 1.0 # default, can be omitted
energy-consumption-json-path = "$..total_kwh"
energy-consumption-scale = 1.0 # default, can be omitted
energy-production-json-path = "$..export_total_kwh"
energy-production-scale = 1.0 # default, can be omitted
}
}
}
}
Replace the <tasmota-ir-read-head-ip>
placeholder with the actual IP address of your Tasmota IR read head device.
If you have set a username and password for the device, you have to provide them as well.
Additionally, you have to configure the JSON paths for the power, energy consumption and energy production values to access the actual values within the JSON data. If you have to scale these values, you can provide a scale factor which is 1.0 as a default.
The Tibber Pulse local API can be used as an input source. To use this API, the local HTTP server has to be enabled on the Pulse bridge. How this can be done is described for instance here marq24/ha-tibber-pulse-local.
If this API is enabled on your Tibber bridge, you should set up the /etc/uni-meter.conf
file as follows
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.tibber-pulse"
input-devices {
tibber-pulse {
url = "<tibber-device-url>"
node-id = 1
user-id = "admin"
password = "<tibber-device-password>"
}
}
}
Replace the <tibber-device-url>
and <tibber-device-password>
placeholders with the actual values from your environment.
The node-id
and user-id
are optional and can be omitted if the default values from above are correct. Otherwise,
adjust the values accordingly.
To use the VzLogger webserver as an input source set up the /etc/uni-meter.conf
file as follows and replace the
<vzlogger-host>
and <vzlogger-port>
placeholders with the actual host and port of your VzLogger webserver.
Additionally provide the channel UUIDs of your system.
uni-meter {
output = "uni-meter.output-devices.shelly-pro3em"
input = "uni-meter.input-devices.vz-logger"
input-devices {
vz-logger {
url = "http://<vzlogger-host>:<vzlogger-port>"
energy-consumption-channel = "5478b110-b577-11ec-873f-179XXXXXXXX"
energy-production-channel = "6fda4300-b577-11ec-8636-7348XXXXXXXX"
power-channel = "e172f5b5-76cd-42da-abcc-effeXXXXXXXX"
}
}
}
You will find that information in the VzLogger configuration file. As a
default, the VzLogger is configured in the /etc/vzlogger.conf
file. Make sure that the VzLogger provides its
readings as Web service and extract the needed information from that file:
{
// ...
// Build-in HTTP server
"local": {
"enabled": true, // This has to be enabled to provide the readings via HTTP
"port": 8088, // Port used by the HTTP server
// ...
}
// ...
"meters": [
{
// ...
"channels": [{
"uuid" : "5478b110-b577-11ec-873f-179bXXXXXXXX", // UUID of the energy consumption channel
"middleware" : "http://localhost/middleware.php",
"identifier" : "1-0:1.8.0", // 1.8.0 is the energy consumption channel
"aggmode" : "MAX"
},{
"uuid" : "6fda4300-b577-11ec-8636-7348XXXXXXXX", // UUID of the energy production channel
"middleware" : "http://localhost/middleware.php",
"identifier" : "1-0:2.8.0", // 2.8.0 is the energy production channel
"aggmode" : "MAX"
},{
"uuid" : "e172f5b5-76cd-42da-abcc-effef3b895b2", // UUID of the power channel
"middleware" : "http://localhost/middleware.php",
"identifier" : "1-0:16.7.0", // 16.7.0 is the power channel
}]
}
]
}
After you have adjusted the configuration file, you can start the tool using command
sudo /opt/uni-meter/bin/uni-meter.sh
If everything is set up correctly, the tool should start up, and you should see an output like
24-12-04 07:29:08.006 INFO uni-meter - ##################################################################
24-12-04 07:29:08.030 INFO uni-meter - # Universal electric meter converter 0.9.0 (2024-12-04 04:58:16) #
24-12-04 07:29:08.031 INFO uni-meter - ##################################################################
24-12-04 07:29:08.033 INFO uni-meter - initializing actor system
24-12-04 07:29:10.902 INFO org.apache.pekko.event.slf4j.Slf4jLogger - Slf4jLogger started
24-12-04 07:29:11.707 INFO uni-meter.controller - creating Shelly3EM output device
24-12-04 07:29:11.758 INFO uni-meter.controller - creating VZLogger input device
24-12-04 07:29:16.254 INFO uni-meter.http.port-80 - HTTP server is listening on /[0:0:0:0:0:0:0:0]:80
Now you should be able to connect your Hoymiles storage to the emulator using the Hoymiles app.
To start the tool automatically on boot, you can use the provided systemd service file. To do so, create a symlink
within the /etc/systemd/system
directory using the following command:
sudo ln -s /opt/uni-meter/config/systemd/uni-meter.service /etc/systemd/system/uni-meter.service
Afterward, you can enable the service using the following command so that it will be automatically started on boot:
sudo systemctl enable uni-meter
To start and stop the service immediately run
sudo systemctl start uni-meter
sudo systemctl stop uni-meter
The status of the service can be checked using
sudo systemctl status uni-meter
If you start the tool directly from the command line, all error messages will be printed to the console. If you start the
tool using the systemd service, you can check the log messages in /var/log/uni-meter.log
.
As a default the log level is just set to INFO
. For debugging purposes you can set the log level to DEBUG
or even
TRACE
within the /opt/uni-meter/config/logback.xml
file.
<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN">
<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
<encoder>
<pattern>%d{yy-MM-dd HH:mm:ss.SSS} %-5level %-24logger - %msg%n</pattern>
</encoder>
</appender>
<root level="INFO">
<appender-ref ref="STDOUT" />
</root>
<logger name="uni-meter" level="TRACE" additivity="false">
<appender-ref ref="STDOUT" />
</logger>
</Configuration>
A restart is necessary for these changes to take effect.