AME Software
AME Stack (Software)
.png)
Introduction/ Random Thoughts
Because there is no concurrency on the Arduino platform we have to implement it ourselves. The Arduino standard-loop() is extended through all the classes listed below. loop() is executed as often as possible and gives each class its turn to do "its thing" (in example, detect incoming IP data on the lowest level, or perform and publish a temperature measurement on the highest).
Instances on a higher lever of the stack should not have to know how to configure the instances they use on a lower level. In example, the MQTT level needs TCP connectivity but should not have to know whether this is being provided via Ethernet or WLAN. Because every thing needs a buzzword to market the idea, the concept of providing specific configuration from the outside has been called "Dependency Injection". In the case of the AME stack, a factory configures instances for each level and plugs them together.
AC_ESP8266
AC_ESP8266 provides access to the ESP8266 on C++ level, no more, no less. It uses two (blowing my own horn) clever functions to compose commands and to parse responses based on pattern expansion/ matching.
The buffer used for command assembly and response parsing is the only one in the whole class. Writing and reading IP data is performed streaming. Callbacks announce the arrival of data and loss of connection.
Regarding command assembly and response parsing I think even that could be migrated to streaming mode, though pattern matching would be not that easy, I guess.
Pattern expansion
Commands with parameters can be assembled using a pattern string and a "multi" (more on that later) argument string. The pattern string contains '$' (Dollar) signs with are filled with sections of the argument string. The "multi" string simply is a sequence of string terminated by 0.
In example, pattern string "$, $!" and argument string "Hello\0World\0" get expanded into "Hello, World!".
===Pattern matching
Responses with arguments can be parsed using a pattern string, producing a "multi" result string.
In example, using pattern string "time: $,count: $" to parse response "time: 47, count: 11" yield result string "47\011\0".
The code for parsing is "a bit" tricky (it took me a day to write it and half a day to really understand it a year later). This is why I explain it here, for myself and other.
Here is a description of the algorithm. A pattern consists of wildcards and blocks of text that must match. Wildcards and text blocks must alternate.
Example: Pattern "$ BLOCK_B $ BLOCK_D $" consists of three wildcards and two blocks of text that must match. Another way of seeing this is "wildcards separate fixed blocks of text". What you get is " BLOCK_B " and " BLOCK_D ".
The next task: Locate the fixed text blocks in the text to parse. All text blocks must be found; otherwise, there is no overall match.
Example: Text "BLOCK_A BLOCK_B BLOCK_C BLOCK_D BLOCK_E". BLOCK_B and BLOCK_D are contained in the text. Another way of seeing this is "fixed blocks of text separate 'free text'". What you get is "BLOCK_A", "BLOCK_C", and "BLOCK_D".
Next step: Return matching free text as result.
While this is not too tricky, the code is because it does it all in one go and not in three separate steps. But knowing how it is done is helpful in understanding the code.
Looping
loop() ensures arriving IP data and chip messages get read and interpreted.
Event Consumption
None; this is the lowest level of the stack.
Event Production
If IP data arrives or the connection was lost it informs the listener.
AC_TCP_ESP8266
AC_TCP_ESP8266 provides streaming TCP functionality based on - you guessed it - the ESP8266. It deals with loss of TCP and WLAN connectivity.
It does what is required to get the ESP working again, which, frankly sometimes is not easy and a pain in the back. This includes resetting the chip via software and - optionally - hardware (it uses the reset pin of the ESP).
If you want to, it even can turn the ESP off and on again. I think that does exhaust all options available.
Looping
loop() just calls loop() on ESP level.
Event Production
See next point.
Event Consumption
If IP data arrives is passes it on to the next higher level.
AC_MTTQ
AC_MTTQ provides standard MQTT functions with QoS 0 or 1. More honestly QoS "0.5" as it properly produces ACK messages to the broker, but arriving ACKs are discarded and missing ACKs don't prompt retries).
Unlike other libaries AC_MQTT does not get unhappy if a ping response gets lost while performing publish and subscribe operations.
Looping
loop() checks for the need to ping the broker if there was no recent network activity (to keep the connection alive) and whether the wait for a ping request has timed out. It checks, too, if the TCP connection is working.
It probably would be better design to leave that to the TCP level. Some philosophy is involved here as well: Should the TCP level make sure a created connection is being maintained, or is it better to back-delegate error handling to the caller as requirements for error handling may depend on the use of the TCP API?
Event Production
If a MQTT publication has been received it informs the listener.
Event Consumption
If IP data has been received it decodes the MQTT message.
AC_IoT/ Devices
Devices are derived from AC_IoT. AC_IoT provides convenience on device level with concepts such as centrally configured device name and MQTT-ID that get used wherever appropriate.
In example, a device with MQTT-ID "garden01" uses "garden01" as a prefix for topics in publications and subscriptions of its capabilities.
When the MQTT connection is established AC_IoT reports the version of the IoT framework, the device version, the IP and MAC addresses, and the number of connect attempts.
In example, the messages
- "garden01/versionIoT 1.038"
- "garden01/version 1.010"
- "garden01/connectAttempts 12"
- "garden01/ip 192.168.2.76"
- "garden01/mac 5d:de:ad:be:ef"
would be sent.
Looping
loop() re-establishes the MQTT connection if it has been lost. This currently happens on this level as information about device and capabilities is being published when a connection is made (again). It probably would be better design to leave connection maintenance to the MQTT level and just get notified when there was a connection problem and the connection got reestablished.
Event Production
See next point.
Event Consumption
Passes MQTT publications on to capabilities if the topic name matches or the publication is of generic nature.
Capabilities
Capabilities can easily be added to a device. For a gardening system you maybe add capabilities "temperature", "spoil moisture", "rain", and "relay" (ie, to turn an irrigation system on and off).
A capability encapsulates hardware intricacies, possess a built-in name, and provides configuration options such as a capability ID (a number), update intervals, I/O pins and so on.
In example, a capability encapsulating a BMP280 air pressure and temperature sensor with the built-in name "pressure" has been assigned capability ID "1". The capability will produce publications at a configurable interval using the name of the device, the name of the capability, the capability ID, and the attribute name.
Building on the example above, publications would use the topics
- "garden01/pressure01/pressure" and
- "garden01/pressure01/temperature"
Additionally, Capabilities report their version and input and output parameters when the MQTT connection is established. In example, the messages
- "garden01/pressure01/version 1.001"
- "garden01/pressure01/capa01 <pressure:<d>" and
- "garden01/pressure01/capa02 <temperature:<d>"
would be sent, indicating the production of two output attributes with the names "pressure" and "temperature" with numeric output format.
Looping
Optional. Can be used to in example perform and publish a temperature measurement every 60 seconds.
Event Production
May publish messages via MQTT.
Event Consumption
May subscribe to MQTT topics.
Arduino
Communications between Arduino and ESP8266 is via serial interface. The timing demands are high as the ESP mercilessly transmits arriving IP data at full speed. SoftwareSerial definitely is not an option. The hardware serial interface usually is used to program the Arduino, though. I use SPI programming und logging via I2C to free up the hardware interface (details). And since we don't need a bootloader anymore, we gain 2kB of flash memory as well.
To avoid problems I increased the size of the receive buffer from 64 to 256 bytes. This does eat RAM memory, but makes for safer operation. Your mileage may vary: if your messages are short 64 Bytes is just fine.
To adjust the receive buffer size, add the line
#define SERIAL_RX_BUFFER_SIZE 256
in HardwareSerial.h before the first #if. In newer installations of the Arduino IDE on Windows the file does not reside south of the program folder, but is located in AppData/Local, in example
C:\Users\<username>\AppData\Local\Arduino15\packages\arduino\hardware\avr\1.6.16\cores\arduino