OTA (Over The Air) - ESP32 - Arduino IDE

Article from own publication - Team project - TUKE 2019/2020 - Remote software update on ESP32 platform - modified.

Over The Air (OTA) is a designation for new firmware distribution and configuration methods for wireless devices. The advantage is the ability to distribute new software through a central point that can send software to all devices that support the method. In practice, we encounter OTA mainly from Windows, Android, or iOS operating systems, where the central point distributes the operating system update and all devices can notify the user of the update.

Based on the initial configuration of the device, the update can be downloaded and installed immediately or after confirmation by the user. Remote software update methods have also found application in the IoT (Internet of Things) industry. Intelligent devices can also be updated remotely, for example via a LAN, the Internet using a central point, or another implementation that supports OTA.

The ESP32 platform is also one of the IoT platforms that support remote software updates. As it can be programmed in different languages ​​(Wiring [Arduino], ESP-IDF, MicroPython, Lua, Node.Js and others), I have made a brief overview of the possibilities of OTA methods in Wiring (Arduino) and ESP-IDF (Espressif IoT Development Framework ).

The ESP32 has flash memory that can be customized for any application. In the case of using OTA, its division is implemented into the flash memory via Partition tables according to the rules in this table. By default without OTA, memory for the program is used - factory_app of 1MB and 3MB are reserved for the SPIFFS sector (Arduino IDE case).

ESP32 has 4MB flash memory as standard, there are also 8MB / 16MB versions. In most cases using OTA, partitions for OTA_1, OTA_2, OTA_DATA are added to the flash memory, while OTA_1.2 partitions have a size of 1MB in ESP-IDF. For OTA_DATA, a space of 2kB is reserved by default. The partition table is described by a simple .csv file.

Alternatively, there are also partition tables for the max app, where almost all the flash memory is reserved only for the main one - factory_app. The OTA_DATA partition is responsible for which partition will currently be bootable, it contains its name. If the partition is empty, the factory app will boot. If the factory app does not exist (Arduino core case with OTA partitions), OTA_0 will boot. The OTA_DATA partition is overwritten when the software is updated before booting and restarting ESP32.

OTA in Arduino IDE
Examples of Wiring implementations include the ability to update software over a LAN through an OTA network port directly from the Arduino IDE. For example of a software update via LAN - Basic OTA, the user who uploads the program to the device is required to be on the same network as ESP32. WiFi network data is set in the source code and is part of the example

For the first time, the OTA program must be physically uploaded to the ESP32 via a USB cable (or programmer). In order to subsequently upload the updated program to ESP32 via the OTA method, the user is required to have Python installed, which controls the entire update process (uploading a new program over the network) to ESP32. After the initial loading of the OTA example, in the Arduino IDE section, in addition to COM ports, a network port is displayed in the Tools → Ports section, through which it is possible to access ESP32 and upload a new program through it.

The network port is in the form esp32-MAC_ADDRESS and IP_ADDRESS, or it is possible to define the Hostname via mDNS service, which can also be started on ESP32 and the network port is displayed in the form: HOSTNAME at IP_ADDRESS.

After a successful update, the new program can be uploaded directly from the Arduino IDE development environment. It should be added that the new program must also contain the OTA part of the previous program if the user wants to update the program again in the future. If you upload any program without the OTA part, the program will be loaded, run without the possibility of another update (OTA is not included in the program).

ESP32 always receives a new program via OTA update even if a duplicate (same) program is loaded. It is also possible to run other functionalities in the program, such as selecting a specific OTA port, verifying the update from a verified source using a known password, or its hash in md5 format. The second way to update the software is to use the OTA Web Updater example.

In principle, it works by connecting the ESP32 to a WiFi network and launching a simple web interface on a web server on an HTTP port that can be accessed via a browser. It is possible to access the ESP32 IP address, or via a domain name, which can be set via the mDNS service - in the picture the domain name esp32webupdate.

However, in the case of the HTTP port, the address of the website is in the form domain_name.local for its correct loading in the browser. The web interface includes a form through which it is possible to upload a binary file of a new program and thus update the software. The binary file can be generated in the Arduino IDE and saved for later upload via the form.

Here, too, for the next update, the new file must contain the OTA Web Updater in addition to the new part of the program. The program is recorded for the first time via a USB cable / programmer. The form accepts any binary file, even a duplicate. For higher security, it is possible to use the login form above the original upload form. The advantage is also the possibility to connect to ESP32 from another network, the Internet if ESP32 works on a public IP address.

More advanced OTA implementations are included in the ESP-IDF framework, to which various other development tools can be used to run hardware and software security mechanisms for firmware integrity.

Integrity can be ensured by embedding a digital signature in the firmware binary, by verifying it at the Bootloader level at boot time, or by downloading updates itself. OTA in ESP-IDF uses Remote Updates - distributed by a remote server, from which ESP32 downloads the available firmware.

ESP-IDF includes mechanisms for verifying the firmware version, and if it is the same as the version of the existing running software, the boot flag will not change even if the content update is different.