Modular System for Automotive Monitoring and Assistance (MSAMA)

Author: Duțu Alin Călin
Master: SRIC

Introduction

Context

A friend bought a new car without any other equipment besides the standard. This car was way bigger than his older one and took time to get used to its size. While executing forward parking, he would constantly think that he was too close to the front car or the wall, and he would always use the hand brake, get out, and check to see that there was more space remaining. He complained that he didn't have a front parking sensor on his car and I don't believe he is the only one having this problem.

Problem Description

In 2022 it became mandatory for car companies to include an essential kit with assistance and safety sensors in their new models, but some models still lack essential sensors. However, according to DRPCIV, at the end of 2023, more than 81% of the cars in Romania had been used for more than 10 years [1], so it is clear that most of them don't have the essential set of sensors. So this project aims to implement a system that makes the sensor integration in cars straightforward and easy to understand.

Architecture

System overview

Hardware

Components List

4 x ESP-WROOM-32
1 x Arduino Uno
1 x NodeMCU32S
2 x HC-SR04
1 x DHT11
1 x Wi-Fi ESP32-CAM
1 x BMP280
5 x MB102, 3.3V - 5V, 700mA
6 x 9V battery cable adapters

Components Diagrams

Camera Module

The camera module is used mainly to capture and provide the video output, process it, and manage Wi-Fi connections

The camera component (ESP32-CA) is situated at the top layer and is used mainly to capture and provide the video output to the motherboard
The motherboard component (ESP32-CAM-MB) is situated at the bottom layer and is used to process the captured video and manage Wi-Fi connections for the live server

The module itself and the pinout:

Pollution Module

The pollution module consists of an MQ7 component that measures the quantity of CO and sends it to the ESP32 component, which transmits the data to the broker through Wi-Fi.
The circuit diagram:

Pressure Module

The pressure module consists of a BMP280 component that measures a tire's pressure, temperature, and altitude and sends the values to the ESP32 component, which transmits the data to the broker through Wi-Fi.
The circuit diagram:

Proximity Module

The proximity module consists of 2 HC-SR04 components that measure the distance between the sensor and the closest object and sends the values to the ESP32 component, which transmits the data to the broker through Wi-Fi. One sensor is for the front and one for the back.
The circuit diagram:

Temperature Module

The temperature module consists of a DHT11 component that measures the inside temperature and sends the values to the Arduino Uno component, which transmits the data to the broker through UART.
The circuit diagram:

Broker Module

– Reference here
The broker module consists of an ESP32 that will mainly collect all the data sent to it, process it, and send it to the Firebase module through Wi-Fi. Additionally, the module has a button that will trigger the gear_mode which will be discussed in the [Software section].
The circuit diagram:

Firebase Module

The Firebase module has a simple ESP that will send the data processed by the broker and send it to the Firebase database.

Software

MQTT protocol

MQTT is an OASIS standard messaging protocol for the Internet of Things (IoT). It is designed as an extremely lightweight publish/subscribe messaging transport which is ideal for connecting remote devices with a small code footprint and minimal network bandwidth. MQTT today is used in various industries, such as automotive, manufacturing, telecommunications, oil and gas, etc. [2]. For the project, the MQTT is used to easily transfer data from multiple sensors to a single broker that will process all the data provided by the sensors with some functionality additions. The MQTT communication visualization is presented as follows:

Sensor modules

At initialization, every module besides the Temperature module will connect to the provided Wi-Fi followed by the connecting to the MQTT broker, and will retry until it connects. If the connection with the MQTT broker is lost, the sensors at the beginning of a new loop cycle will retry until the connection with the broker is set.

Every sensor will communicate with its microcontroller through either SPI or digital reading. Every client will asynchronously get the values from the sensors and send them to the broker through Wi-Fi if the microcontroller is an ESP 32 or UART if it is an Arduino UNO.

Additionally, the proximity sensor will wait for the gear_mode to change to parking mode to send proximity data. The gear_mode shall be interpreted as 1 when the car is parking and 0 otherwise. The gear_mode is controlled by the broker.

Broker

The broker's main functionality is to aggregate data from sensors. It will receive data through MQTT protocol topics, aggregate them, and send them to the Firebase module through a specific MQTT topic for Firebase.

At initialization, the broker will connect to the Wi-Fi, initialize data variables to empty strings, and wait for new connections and data to be received. If no data has been received or the Firebase Module is not connected, the broker won't send any data to Firebase.

Every time a sensor connects, it will show in Serial that it is connected. If a client disconnects, there will be a timeout sequence of around 70 seconds which is enough time for sensors to reconnect. At the end of a connection, the broker will announce the loss of connection in the Serial interface and set the corresponding variable to an empty string.

The broker is also in control of the gear mode feature which is controlled at the press of a button. If the gear_mode is set to 0, everything will work as usual. However, if the button is pressed, the gear_mode will change, the broker will send it to the Proximity Module through the gear_mode topic and then, it will start receiving data from the proximity sensors as well. Due to the importance of this information, it must be processed quickly, so the broker will send more data to Firebase in a significantly shorter amount of time.

Switching back to normal mode will turn off the proximity sensors and the data sending speed will get back to normal.

Firebase

The Firebase module at initialization will connect to the Internet via Wi-Fi and try to establish a connection with the Firebase database and it will retry until it is established. After the Firebase connection is set, the module will connect to the MQTT broker and by doing so, will let the broker know that the Firebase connection is established.

MSAMA App

This app was created with the MIT App Inventor tool along with other plugins that completed the implementation.

The MSAMA app consists of 3 screens:

Car status screen
Dashcam screen
Parking mode screen

Car status screen

This screen is used to show the data saved in the Firebase Database. The app will fetch the database once every 2 seconds and will update the data accordingly. If an empty string is received for a specific value, then its text field won't show only the unit of measurement only but rather make it disappear from the screen.

Additionally, if there is a camera link that is associated with the camera being on, the camera button will be visible, but if not, it will be invisible and therefore inaccessible.

Dashcam screen

The dashcam screen is used to show live what is the camera capturing. The user can go back to the first screen by pressing the back button from the phone UI.

Parking mode screen

The parking mode screen is automatically activated when the gear_mode is set to 1. The user cannot go back through the phone UI, as a safety feature. It will automatically go back to the first screen when gear_mode is set to 0.

On this screen, the app will show visual and auditive feedback based on the proxy values fetched in the database.

Source code

The source code is available at this link.

Results

Conclusions and Future work

The project successfully demonstrates the implementation of a Modular System for Automotive Monitoring and Assistance. The system effectively collects real-time data. Sends and stores data through a Firebase database using an efficient and fast protocol implemented over MQTT. The user can always retrieve the data stored in the database and interact with it. The system won't be affected by a sensor malfunction or connection drop which assures continuity. The app can interact with the user depending on certain conditions.

Future work could focus on enhancing the user experience and functionality of the system. The app could receive future UI updates that enhance user experience. Additionally, the integration of new sensors in the MSAMA system could significantly enhance the functionality available to the user.

References

Table of Contents