Recent changes

ESP32 Home Monitoring System With Alerts

  • Author: Andrei Buhnici
  • Email: andrei.buhnici@stud.acs.pub.ro
  • Master: SSA

Introduction

This project is about a home monitoring system integrated with Thingsboard and Twilio for sms alerts. Since you can never be safe even with a locked door and an extremely good nose for gas leaks, a home monitoring system is essential for every house. A motion detector, sound detector and gas detector with cloud monitoring and phone alerts should be just enough for every household. And since we want to save as much energy as possible and want to know what happens when we leave the house, the monitoring can be turned off/on from the cloud and photos of the house are taken only when enough motion is detected.

Context

First of all the main flow of the project is the most important. I have a simple ESP32 which receives data from 3 sensors. An infrared sensor for motion detection HC-SR501 IR, a sound detection sensor KY-037 and a gas/smoke/fire sensor MQ-2, all from this kit. The infrared sensor and sound sensor send digital data, meaning they send 1 when configured threshold from integrated potentiometers is reached and/or passed, and 0 otherwise. The gas sensor sends analog data and an integrated led is lit up when configured threshold from potentiometer is reached. This data + additional info is sent to Thingsboard.

On my ESP32 i have configured additional thresholds to trigger alarms when a certain limit is reached. By this i mean when 1 is received for n times for a period of 5 seconds on either the PIR or sound sensor an additional telemetry data named “alarm” is set to 1. For the gas sensor, only if the threshold set by me is reached then the alarm is triggered. For example: 

  PIR threshold set by me: 5 (configurable from Thingsboard)
  PIR count of 1s: 0 (when the board starts and count is reset)
  Timer until count reset: 5 seconds (resets count to 1 if threshold is not reached in time)
  PIR alarm: 0

Now if we receive a stream of 1s in under 5 seconds then count will be 5 and PIR alarm is set to 1. If the readings of 1 take more than 5 seconds to reach the set threshold then the count will be reset and the alarm won't be triggered. Once the alarm is triggered, it won't be set to 1 again until we turn it off first. This is to avoid SMS spam.

The above explanation is valid for PIR and sound sensors. The gas sensor alarm functions almost the same. It only lacks the count and timer variables, because the values are analog and once the threshold value is reached it means it was progressively reached (not spiked, at least usually).

All this data is sent to Thingsboard, the readings and the alarms. The thresholds are configurable, the alarms can be turned off and the telemetry readings can be turned off from Thingsboard's dashboards. If we only want to detect motion we can turn off the readings for sound and gas, and vice versa. Once an alarm is triggered it will enter one of the three filter nodes from Thingsboard rule chain. A message will be set and a SMS will be sent through Twilio. In Twilio i only configured my phone and set a message body from Thingsboard. Once an alarm is triggered it needs to be turned off in order to be triggered again and send a sms.

And this pretty much should've summed up the project, but it seemed to me that it wasn't quite enough so i also added an esp32 cam to send a photo instead of a “Motion detected!” message. So i decided to connect to my main ESP32 an ESP32 cam module with a programmer. The programmer is needed to upload code to the esp32 cam module. The module is configured as a server and when a GET request to home (default path /) is received then it will begin to capture an image and save the encoded image in it's main memory. Then the photo will eventually be saved to Google Cloud. By using Google Cloud scripts the image is saved in my personal drive in a designated folder. After the image is uploaded an URL is returned. The cam module extracts the URL and sends it to the client that made the GET request. The client in our case will be the main ESP32 which will connect through Wi-Fi to the cam module and it will receive the url to the image. When motion alarm is triggered it will first take a photo and then it will upload the image URL along with the motion alarm status to Thingsboard. And now when the alarm is triggered an URL to the photo will be send through SMS to the configured phone number.

Hardware

- ESP32;

- PIR sensor, Sound sensor, Gas sensor;

- ESP32-CAM with programmer and camera;

- Breadboard;

- Wires;

- USB - micro USB cables;

Software

Arduino Code

I have 2 ino files, 1 for the base ESP32 which controls the sensors and sends the data, and the other is for the esp32 cam module to take the picture and save it to my google drive. The base ESP32 code is the Thingsboard example modified to fit my needs and the esp32 cam module contains different examples combined (http server example, take photo example and google drive data upload example).

Base ESP32 code: 

#if defined(ESP8266)
#include <ESP8266WiFi.h>
#define THINGSBOARD_ENABLE_PROGMEM 0
#elif defined(ESP32) || defined(RASPBERRYPI_PICO) || defined(RASPBERRYPI_PICO_W)
#include <WiFi.h>
#endif

#define PIR_PIN 27
#define SOUND_PIN 36
#define GAS_PIN 39

IPAddress camIP(192,168,100,46); //192.168.245.101

#include <Arduino_MQTT_Client.h>
#include <Server_Side_RPC.h>
#include <Attribute_Request.h>
#include <Shared_Attribute_Update.h>
#include <ThingsBoard.h>

constexpr char WIFI_SSID[] = "SSID";
constexpr char WIFI_PASSWORD[] = "PASS";

// See https://thingsboard.io/docs/getting-started-guides/helloworld/
// to understand how to obtain an access token
constexpr char TOKEN[] = "TOKEN";

// Thingsboard we want to establish a connection too
constexpr char THINGSBOARD_SERVER[] = "eu.thingsboard.cloud";
// MQTT port used to communicate with the server, 1883 is the default unencrypted MQTT port.
constexpr uint16_t THINGSBOARD_PORT = 1883U;

// Maximum size packets will ever be sent or received by the underlying MQTT client,
// if the size is to small messages might not be sent or received messages will be discarded
constexpr uint32_t MAX_MESSAGE_SIZE = 1024U;

// Baud rate for the debugging serial connection.
// If the Serial output is mangled, ensure to change the monitor speed accordingly to this variable
constexpr uint32_t SERIAL_DEBUG_BAUD = 9600U;

// Maximum amount of attributs we can request or subscribe, has to be set both in the ThingsBoard template list and Attribute_Request_Callback template list
// and should be the same as the amount of variables in the passed array. If it is less not all variables will be requested or subscribed
constexpr size_t MAX_ATTRIBUTES = 9U;

constexpr uint64_t REQUEST_TIMEOUT_MICROSECONDS = 5000U * 1000U;

// Attribute names for attribute request and attribute updates functionality
constexpr const char MONITOR_MODE_ATTR[] = "monitorMode";
constexpr const char SOUND_MODE_ATTR[] = "soundMode";
constexpr const char GAS_MODE_ATTR[] = "gasMode";
constexpr const char PIR_THRESHOLD_ATTR[] = "pirThreshold";
constexpr const char SOUND_THRESHOLD_ATTR[] = "soundThreshold";
constexpr const char GAS_THRESHOLD_ATTR[] = "gasThreshold";
constexpr const char PIR_ALARM_OFF_ATTR[] = "pirAlarm";
constexpr const char SOUND_ALARM_OFF_ATTR[] = "soundAlarm";
constexpr const char GAS_ALARM_OFF_ATTR[] = "gasAlarm";

// Initialize underlying client, used to establish a connection
WiFiClient wifiClient;

// Initalize the Mqtt client instance
Arduino_MQTT_Client mqttClient(wifiClient);

// Initialize used apis
Server_Side_RPC<9U, 9U> rpc;
Attribute_Request<9U, MAX_ATTRIBUTES> attr_request;
Shared_Attribute_Update<9U, MAX_ATTRIBUTES> shared_update;

const std::array<IAPI_Implementation*, 3U> apis = {
    &rpc,
    &attr_request,
    &shared_update
};

// Initialize ThingsBoard instance with the maximum needed buffer size, stack size and the apis we want to use
ThingsBoardSized<32> tb(mqttClient, MAX_MESSAGE_SIZE, Default_Max_Stack_Size, apis);

int pirState = 0;

int soundState = 0;

int gasValue = 0;

volatile int monitorMode = 0;

volatile int gasMode = 0;

volatile int soundMode = 0;

// handle led state and mode changes
volatile bool attributesChanged = false;

// For telemetry
constexpr int16_t telemetrySendInterval = 500U;
uint32_t previousDataSend;

// Sensor thresholds
volatile int pirThreshold = 5;
volatile int soundThreshold = 5;
volatile long gasThreshold = 2000;
volatile int pirDetectionCount = 0;
volatile int soundDetectionCount = 0;
constexpr int16_t countExpireTime = 5000U;
uint32_t previousExpireTime;

int pirAlarm = 0;
int soundAlarm = 0;
int gasAlarm = 0;

String attributesChangedAlarmTrigger = "";

// List of client attributes for requesting them (Using to initialize device states)
constexpr std::array<const char *, 9U> CLIENT_ATTRIBUTES_LIST = {
  MONITOR_MODE_ATTR,
  GAS_MODE_ATTR,
  SOUND_MODE_ATTR,
  PIR_THRESHOLD_ATTR,
  SOUND_THRESHOLD_ATTR,
  GAS_THRESHOLD_ATTR,
  PIR_ALARM_OFF_ATTR,
  SOUND_ALARM_OFF_ATTR,
  GAS_ALARM_OFF_ATTR
};

/// @brief Initalizes WiFi connection,
// will endlessly delay until a connection has been successfully established
void InitWiFi() {
  Serial.println("Connecting to AP ...");
  // Attempting to establish a connection to the given WiFi network
  WiFi.begin(WIFI_SSID, WIFI_PASSWORD);
  while (WiFi.status() != WL_CONNECTED) {
    // Delay 500ms until a connection has been succesfully established
    delay(500);
    Serial.print(".");
  }
  Serial.println("Connected to AP");
}

/// @brief Reconnects the WiFi uses InitWiFi if the connection has been removed
/// @return Returns true as soon as a connection has been established again
const bool reconnect() {
  // Check to ensure we aren't connected yet
  const wl_status_t status = WiFi.status();
  if (status == WL_CONNECTED) {
    return true;
  }

  // If we aren't establish a new connection to the given WiFi network
  InitWiFi();
  return true;
}

void processGasAlarmOff(const JsonVariantConst &data, JsonDocument &response) {
  Serial.println("Received the set gas alarm off RPC method");

  // Process data
  int new_mode = data;

  Serial.print("Mode to change: ");
  Serial.println(new_mode);
  StaticJsonDocument<1> response_doc;

  if (new_mode != 0 && new_mode != 1) {
    response_doc["error"] = "Unknown mode!";
    response.set(response_doc);
    return;
  }

  gasAlarm = new_mode;

  attributesChanged = true;
  attributesChangedAlarmTrigger = "";

  // Returning current mode
  response_doc["newMode"] = (int)gasAlarm;
  response.set(response_doc);
}

void processPIRAlarmOff(const JsonVariantConst &data, JsonDocument &response) {
  Serial.println("Received the set pir alarm off RPC method");

  // Process data
  int new_mode = data;

  Serial.print("Mode to change: ");
  Serial.println(new_mode);
  StaticJsonDocument<1> response_doc;

  if (new_mode != 0 && new_mode != 1) {
    response_doc["error"] = "Unknown mode!";
    response.set(response_doc);
    return;
  }

  pirAlarm = new_mode;

  attributesChanged = true;
  attributesChangedAlarmTrigger = "";

  // Returning current mode
  response_doc["newMode"] = (int)pirAlarm;
  response.set(response_doc);
}

void processSoundAlarmOff(const JsonVariantConst &data, JsonDocument &response) {
  Serial.println("Received the set sound alarm off RPC method");

  // Process data
  int new_mode = data;

  Serial.print("Mode to change: ");
  Serial.println(new_mode);
  StaticJsonDocument<1> response_doc;

  if (new_mode != 0 && new_mode != 1) {
    response_doc["error"] = "Unknown mode!";
    response.set(response_doc);
    return;
  }

  soundAlarm = new_mode;

  attributesChanged = true;
  attributesChangedAlarmTrigger = "";

  // Returning current mode
  response_doc["newMode"] = (int)soundAlarm;
  response.set(response_doc);
}

void processSetMonitorMode(const JsonVariantConst &data, JsonDocument &response) {
  Serial.println("Received the set monitor mode RPC method");

  // Process data
  int new_mode = data;

  Serial.print("Mode to change: ");
  Serial.println(new_mode);
  StaticJsonDocument<1> response_doc;

  if (new_mode != 0 && new_mode != 1) {
    response_doc["error"] = "Unknown mode!";
    response.set(response_doc);
    return;
  }

  monitorMode = new_mode;

  attributesChanged = true;
  attributesChangedAlarmTrigger = "";

  // Returning current mode
  response_doc["newMode"] = (int)monitorMode;
  response.set(response_doc);
}

void processSetGasMode(const JsonVariantConst &data, JsonDocument &response) {
  Serial.println("Received the set gas mode RPC method");

  // Process data
  int new_mode = data;

  Serial.print("Mode to change: ");
  Serial.println(new_mode);
  StaticJsonDocument<1> response_doc;

  if (new_mode != 0 && new_mode != 1) {
    response_doc["error"] = "Unknown mode!";
    response.set(response_doc);
    return;
  }

  gasMode = new_mode;

  attributesChanged = true;
  attributesChangedAlarmTrigger = "";

  // Returning current mode
  response_doc["newMode"] = (int)gasMode;
  response.set(response_doc);
}

void processSetSoundMode(const JsonVariantConst &data, JsonDocument &response) {
  Serial.println("Received the set sound mode RPC method");

  // Process data
  int new_mode = data;

  Serial.print("Mode to change: ");
  Serial.println(new_mode);
  StaticJsonDocument<1> response_doc;

  if (new_mode != 0 && new_mode != 1) {
    response_doc["error"] = "Unknown mode!";
    response.set(response_doc);
    return;
  }

  soundMode = new_mode;

  attributesChanged = true;
  attributesChangedAlarmTrigger = "";

  // Returning current mode
  response_doc["newMode"] = (int)soundMode;
  response.set(response_doc);
}

void processSetPirThreshold(const JsonVariantConst &data, JsonDocument &response) {
  Serial.println("Received the set pir threshold RPC method");

  int new_threshold = data;

  Serial.print("Threshold to change: ");
  Serial.println(new_threshold);
  StaticJsonDocument<1> response_doc;

  if (new_threshold < 1) {
    response_doc["error"] = "Invalid threshold!";
    response.set(response_doc);
    return;
  }

  pirThreshold = new_threshold;

  attributesChanged = true;
  attributesChangedAlarmTrigger = "";

  response_doc["newThreshold"] = (int)pirThreshold;
  response.set(response_doc);
}

void processSetSoundThreshold(const JsonVariantConst &data, JsonDocument &response) {
  Serial.println("Received the set sound threshold RPC method");

  int new_threshold = data;

  Serial.print("Threshold to change: ");
  Serial.println(new_threshold);
  StaticJsonDocument<1> response_doc;

  if (new_threshold < 1) {
    response_doc["error"] = "Invalid threshold!";
    response.set(response_doc);
    return;
  }

  soundThreshold = new_threshold;

  attributesChanged = true;
  attributesChangedAlarmTrigger = "";

  response_doc["newThreshold"] = (int)soundThreshold;
  response.set(response_doc);
}

void processSetGasThreshold(const JsonVariantConst &data, JsonDocument &response) {
  Serial.println("Received the set gas threshold RPC method");

  int new_threshold = data;

  Serial.print("Threshold to change: ");
  Serial.println(new_threshold);
  StaticJsonDocument<1> response_doc;

  if (new_threshold < 1) {
    response_doc["error"] = "Invalid threshold!";
    response.set(response_doc);
    return;
  }

  gasThreshold = new_threshold;

  attributesChanged = true;
  attributesChangedAlarmTrigger = "";

  response_doc["newThreshold"] = (int)gasThreshold;
  response.set(response_doc);
}

// Optional, keep subscribed shared attributes empty instead,
// and the callback will be called for every shared attribute changed on the device,
// instead of only the one that were entered instead
const std::array<RPC_Callback, 9U> callbacks = {
  RPC_Callback{"setMonitorMode", processSetMonitorMode},
  RPC_Callback{"setGasMode", processSetGasMode},
  RPC_Callback{"setSoundMode", processSetSoundMode},
  RPC_Callback{"setPirThreshold", processSetPirThreshold},
  RPC_Callback{"setSoundThreshold", processSetSoundThreshold},
  RPC_Callback{"setGasThreshold", processSetGasThreshold},
  RPC_Callback{"setPirAlarmOff", processPIRAlarmOff},
  RPC_Callback{"setSoundAlarmOff", processSoundAlarmOff},
  RPC_Callback{"setGasAlarmOff", processGasAlarmOff}
};

void processClientAttributes(const JsonObjectConst &data) {
  for (auto it = data.begin(); it != data.end(); ++it) {
    if (strcmp(it->key().c_str(), MONITOR_MODE_ATTR) == 0) {
      const uint16_t new_mode = it->value().as<uint16_t>();
      monitorMode = new_mode;
    }

    if (strcmp(it->key().c_str(), GAS_MODE_ATTR) == 0) {
      const uint16_t new_mode = it->value().as<uint16_t>();
      gasMode = new_mode;
    }

    if (strcmp(it->key().c_str(), SOUND_MODE_ATTR) == 0) {
      const uint16_t new_mode = it->value().as<uint16_t>();
      soundMode = new_mode;
    }

    if (strcmp(it->key().c_str(), PIR_THRESHOLD_ATTR) == 0) {
      const uint16_t new_value = it->value().as<uint16_t>();
      pirThreshold = new_value;
    }

    if (strcmp(it->key().c_str(), SOUND_THRESHOLD_ATTR) == 0) {
      const uint16_t new_value = it->value().as<uint16_t>();
      soundThreshold = new_value;
    }

    if (strcmp(it->key().c_str(), SOUND_ALARM_OFF_ATTR) == 0) {
      const uint16_t new_value = it->value().as<uint16_t>();
      soundAlarm = new_value;
    }

    if (strcmp(it->key().c_str(), PIR_ALARM_OFF_ATTR) == 0) {
      const uint16_t new_value = it->value().as<uint16_t>();
      pirAlarm = new_value;
    }

    if (strcmp(it->key().c_str(), GAS_ALARM_OFF_ATTR) == 0) {
      const uint16_t new_value = it->value().as<uint16_t>();
      gasAlarm = new_value;
    }

    if (strcmp(it->key().c_str(), GAS_THRESHOLD_ATTR) == 0) {
      const uint16_t new_value = it->value().as<uint16_t>();
      gasThreshold = new_value;
    }
  }
}

// Attribute request did not receive a response in the expected amount of microseconds 
void requestTimedOut() {
  Serial.printf("Attribute request timed out did not receive a response in (%llu) microseconds. Ensure client is connected to the MQTT broker and that the keys actually exist on the target device\n", REQUEST_TIMEOUT_MICROSECONDS);
}

const Attribute_Request_Callback<MAX_ATTRIBUTES> attribute_client_request_callback(&processClientAttributes, REQUEST_TIMEOUT_MICROSECONDS, &requestTimedOut, CLIENT_ATTRIBUTES_LIST);

void setup() {
  // Initialize serial connection for debugging
  Serial.begin(SERIAL_DEBUG_BAUD);

  pinMode(PIR_PIN, INPUT);
  pinMode(SOUND_PIN, INPUT);
  pinMode(GAS_PIN, INPUT);

  delay(1000);

  InitWiFi();

  delay(1000);
}

void loop() {
  delay(10);

  if (!reconnect()) {
    return;
  }

  if (!tb.connected()) {
    // Connect to the ThingsBoard
    Serial.print("Connecting to: ");
    Serial.print(THINGSBOARD_SERVER);
    Serial.print(" with token ");
    Serial.println(TOKEN);
    if (!tb.connect(THINGSBOARD_SERVER, TOKEN, THINGSBOARD_PORT)) {
      Serial.println("Failed to connect");
      return;
    }
    // Sending a MAC address as an attribute
    tb.sendAttributeData("macAddress", WiFi.macAddress().c_str());

    Serial.println("Subscribing for RPC...");
    // Perform a subscription. All consequent data processing will happen in
    // processSetLedState() and processSetLedMode() functions,
    // as denoted by callbacks array.
    if (!rpc.RPC_Subscribe(callbacks.cbegin(), callbacks.cend())) {
      Serial.println("Failed to subscribe for RPC");
      return;
    }

    Serial.println("Subscribe done");

    // Request current states of client attributes
    if (!attr_request.Client_Attributes_Request(attribute_client_request_callback)) {
      Serial.println("Failed to request for client attributes");
      return;
    }

    attributesChanged = true;
    attributesChangedAlarmTrigger = "0";
  }

  if (attributesChanged) {
    attributesChanged = false;
    tb.sendTelemetryData(MONITOR_MODE_ATTR, monitorMode);
    tb.sendAttributeData(MONITOR_MODE_ATTR, monitorMode);

    tb.sendTelemetryData(GAS_MODE_ATTR, gasMode);
    tb.sendAttributeData(GAS_MODE_ATTR, gasMode);

    tb.sendTelemetryData(SOUND_MODE_ATTR, soundMode);
    tb.sendAttributeData(SOUND_MODE_ATTR, soundMode);

    tb.sendTelemetryData(PIR_THRESHOLD_ATTR, pirThreshold);
    tb.sendAttributeData(PIR_THRESHOLD_ATTR, pirThreshold);

    tb.sendTelemetryData(SOUND_THRESHOLD_ATTR, soundThreshold);
    tb.sendAttributeData(SOUND_THRESHOLD_ATTR, soundThreshold);

    tb.sendTelemetryData(GAS_THRESHOLD_ATTR, gasThreshold);
    tb.sendAttributeData(GAS_THRESHOLD_ATTR, gasThreshold);

    if (attributesChangedAlarmTrigger == "pirAlarm" || attributesChangedAlarmTrigger == "0") {
      tb.sendTelemetryData(PIR_ALARM_OFF_ATTR, pirAlarm);
      tb.sendAttributeData(PIR_ALARM_OFF_ATTR, pirAlarm);
    }

    if (attributesChangedAlarmTrigger == "soundAlarm" || attributesChangedAlarmTrigger == "0") {
      tb.sendTelemetryData(SOUND_ALARM_OFF_ATTR, soundAlarm);
      tb.sendAttributeData(SOUND_ALARM_OFF_ATTR, soundAlarm);
    }

    if (attributesChangedAlarmTrigger == "gasAlarm" || attributesChangedAlarmTrigger == "0") {
      tb.sendTelemetryData(GAS_ALARM_OFF_ATTR, gasAlarm);
      tb.sendAttributeData(GAS_ALARM_OFF_ATTR, gasAlarm);
    }
  }

  if (millis() - previousExpireTime > countExpireTime) {
    previousExpireTime = millis();

    pirDetectionCount = 0;
    soundDetectionCount = 0;
  }

  // Sending telemetry every telemetrySendInterval time
  // Trigget once if threshold is hit and keep alarm on until it's disabled
  if (millis() - previousDataSend > telemetrySendInterval) {
    previousDataSend = millis();

    if (gasMode == 1) {
      gasValue = analogRead(GAS_PIN);
      tb.sendTelemetryData("gas", gasValue);

      if (gasValue >= gasThreshold && gasAlarm != 1) {
        gasAlarm = 1;
        attributesChanged = true;
        attributesChangedAlarmTrigger = "gasMode";
        Serial.println("Gas level excedeed!");
      }
    }

    if (monitorMode == 1) {
      pirState = digitalRead(PIR_PIN);

      tb.sendTelemetryData("PIR_state", pirState);

      if (pirState == HIGH) {                 
        Serial.println("Somebody is here!");

        pirDetectionCount++;
        if (pirDetectionCount >= pirThreshold && pirAlarm != 1) {
          String imageLink = triggerCameraSnap();
          tb.sendTelemetryData("imageLink", imageLink.c_str());
          pirDetectionCount = 0;
          pirAlarm = 1;
          attributesChanged = true;
          attributesChangedAlarmTrigger = "pirAlarm";
        }
      }
    }

    if (soundMode == 1) {
      soundState = digitalRead(SOUND_PIN);
      if (soundState == HIGH) {
        Serial.println("Sound detected!");

        tb.sendTelemetryData("sound_state", soundState);

        soundDetectionCount++;
        if (soundDetectionCount >= soundThreshold && soundAlarm != 1) {
          soundDetectionCount = 0;
          soundAlarm = 1;
          attributesChanged = true;
          attributesChangedAlarmTrigger = "soundAlarm";
        }
      }
    }

    tb.sendAttributeData("rssi", WiFi.RSSI());
    tb.sendAttributeData("channel", WiFi.channel());
    tb.sendAttributeData("bssid", WiFi.BSSIDstr().c_str());
    tb.sendAttributeData("localIp", WiFi.localIP().toString().c_str());
    tb.sendAttributeData("ssid", WiFi.SSID().c_str());
  }

  tb.loop();
}

String triggerCameraSnap() {
  WiFiClient client;
  Serial.print("Connecting to CAM at ");
  Serial.print(camIP);
  Serial.print(":80 … ");
  if (!client.connect(camIP, 80)) {
    Serial.println("failed");
    return "";
  }
  Serial.println("ok");

  client.print(
    "GET / HTTP/1.1\r\n"
    "Host: " + camIP.toString() + "\r\n"
    "Connection: close\r\n\r\n"
  );

  String status = client.readStringUntil('\n');
  Serial.print("Status: ");
  Serial.println(status);

  while (client.connected()) {
    String line = client.readStringUntil('\n');
    if (line == "\r") break;
  }

  String body = client.readString();
  body.trim();

  int linkStart = body.lastIndexOf('\n') + 1;
  String link = body.substring(linkStart);

  link.trim();

  Serial.println("Extracted link:");
  Serial.println(link);

  client.stop();
  return link;
}

ESP32 Cam Module Code: 

#include <WiFi.h>
#include <WiFiClientSecure.h>
#include "soc/soc.h"
#include "soc/rtc_cntl_reg.h"
#include "Base64.h"

#include "esp_camera.h"

const char* ssid     = "SSID";
const char* password = "PASS";
const char* myDomain = "script.google.com";
String myScript = "/macros/s/SCRIPT_ID/exec";
String myFilename = "filename=ESP32-CAM.jpg";
String mimeType = "&mimetype=image/jpeg";
String myImage = "&data=";

WiFiServer server(80);

int waitingTime = 30000; //Wait 30 seconds for google response.

#define PWDN_GPIO_NUM     32
#define RESET_GPIO_NUM    -1
#define XCLK_GPIO_NUM      0
#define SIOD_GPIO_NUM     26
#define SIOC_GPIO_NUM     27

#define Y9_GPIO_NUM       35
#define Y8_GPIO_NUM       34
#define Y7_GPIO_NUM       39
#define Y6_GPIO_NUM       36
#define Y5_GPIO_NUM       21
#define Y4_GPIO_NUM       19
#define Y3_GPIO_NUM       18
#define Y2_GPIO_NUM        5
#define VSYNC_GPIO_NUM    25
#define HREF_GPIO_NUM     23
#define PCLK_GPIO_NUM     22

void setup()
{
  WRITE_PERI_REG(RTC_CNTL_BROWN_OUT_REG, 0);
  
  Serial.begin(115200);
  delay(10);
  
  WiFi.mode(WIFI_STA);

  Serial.println("");
  Serial.print("Connecting to ");
  Serial.println(ssid);
  WiFi.begin(ssid, password);  

  while (WiFi.status() != WL_CONNECTED) {
    Serial.print(".");
    delay(500);
  }

  Serial.println("");
  Serial.println("STAIP address: ");
  Serial.println(WiFi.localIP());
    
  Serial.println("");

  camera_config_t config;
  config.ledc_channel = LEDC_CHANNEL_0;
  config.ledc_timer = LEDC_TIMER_0;
  config.pin_d0 = Y2_GPIO_NUM;
  config.pin_d1 = Y3_GPIO_NUM;
  config.pin_d2 = Y4_GPIO_NUM;
  config.pin_d3 = Y5_GPIO_NUM;
  config.pin_d4 = Y6_GPIO_NUM;
  config.pin_d5 = Y7_GPIO_NUM;
  config.pin_d6 = Y8_GPIO_NUM;
  config.pin_d7 = Y9_GPIO_NUM;
  config.pin_xclk = XCLK_GPIO_NUM;
  config.pin_pclk = PCLK_GPIO_NUM;
  config.pin_vsync = VSYNC_GPIO_NUM;
  config.pin_href = HREF_GPIO_NUM;
  config.pin_sscb_sda = SIOD_GPIO_NUM;
  config.pin_sscb_scl = SIOC_GPIO_NUM;
  config.pin_pwdn = PWDN_GPIO_NUM;
  config.pin_reset = RESET_GPIO_NUM;
  config.xclk_freq_hz = 20000000;
  config.pixel_format = PIXFORMAT_JPEG;
  config.frame_size = FRAMESIZE_VGA;  // UXGA|SXGA|XGA|SVGA|VGA|CIF|QVGA|HQVGA|QQVGA
  config.jpeg_quality = 10;
  config.fb_count = 1;
  
  esp_err_t err = esp_camera_init(&config);
  if (err != ESP_OK) {
    Serial.printf("Camera init failed with error 0x%x", err);
    delay(1000);
    ESP.restart();
  }

  Serial.println("");
  Serial.println("WiFi connected.");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
  server.begin();
}


void loop() {
  WiFiClient client = server.available();
  if (!client) {
    delay(5);
    return;
  }

  while (!client.available()) {
    delay(1);
  }

  String requestLine = client.readStringUntil('\r');
  client.readStringUntil('\n');
  Serial.println("Request: " + requestLine);

  if (requestLine.startsWith("GET / ")) {
    String link = saveCapturedImage();
    if (link.length()) {
      client.print("HTTP/1.1 200 OK\r\n"
                   "Content-Type: text/plain\r\n"
                   "Connection: close\r\n\r\n");
      client.print(link);
      Serial.println("Returned link: " + link);
    } else {
      client.print("HTTP/1.1 500 Internal Server Error\r\n"
                   "Content-Type: text/plain\r\n"
                   "Connection: close\r\n\r\n");
      client.print("ERROR");
      Serial.println("Upload failed");
    }
  } else {
    client.print("HTTP/1.1 404 Not Found\r\n"
                 "Connection: close\r\n\r\n");
  }

  delay(1);
  client.stop();
}

String saveCapturedImage() {
  WiFiClientSecure client;
  client.setInsecure();

  Serial.println("Connecting to " + String(myDomain));
  
  if (client.connect(myDomain, 443)) {
    Serial.println("Connection successful");
    
    camera_fb_t * fb = esp_camera_fb_get();
    if (!fb) {
      Serial.println("Camera capture failed");
      delay(1000);
      ESP.restart();
      return "";
    }
  
    char *input = (char *)fb->buf;
    char output[base64_enc_len(3)];
    String imageFile = "";
    for (int i=0; i<fb->len; i++) {
      base64_encode(output, (input++), 3);
      if (i % 3 == 0) imageFile += urlencode(String(output));
    }

    esp_camera_fb_return(fb);

    Serial.println("Sending a captured image to Google Drive.");
    String Data = myFilename + mimeType + myImage;
    
    client.println("POST " + myScript + " HTTP/1.1");
    client.println("Host: " + String(myDomain));
    client.println("Content-Length: " + String(Data.length() + imageFile.length()));
    client.println("Content-Type: application/x-www-form-urlencoded");
    client.println();
    client.print(Data);
    for (int i = 0; i < imageFile.length(); i += 1000) {
      client.print(imageFile.substring(i, i + 1000));
    }
    
    Serial.println("Waiting for server response...");
    long int startTime = millis();
    String response = "";
    while (!client.available()) {
      if ((millis() - startTime) > waitingTime) {
        Serial.println("Timeout waiting for response");
        return "";
      }
      delay(100);
    }

    String redirectURL = "";
    while (client.available()) {
      String line = client.readStringUntil('\n');
      if (line.startsWith("Location: ")) {
        redirectURL = line.substring(10);
        redirectURL.trim();
        break;
      }
      if (line == "\r") break;
    }
    client.stop();

    if (redirectURL == "") {
      Serial.println("No redirect found");
      return "";
    }

    Serial.println("Redirect URL: " + redirectURL);
    WiFiClientSecure redirectClient;
    redirectClient.setInsecure();

    Serial.println("Connecting to redirect URL...");
    if (!redirectClient.connect("script.googleusercontent.com", 443)) {
      Serial.println("Failed to connect to redirect");
      return "";
    }

    int pathStart = redirectURL.indexOf(".com");
    if (pathStart == -1) {
      Serial.println("Invalid redirect URL");
      return "";
    }
    String path = redirectURL.substring(pathStart + 4);

    Serial.println("Sending GET request:");
    Serial.println("GET " + path + " HTTP/1.1");

    redirectClient.println("GET " + path + " HTTP/1.1");
    redirectClient.println("Host: script.googleusercontent.com");
    redirectClient.println("Connection: close");
    redirectClient.println();

    // Parse response
    Serial.println("Reading response...");
    String finalLink = "";
    bool headersEnded = false;
    while (redirectClient.connected()) {
      while (redirectClient.available()) {
        String line = redirectClient.readStringUntil('\n');
        line.trim();
        Serial.println("> " + line);

        if (!headersEnded && (line == "" || line == "\r")) {
          headersEnded = true;
        } else if (headersEnded) {
          if (line.startsWith("<pre>") && line.endsWith("</pre>")) {
            finalLink = line.substring(5, line.length() - 6);
            break;
          } else if (line.startsWith("https://drive.google.com")) {
            finalLink = line;
            break;
          }
        }
      }
      if (finalLink != "") break;
    }
    redirectClient.stop();

    if (finalLink != "") {
      Serial.println("Final Drive link:");
      Serial.println(finalLink);
      return finalLink;
    } else {
      Serial.println("Failed to extract Drive link from redirect");
    }
    return "";
  } else {
    Serial.println("Connection to " + String(myDomain) + " failed.");
    client.stop();
    return "";
  }
}

String urlencode(String str) {
  String encodedString = "";
  char c, code0, code1;
  for (int i = 0; i < str.length(); i++) {
    c = str.charAt(i);
    if (c == ' ') encodedString += '+';
    else if (isalnum(c)) encodedString += c;
    else {
      code1 = (c & 0xf) + '0';
      if ((c & 0xf) > 9) code1 = (c & 0xf) - 10 + 'A';
      c = (c >> 4) & 0xf;
      code0 = c + '0';
      if (c > 9) code0 = c - 10 + 'A';
      encodedString += '%';
      encodedString += code0;
      encodedString += code1;
    }
    yield();
  }
  return encodedString;
}

Besides these 2 ino files i have 3 javascript rule nodes in Thingsboard's rule chain which just filter for the correct telemetry (motion, sound, gas) and convert the message from “gasAlarm = 1” to a string like ”[Telemetry_Name] detected! [photo_url]”. Then this message is sent to Twilio and is routed to the configured mobile phone number.

Google Drive

As i said previously i am also using a google script to actually store the photo taken by the esp32 cam module to my google drive. The script is needed to handle the image upload and return the actual url to the image. The script is at https://script.google.com. 

function doPost(e) {
  var data = Utilities.base64Decode(e.parameters.data);
  var nombreArchivo = Utilities.formatDate(new Date(), "GMT-3", "yyyyMMdd_HHmmss") + ".jpg";
  var blob = Utilities.newBlob(data, e.parameters.mimetype, nombreArchivo);

  var folder = DriveApp.getFoldersByName("ESP32-CAM").hasNext()
    ? DriveApp.getFoldersByName("ESP32-CAM").next()
    : DriveApp.createFolder("ESP32-CAM");

  var file = folder.createFile(blob);
  file.setSharing(DriveApp.Access.ANYONE_WITH_LINK, DriveApp.Permission.VIEW);

  var publicLink = "https://drive.google.com/uc?id=" + file.getId();

  PropertiesService.getScriptProperties().setProperty("latestImageLink", publicLink);

  return ContentService.createTextOutput(publicLink);
}


function doGet(e) {
  var link = PropertiesService.getScriptProperties().getProperty("latestImageLink");
  if (link) {
    return ContentService
      .createTextOutput(link)
      .setMimeType(ContentService.MimeType.TEXT);
  } else {
    return ContentService
      .createTextOutput("No image link available.")
      .setMimeType(ContentService.MimeType.TEXT);
  }
}

Thingsboard

In Thingsboard there is not much coding going on, i just configured some buttons which connect to the RPC methods from the ESP32 code. I also wrote some filter and transform functions in JS for the Thingsboard rule chain. It was needed in order to choose a path based on the alarm triggered and configure a message. Example: 

Filter:
return msg.soundAlarm == 1;

Transform:
msg.body = "Sound was detected";
delete msg.soundAlarm;
return {msg: msg, ...};

And also there is the telemetry data and the dashboard shown in the context section.

Challenges

The most challenging thing was to actually take the photo, because it needed a bit more setup with the google cloud and the actual taking of the photo. And also i really struggled with the google script because it was returning a redirect response and i had to figure out how to follow the redirect link in order to get the image URL. It took me a lot of time to realise i can just extract the url from the body of the redirect message and just make another request. Also i first wanted to send the image url through BLE to the main ESP32 and it took me a while to realise that BLE and Wi-Fi don't really work simultaneously, because there is interference and communication ordering and stuff. So i had to give that up and find another way, because i really didn't want to connect wires since i was skeptical about how to connect them. So i eventually came up with a web server and then it went smoothly.

Another big challenge was how to send SMS only once and not every time an “alarm” is triggered. And to achieve what i have now i reached the limit of free sms messages sent in 3 different days. I first tried to keep count of alarms in Thingsboard but it didn't went well and i just figured that i can manage the alarm stuff locally and just set them off from the cloud and that's exactly what i did. Now an alarm triggers once until it's turned off.

And the last challenge i encountered was setting the right values for potentiometers and the thresholds set by me.

There was a lot of trial and error during this project but thankfully nothing was fried :D.

References

- https://thingsboard.io/docs/devices-library/esp32-dev-kit-v1/

- https://docs.sunfounder.com/projects/umsk/en/latest/03_esp32/esp32_lesson04_mq2.html

- https://www.electronicwings.com/esp32/pir-sensor-interfacing-with-esp32

- https://circuitdigest.com/microcontroller-projects/interface-ky038-sound-sensor-with-esp32

- https://randomnerdtutorials.com/esp32-cam-take-photo-save-microsd-card/

- https://www.niraltek.com/blog/how-to-take-photos-and-upload-it-to-google-drive-using-esp32-cam/

- https://ocw.cs.pub.ro/courses/iothings/laboratoare/2022/lab3

- https://thingsboard.io/docs/user-guide/ui/rule-chains/

SRIC Administrative Resources

Notare
Catalog
Calendar

SRIC Lectures (2025)

SRIC Labs (2024-2025)

SRIC Labs (2023)

SRIC Projects (2023)

Descriere Proiecte

SRIC Archive

AAC Administrative Resources

Attendance

Grading

AAC Lectures (2023-2024)

AAC Labs (2023-2024)

Projects

Table of Contents

iothings/proiecte/2025sric/homemonitoringsystemwithalerts.txt · Last modified: 2025/05/24 14:37 by andrei.buhnici
Old revisions
Media ManagerBack to top
CC Attribution-Share Alike 3.0 Unported
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0