Author: Adriana-Iuliana Boncu

Master: AAC2

The objective of this paper is to create an intelligent doorbell system. This system can be divided into two subsystems, one that takes care of capturing images and classifying them according to known faces, and the second, upon receiving the command from the first system to activate a motor to open the door and also to have the possibility to ring the bell by pressing a button.

Door Opening System

• 28BYJ-48 5V Stepper Motor
• Driver ULN2003
• Buzzer
• Button
• 10K resistor
• ESP-WROOM-32

Video Capturing System:

• Raspberry Pi 4 Model B/2GB
• USB Camera

1. Door Opening System

For the automatic door opener I chose to use an ESP-WROOM-32. This is a module with which a wide variety of applications can be realized, from low-power sensors to the most demanding tasks, such as voice encoding, music streaming and P3 decoding. The basis of this module is the ESP32-D0EDQ6 chip. This chip is designed to be scalable and adaptive. It contains two CPUs that can be controlled individually, and the clock frequency can be adjusted in the range (80 MHz;240 MHz). The user can turn off the CPU and use the coprocessor to monitor the occurrence of changes, consuming fewer resources.

Due to the fact that it integrates Bluetooth, Bluetooth LE and Wi-Fi ensures the possibility of many applications. With the help of the WI-FI module, whose specifications are specified in Table 1, which can provide a direct connection to the Internet, the ESP32 will be able to communicate with the Raspberry Pi board to receive commands for the automatic opening and closing of the door.

The ESP32 also takes care of driving the stepper motor that allows the door to be opened and the operation of a doorbell at the push of a button. The wiring diagram with all components included can be found in fallowing Figure.

As can be seen in Figure 1, the stepper motor driver is connected to pins D13,D12,D14,D27 (IN1,IN2,IN3,IN4-stepper driver pins in order), the button is connected to the pin D34, and the buzzer is connected to pin D32.

2. Video Capturing System

For this system I used a web camera and the Raspberry Pi 4 platform, as can be seen in the figure below.

Since the two systems communicate through the two UDP & TCP protocols, I connected the platform to the Internet via an ethernet cable. Also, to have a preview of the camera, I connected a display through one of the two micro HDMI ports.

In order to have a secure connection, we followed the example discussed in this paper that combines two communication protocols TCP and UDP, creating 5 layers of security for sending the command. These layers are: Encryption, Compression, Key generation, Browsing data and Combination of UDP and TCP

1. Encryption & Decryption

For data encryption, I chose to use the Diffie-Hellman algorithm with which a secret key can be established that is unknown only to the one who sends the command and the one who receives it.

As can be seen in figure, this algorithm is carried out in several steps. To begin with, each device knows two shared keys. Each of these devices randomly generates an integer with which it performs the mathematical calculation presented in point 3, obtaining a result. This result is sent using the TCP communication protocol between the devices. With the help of the information available up to this moment, the operation from point 4 is carried out, which will result in the common key of those devices.

With the help of this key, the “open” command is encrypted with the help of the Caesar cipher. Caesar's cipher is also called displacement cipher because with the help of an integer and the alphabet each character is replaced by another. In the girl's case, with the help of the previously obtained key, all characters will be replaced by the 9th letter of the alphabet from the position of the letter in question.

2. Compression & Decompression

In this level, several calculations are made. First, the string is written in ascii, obtaining values from which the value 97 is subtracted, calculating two vectors that represent the mean and the difference. The values from these two vectors are written in binary and will be passed through the next security layer.

3. Key generation

On the side of the device that sends the command, the key is generated that will be sent using the UDP protocol. The key is generated from the mean and difference vectors calculated upon compression.

4. Browsing data

The next level refers to how the data is traversed. For example, if we have a string of numbers from 1 to 10, it can be read as in the example in the figure.

5. Combination of UDP and TCP

Implementation

1. Door Opening System

All the security levels together with the functionalities for the motor, button and buzzer were implemented and written in the following code that was uploaded to the ESP32 using the Arduino IDE. It should be mentioned that the reverse of the security actions are carried out at the level of the ESP because the information is encrypted at the level of the Raspberry Pi.

#include <WiFi.h>
#include<stdio.h>
#include<math.h>
#include <string>
#include <cstring>
#include <WiFiUdp.h>
 
// this will assign the name PushButton to pin numer 15
const int PushButton = 34;
// Pin connected to buzzer
int speakerPin = 32;
// caracteristicile tonului folosit la sonerie
const int length = 32; // numarul de note piesa
const char * note = "ggagsed deggsgg ggagsed deggsgg ";
const int beats[]= {2,2,1,1,1,1,4,2,2,2,2,2,2,4,2,2,2,2,1,1,1,1,4,2,2,2,2,2,2,4,2,2,3,1,1,1,1};
const int tempo =150;
 
// definire pini pentru motorul de deschidere
#define IN1 13
#define IN2 12
#define IN3 14
#define IN4 27
#define FULL_ROTATION 1270
// faze ale motorului 
const int phases1[] = {0, 0, 0, 0, 0, 1, 1, 1};
const int phases2[] = {0, 0, 0, 1, 1, 1, 0, 0};
const int phases3[] = {0, 1, 1, 1, 0, 0, 0, 0};
const int phases4[] = {1, 1, 0, 0, 0, 0, 0, 1};
 
int Phase = 0;
int Speed = 100; //MUST BE 1 – 100
 
const char* ssid = "330";
const char* password = "pandapanda";
char packetBuffer[255]; //buffer pentru a tine datele ce sunt primite
 
 
// portul pentru trimitere data UDP
const int udpPort = 1234;
 
//instantiere UDP
WiFiUDP Udp;
WiFiServer server(80);
 
time_t start_criptare, end_criptare, start_decompresie, end_decompresie, start_parcurgere, end_parcurgere, start_generare_cheie, end_generare_cheie, start_TCP, end_TCP, start_UDP, end_UDP;
 
void playTone(int tone, int duration)
{
 for (long i = 0; i < duration * 1000L; i += tone * 2)
 {
 digitalWrite(speakerPin, HIGH);
 delayMicroseconds(tone);
 digitalWrite(speakerPin, LOW);
 delayMicroseconds(tone);
 }
}
void playNote(char note, int duration)
{
 const char names[] = { 'c', 'd', 'e', 'f', 's', 'g', 'a', 'v', 'b', 'C', 'D', 'E' }; // note
 const int tones[] = { 1915, 1700, 1519, 1432, 1352, 1275, 1136, 1073, 1014, 956, 852, 758 };
// frecvente
 for (int i = 0; i < 12; i++)
 {
 if (names[i] == note) playTone(tones[i], duration);
 } 
}
 
void descriptare_ceaser(char message[], int key){
  char ch;
  for(int i = 0; message[i] != '\0'; ++i){
    ch = message[i];
    if(ch >= 'a' && ch <= 'z'){
    ch = ch - key;
    if(ch < 'a'){
    ch = ch + 'z' - 'a' + 1;
    }
    message[i] = ch;
    }
    else if(ch >= 'A' && ch <= 'Z'){
    ch = ch - key;
      if(ch < 'A'){
      ch = ch + 'Z' - 'A' + 1;
      }
    message[i] = ch;
    }
  } 
}
 
int binaryToDecimal(int n)
{
    int num = n;
    int dec_value = 0;
    int base = 1;
    int temp = num;
    while (temp) {
        int last_digit = temp % 10;
        temp = temp / 10;
        dec_value += last_digit * base;
        base = base * 2;
    }
    return dec_value;
}
 
int * rezolvare_sisteme(int a[],int dif[]){
 
    int determinant = 2;
    static int solutii[8];
    for(int k=0, i=0 ;k<8;k=k+2,i=i+1)
    {
      solutii[k] = (dif[i]+a[i])/determinant;
      solutii[k+1] = (a[i]-dif[i])/determinant;
    }
 
  return solutii;
}
int * restaurare_cheie_proba(int s[2][17]){
 
  static int key[34];
  int k=0;    
  for(k=0;k<34-1;k=k+2){
    key[k]=s[0][k/2];
    key[k+1]=s[1][k/2];
  }
 
  return key;
}
int * restaurare_cheie(int s[]){
 
  static int * key;
  static int sir1[2][17];
  int k=0,j=0;    
  for(k=0,j=0;k<17;k++,j++){
    sir1[0][k] = s[k];
    sir1[1][k] = s[k+17];
  }
  key = restaurare_cheie_proba(sir1);
  return key;
}
 
int decriptare_dif(int diff_b_com){
  int dec_diff_b_com;
  if(diff_b_com > 100000){
  dec_diff_b_com=binaryToDecimal(diff_b_com-100000);
  dec_diff_b_com=-dec_diff_b_com;
  } 
  else {
    dec_diff_b_com=binaryToDecimal(diff_b_com);
  }
  return dec_diff_b_com;
}
 
char * determinare_comanda (int key[], int med[]){
  int ka=0,kb=0;
  int diff_b_com0=0,diff_b_com1=0,diff_b_com2=0,diff_b_com3=0;
  int medie0=0,medie1=0,medie2=0,medie3=0;
      start_generare_cheie=micros();
  for(int g=0;g<24;g++){
 
        if(g < 6)
      medie0=medie0*10 + med[g];
        if(g >= 6 & g < 12)
      medie1=medie1*10 + med[g];
        if(g >= 12 && g < 18)
      medie2=medie2*10 + med[g];
        if(g >= 18 && g < 24)
      medie3=medie3*10 + med[g];
    }
 
  for(int k=0;k<34;k++){
    if(k < 6)
      diff_b_com0=diff_b_com0*10 + key[k];
      if((k>=6 && k<10) || (k >=16 && k < 17 ))
        ka=ka*10+key[k];
      if(k>=10 && k<16)
        diff_b_com1 =diff_b_com1*10+ key[k];
      if((k>=17 && k<20) || (k>=26 && k<28 ))
        kb=kb*10+key[k];
      if(k>=20 && k<26)
        diff_b_com2 = diff_b_com2*10+key[k];
    if(k>=28 && k<34)
        diff_b_com3 = diff_b_com3*10 + key[k];  
  } 
    end_generare_cheie=micros();
    start_decompresie=micros();
   //decriptare
  int ka_dec = binaryToDecimal(ka);
  int kb_dec = binaryToDecimal(kb);
 
 
  if (ka_dec != 1 || kb_dec != 7){
    Serial.print("Chei nepotrivite! Utilizator neidentificat!");
    return "Null";
    }
  int dec_diff_b_com0 = decriptare_dif(diff_b_com0);
  int dec_diff_b_com1 = decriptare_dif(diff_b_com1);
  int dec_diff_b_com2 = decriptare_dif(diff_b_com2);
  int dec_diff_b_com3 = decriptare_dif(diff_b_com3);
 
 
  int m0 = binaryToDecimal(medie0);
  int m1 = binaryToDecimal(medie1);
  int m2 = binaryToDecimal(medie2);
  int m3 = binaryToDecimal(medie3);
 
  int medie[4]={2*m0,2*m1,2*m2,2*m3};
  int diferente[4]={dec_diff_b_com0,dec_diff_b_com1,dec_diff_b_com2,dec_diff_b_com3};
 
  static char comanda_finala[9];
  int * comanda = rezolvare_sisteme(medie,diferente);
 
  for(int j=0; j<8; j++){
    char c=(char)(comanda[j]+97); //revenire la alfabet conform cod ascii
    comanda_finala[j] = c ;
  }   
  comanda_finala[8]='\0';
  end_decompresie=micros();
  return comanda_finala;
}
 
//functie pentru returnarea valori a ^ b mod P
int power(int a,int b,int P)
{
    if (b == 1)
        return a;
 
    else
        return (((int)pow(a, b)) % P);
}
 
void IncrementPhase(int rotationDirection){
Phase += 8;
Phase += rotationDirection;
Phase %= 8;
}
 
 
void stepper(int count){
int rotationDirection = count < 1 ? -1 : 1;
count *= rotationDirection;
 
  for (int x = 0; x < count; x++)
  {
 
  digitalWrite(IN1, phases1[Phase]);
  digitalWrite(IN2, phases2[Phase]);
  digitalWrite(IN3, phases3[Phase]);
  digitalWrite(IN4, phases4[Phase]);
 
  IncrementPhase(rotationDirection);
  delay(100/Speed);
  }  
}
    int C_cunoscut = 23;
    int S_cunoscut = 9;
 
    int S_secret = rand() % 10 + 1; //valoarea secreta din server
 
void setup()
{
    Serial.begin(115200);
    delay(1000);
 
    // We start by connecting to a WiFi network
 
    Serial.println();
    Serial.println();
    Serial.print("Se conecteaza la ");
    Serial.println(ssid);
 
    WiFi.begin(ssid, password);
 
    while (WiFi.status() != WL_CONNECTED) {
        delay(500);
        Serial.print(".");
    }
 
    Serial.println("");
    Serial.println("WiFi conectat.");
    Serial.println("Adresa IP : ");
    Serial.println(WiFi.localIP());
 
    server.begin();
 
    Serial.println("Valoarea aleasa de client cunoscuta este: ");
    Serial.println(C_cunoscut);
    Serial.println("Valoarea aleasa de server cunoscuta este: ");
    Serial.println(S_cunoscut);
 
    // Initializeaza udp-ul 
    Udp.begin(udpPort);
 
 
    pinMode(IN1, OUTPUT);
    pinMode(IN2, OUTPUT);
    pinMode(IN3, OUTPUT);
    pinMode(IN4, OUTPUT);
 
    // This statement will declare pin 15 as digital input 
    pinMode(PushButton, INPUT);
 
    // Defines the Buzzer pin as output 
    pinMode(speakerPin,OUTPUT);  
}
char * returnare_comanda_comunicatieUDP(int packetSize){
    int  key_neformata[58];
    int * key;
    int medie[24];
    static char * comanda;
    Serial.print("Pachet primit de dimensiunea ");
    Serial.println(packetSize);
    Serial.print("De la ");
    IPAddress remoteIp = Udp.remoteIP();
    Serial.print(remoteIp);
    Serial.print(", port ");
    Serial.println(Udp.remotePort());
 
    // read the packet into packetBufffer
    int len = Udp.read(packetBuffer, 255);
    if (len > 0) {
      packetBuffer[len] = 0;
    }
    Serial.println("Continut primit:");
    Serial.println(packetBuffer);
 
    String s = (String) packetBuffer; 
    for(int i=0;i<34;i++){
     key_neformata[i] = (int)(s[i]-48);
    }
    for(int i=34;i<58;i++){
     medie[i-34] = (int)(s[i]-48);
    }
    start_parcurgere=micros();
    key = restaurare_cheie(key_neformata);
    end_parcurgere=micros();
 
    Serial.println("Comanda:" );
      comanda = determinare_comanda(key,medie);
 
  return comanda;
}
 
void afisare_timpi(){
      double time_taken_criptare = double(end_criptare - start_criptare);
      double time_taken_decompresie = double(end_decompresie - start_decompresie);
      double time_taken_parcurgere = double(end_parcurgere - start_parcurgere);
      double time_taken_gen_cheie = double(end_generare_cheie - start_generare_cheie);
      double time_taken_TCP = double(end_TCP - start_TCP);
      double time_taken_UDP = double(end_UDP - start_UDP);
 
      Serial.print("----- Timp decriptare: " );
      Serial.println(time_taken_criptare);
      Serial.print("----- Timp decompresie: " );
      Serial.println(time_taken_decompresie);
      Serial.print("----- Timp parcurgere: " );
      Serial.println(time_taken_parcurgere);
      Serial.print("----- Timp generare cheie: " );
      Serial.println(time_taken_gen_cheie);
      Serial.print("----- Timp TCP: " );
      Serial.println(time_taken_TCP);
      Serial.print("----- Timp UDP: " );
      Serial.println(time_taken_UDP);
}
int k_secreta_server;
void loop(){
  char * comanda;
  int ka_kb;
  // Daca este valabil se citeste un pachet
  int packetSize = Udp.parsePacket();
 
  if (packetSize) {
      start_UDP=micros();
      comanda = (char *)returnare_comanda_comunicatieUDP(packetSize);
      end_UDP=micros();
 
      start_criptare=micros();
      descriptare_ceaser(comanda, k_secreta_server);
      end_criptare=micros();
 
     Serial.print("Comanda");
     for(int j=0; j<8; j++){
      Serial.print(comanda[j]);
  }
    Serial.println();
      char corect[] = {'d','e','s','c','h','i','d','e','\0'}; 
 
      if( strcmp(comanda, corect) == 0)
    {
        Serial.println("equal");
        stepper(FULL_ROTATION);
          delay(3000);
        stepper(-FULL_ROTATION);
    }
    else  Serial.println("not equal");
    afisare_timpi();
  }
 
 //realizare conexiune TCP pentru calcularea unei chei cu algoritmul DH
 WiFiClient client = server.available();   // verifica daca apar clienti
      start_TCP=micros();
  if (client) {                             // daca este detectat un client
    Serial.println("Client nou.");           // print a message out the serial port
 
    while (client.connected()) {            // cat timp clientul este conectat
      if (client.available()) {             // daca clientul transmite date
        int x_client = (client.read());             // se citesc datele
        Serial.println("Valoarea clientului secreta (int): ");
        Serial.println(x_client);                   
        int x_server = power(S_cunoscut,S_secret,C_cunoscut);
        client.write(x_server); 
        k_secreta_server = power(x_client, S_secret, C_cunoscut);
        Serial.println("Cheia secreta este: ");
        Serial.println(k_secreta_server);
        ka_kb=k_secreta_server;
      }
        // inchide conexiunea
        client.stop();
    }
      Serial.println("Clientul s-a deconectat.");
  }
      end_TCP=micros();
 
  int Push_button_state = digitalRead(PushButton);  // in cazul apasarii butonului pentru sonerie va incepe soneria
   if (Push_button_state == HIGH){
    Serial.println(" ---- Buton apasat ---- ");
 
     for(int n = 0 ; n < 73; n++){
        playNote(note[n], beats[n] * tempo);  
         delay(tempo / 2);
         }
 
   }
 
  //Asteapta o secunda
  delay(1000);
}

2. Video Capturing System