• Author: Niculescu Rares Cosmin
• Email: rares.niculescu@stud.acs.upb.ro
• Master: GMRV
• Source code, video and presentation: gitlab TODO

The inspiration came from the 2014-2018 period when I was a part-time DJ at a club in my hometown. In addition to the DJ equipment, I also had a lighting setup consisting of several RGB LED bars that could be programmed and controlled using a DMX console to create a vibrant atmosphere at the party. Over time, I developed a true hobby for programming the LED bars, elements that are present at any show/party with at least 100 participants.

The purpose of this project is to simulate the behavior of a RGB LEDBAR. The user will be able to interact with the LED bar through a web interface that will change the lighting mode of the LEDs. The LEDs can be manually controlled (the final color will be set using sliders), automatically controlled using specific algorithms for coloring the LEDs, similar to those found in Christmas lights, or customized through a schedule created by the user (that will be uploaded in a Firebase Database). The automatic mode will provide 4 different lighting possibilities.

• ESP-WROOM-32 - ESP32 Sparrow
• Breadboards - 1 half-breadboard + 1 full breadboard
• Wires - as many as possible :)
• 1k resistors - 12 of them are necessary in this project
• Common cathode RGB Led - 12 of them are necessary in this project

• Firebase Realtime database - For saving the schedule for the custom mode
• Simple Web Page - a simple Web Page designed to encompass all 3 functionalities. The Web Page is stored in the Flash Memory of the ESP32.
• Arduino IDE alongside the required libraries for ESP32: SPIFFS, WiFi, AsyncTCP, ESPAsyncWebServer, Firebase_ESP_Client

Shortly, the architecture has the classic client-server structure, where http client is the user controlling the ledbar, through the web interface, and the esp32 + “firebase” is the server.

• The ESP32 needs a WiFi connection and the API Key + the RTDB URL.
• Once the setup is complete the user can access the application.
• The serial monitor will output the local IP where the user will need to connect.

• The user will navigate through the simple web interface by choosing the lighting mode.
• In the manual mode the user will control the ledbar through 9 different sliders (since we have 6 individual columns, grouped by 2). Each slider modifies the intensity of a RGB channel.
• In the automatic mode the user will control the ledbar by selecting an individual lighting mode. There are 4 buttons on this page and the user can choose between: “Christmas Mode”, “Running”, “Blinking”, “Full Auto”.
• In the custom mode the user will control the ledbar through an schedule uploaded in the Firebase DB. At first the LEDBAR is completely red and the user must not interact with the Web Page. After the ESP32 downloads the content from DB, the LEDBAR turns green and it's ready to “play” the content. The user will manually trigger the schedule by pressing the “START!” button.

• The web interface is locally stored in the ESP32 SPIFFS.
• Initial State
• 
• Manual Mode
• 
• Auto Mode
• 
• Custom Mode
• 

For starters I set up the environment similar to the demos found in labs 3 and 4. For the Web server part, I used the Async Web Server library and configured it similarly to the “SPIFFS Webserver” demo. In other words, the ESP32 web server displays the control page, and when the user interacts with the elements of the page the browser makes a request on a URL, ESP32 receives the request and parses it and, finally, it sends a PWM signal with the corresponding value to the GPIOs. It will also change some internal parameters. As I mentioned before the web page is stored in the internal memory.

After that I have configured the Firebase Database. I have assigned the API key and the RTDB URL and I have signed up on the firebase.

The main idea of the project is to switch between individual “states”. For example state 1 is for the manual mode, state 2 is for “auto mode - Christmas Mode and so on.” Inside the loop function is where the behavior is altered. For the manual and auto mode the behavior is modified 100% by the “internal code” I have implemented. For the custom mode the ESP32 interrogates the Firebase DB to see if there is any schedule to download. If there is none, the ESP32 will wait until the DB updates the user changes the lighting mode. If there is any content in the DB, the ESP32 will download the schedule and save it in a local file. Once the download is complete the ESP32 waits for the user to actually trigger the schedule.

• Finding an easy way for a user create their own machine learning model, with minimal knowledge required. The main idea of the system was that all the user has to do, is to provide pictures of their house cats. I have tried multiple free models with varying degrees of success, but none as good as Google's Teachable Machine, with provided a nice web interface to generate classifiers and export them as tensorflow models.
• Testing was, and still is, hard to do without an actual cat. I have tested the model with the web interface by uploading pictures already taken, but most importantly I had to test the predictions with pictures from esp32 camera. What I did was to put my phone, which displayed a picture of a cat and trigger the motion sensor. Since the camera has quite low resolution, combined with the fact that the cat was presented on a not-so-good screen, the predictions of the model were less accurate.
• The algorithm for cat detection used in the esp32 could still be improved. For example the esp may take multiple pictures when the motion sensor is triggered, in case the cat is not detected properly even if it was there. Also, it would be nice to provide our own classifier model and web interface, and not be reliant on Teachable Machine.
• The project is also dependent on a stable WiFi connection. It would be nice if the esp32 saved the pictures in it's ram or on persistent storage and send them whenever a connection is available. Or, even better, use a SIM card for internet connectivity so the hardware could easily work outdoors.
• The board could also connect to the internet only periodically in order to conserve energy.

This project provides an easy way to control and monitor the behavior of your house pets. Although here still is room for improvement the ease of setting up the project, and on the finesse of the detection algorithm, it provides a good basis for further development. All of the basic requirements are implemented and quite useful, requiring just a bit more polishing. In summary, this project integrates robust web technologies and relatively cheap hardware in order to fulfill an useful task.

• https://media.digikey.com/pdf/Data%20Sheets/DFRobot%20PDFs/DFR0602_Web.pdf
• https://www.mpja.com/download/31227sc.pdf
• https://teachablemachine.withgoogle.com/
• https://firebase.google.com/docs/functions
• https://www.tensorflow.org/js/tutorials