Firmware · Hardware
Washing machine monitor with a camera and an accelerometer
An ESP32-S3 board with a camera and screen “reads” the panel LEDs of a Mabe washing machine and its vibration to know the current cycle stage and notify Home Assistant.
- Year
- 2026
- Status
- Working prototype
- Role
- Firmware, computer vision and calibration
- Repository
- Repository coming soon
01 · Problem
The washing machine has no connectivity and does not tell you when it is done. The only clue to the cycle stage are the panel LEDs: fill, soak, wash, rinse, spin, pause, lid lock and load sensing.
02 · Approach
A Waveshare ESP32-S3 board (touchscreen, OV2640 camera and QMI8658 accelerometer) faces the panel. An OpenCV script analyzes calibrated regions over the 8 LEDs —including blinking, with a sample history— and the board combines that state with the vibration RMS in a state machine with hysteresis.
03 · Result
It detects filling, soaking, washing, rinsing, spinning, paused and done; publishes the state over MQTT for Home Assistant and shows it on screen with LVGL. It includes tools to calibrate the regions on the live image and tune thresholds without reflashing.
04 · Stack
- ESP32-S3
- OV2640 camera
- QMI8658
- LVGL
- FreeRTOS
- MQTT
- Python
- OpenCV
How it decides the stage
- LEDs. Every 2 s a frame is grabbed and each region’s brightness is normalized against the image median. A 4-sample history tells a solid LED from a blinking one.
- Vibration. The accelerometer samples at 50 Hz; over a 2 s window gravity is removed, the RMS is computed and classified into four levels: quiet, mild, agitating and spinning.
- State machine. LEDs lead; vibration breaks ties (e.g. strong agitation with no clear LED = washing). A transition is only confirmed after 6 consecutive 500 ms evaluations, and “done” is held for 5 minutes.
Why the analysis moved off the chip
The first version detected the LEDs on the ESP32 itself, but switching the sensor’s pixel format to analyze frames made the video stream unstable. I moved detection to a Python script on the PC that publishes the LED state over MQTT; the board consumes it exactly as before and the stream is now stable.