🧪 Reliability & CI Improvements

This update also focuses on improving the project’s reliability and development workflow.

What’s New

• Added 20 Pytest test cases covering the core functionality of the system, helping verify that critical components behave correctly and reducing the risk of regressions.

• Configured GitHub Actions to automatically execute the entire test suite after every commit and pull request, ensuring new changes don’t break existing functionality.

Why It Matters

Automated testing and Continuous Integration (CI) make the project more robust, maintainable, and production-ready. Every code change is now automatically validated before it becomes part of the project, providing faster feedback during development and increasing confidence in future updates.

🐳 Docker Setup: 100% Functional

Docker support for the Edge-AI Home Monitoring System is now fully working end-to-end.

What’s New

• First-Run Setup via HTML: Instead of manual configuration steps, the very first launch now opens a lightweight HTML setup page that walks through the initial configuration once (credentials, device registration basics, etc.). After this one-time run, it’s never needed again — the container remembers everything through the persisted config volumes.
• One Command to Rule Them All: The whole system — Flask backend, Docker container, and now the first-run setup — is launched with a single script:
  • Windows: start.bat
  • Linux: start.sh

No more manually running docker compose, checking dependencies, or following multi-step instructions. Just run the script for your OS and the entire stack comes up ready to use.

Why It Matters

This closes the loop on the Docker migration from a few devlogs ago. Before, Docker removed the Python/venv/dependency headaches but still required knowing the right compose commands. Now the onboarding experience is genuinely “clone → run one script → done,” with the HTML first-run page handling the last bit of manual setup that used to require touching config files by hand.

🚀 One of the Biggest Updates: Docker-Based Setup & Minimal Installation

The Edge-AI Home Monitoring System has just received its most significant infrastructure overhaul yet!

We have completely re-engineered the deployment workflow. Instead of dealing with manual environment configurations, matching Python versions, and tracking down dependency conflicts, the entire ecosystem can now be spun up instantly with a single command.

---

🐳 What’s New: Containerized Architecture

By migrating the platform to a fully containerized backend, we’ve moved away from fragile local setups closer to a production-grade deployment.

Key Improvements:

• Isolated Environment: The Flask backend and all its core libraries run inside an isolated Docker container, completely immune to host-system configuration drifts.
• Automated Dependency Management: All heavy-lifting dependencies are resolved and cached automatically inside the image.
• State & Configuration Persistence: Your custom devices_config.json, automation rules, and local database models are securely mapped via Docker volumes, meaning they persist perfectly across container restarts and updates.
• High Portability: Moving the hub from a development PC to a dedicated home server (like a Fedora Linux box or a Raspberry Pi) is now completely seamless.

---

⚡ Setup: Before vs. After

The difference in onboarding friction is night and day. Look at how the installation footprint has shrunk:

Old Manual Setup New Docker Workflow ❌ Install Python manually & match versions 📦 Pre-configured runtime environment ❌ Create virtual environments (venv) 📦 Isolated container filesystems ❌ Debug pip dependency conflicts 📦 One-time automated image build ❌ Manually set environment variables 📦 Declarative configuration via Compose ❌ Run background scripts step-by-step 📦 Single-command multi-service launch

The New Minimal Installation Path:

Now, launching your privacy-first smart home hub requires nothing more than standard container tools:

# Clone the repository
git clone https://github.com/travletothefurureprogramming/Edge-AI-Home-Monitoring-System

# Navigate to the project root
cd Edge-AI-Home-Monitoring-System

# Spin up the entire system in detached mode
docker compose up -d

Enchanse CLI app using questionary and start making a clean README with clear explanation on how to use it

Fix some bugs and create a clean repository with a good README and the github release.

UI Modernization & Native Sonos Integration 🎵🎨

A massive structural and visual update has landed! The Edge-AI Home Monitoring System dashboard has been rewritten into a production-grade, single-page interface using the modern Inter typeface family. Alongside this visual overhaul, we are introducing native, local-only support for Sonos Smart Audio Devices.

---

What’s New

1. High-End Minimalist UI & CSS Token Architecture

The new interface shifts away from crowded layouts to a highly scannable, component-driven architecture featuring a strict color and spacing utility system built directly into :root.

• Unified Palette System: Built entirely on minimal token foundations: light-mode canvas backgrounds (#fafafa), precise surface modules (#ffffff), and structural crisp border outlines (#e7e7e7 / #d9d9d9).
• Blur-Effects Navigation Header (.topbar): A sticky system header utilizing backdrop-filter: blur(8px) that embeds the new brand mark (A Home) alongside a live status strip displaying the total device count, registered automations, and an immediate visual green heartbeat indicator (.live-dot).
• Decoupled Grid Control (.grid): Devices are rendered dynamically using a responsive auto-fill layout (minmax(240px, 1fr)) with explicit styling classes for device actions (.btn.on for soft green feedback, .btn.off for soft red, and .btn.accent for special triggers).
• Asynchronous AI Assistant Component (.ai-panel): A dedicated section housing the local LLM interaction logic, optimized with a smooth CSS-animated text typing loop (@keyframes dotfade) to indicate processing states without blocking UI interactions.

2. Native Sonos Smart Audio Core Integration

Audio control joins our local device tree. The system now natively communicates with Sonos devices over the local network via unified backend hooks, matching our privacy-first design patterns.

The control grid maps the specific target API endpoints directly to native UI inputs, enabling immediate transmission of core execution parameters:

• play — Resumes audio streaming (.btn.on)
• pause — Pauses current media layout tracking
• stop — Breaks audio buffering completely (.btn.off)
• volume_up & volume_down — Alters hardware volume steps

Real-Time “Now Playing” UI Subview

Sonos control cards now dynamically build and inject an interactive media HUD element (.now-playing).

• Dynamic Media Binding: Communicates with the new /api/music/current endpoint via explicit POST JSON payloads ({ room, type, number, command: 'get', device }).
• Visual Playback Flags: Implements dynamic active toggles (.now-playing.playing) that shift asset opacity states depending on network audio feedback.
• Non-Blocking Control Pulls: The inline manual update button (.np-refresh) relies on an isolated asynchronous CSS loop wrapper (@keyframes spin) allowing updates without triggering full dashboard asset re-renders.

Samsung Smart TV Support Added 📺

The Edge-AI Home Monitoring System now includes full support for Samsung Smart TVs running Tizen OS, expanding the platform’s smart device ecosystem with another major TV manufacturer.

This integration uses the Samsung TV WebSocket API, allowing the system to communicate directly with compatible TVs over the local network without requiring additional hardware such as IR blasters.

What’s New

Added a new Samsung TV backend integration using the samsungtvws Python library.

Implemented support for:

• Power control (ON/OFF)
• Remote key commands
• Volume control
• Device information retrieval

Samsung TVs are now registered as a native device type inside the system, following the same architecture used by other integrations such as LG WebOS, Daikin, Tapo, Hue, Yeelight, Shelly, and Broadlink.

Why It Matters

Until now, TV support was mainly focused on LG Smart TVs and IR-controlled televisions through Broadlink.

With Samsung Smart TV support, the system can now directly control another major smart TV ecosystem using its official network interface.

This improves:

• Response time compared to IR control
• Local-only communication (no cloud dependency)
• Compatibility with more homes

Technical Details

The integration communicates with Samsung TVs through:

• Samsung Tizen WebSocket API
• Local network TCP communication
• Device authentication pairing

The system keeps the same device abstraction model used by the rest of the project, making Samsung TVs behave like any other controllable smart device.

Challenges

Some challenges during implementation:

• Handling Samsung’s first-time pairing approval system
• Managing WebSocket connections reliably
• Ensuring compatibility between different Tizen TV models
• Mapping Samsung remote key codes into the existing device command structure

Next Steps

Future improvements:

• Add automatic Samsung TV discovery
• Add real-time TV status feedback (power state, current app, volume)
• Add voice assistant commands for Samsung TVs - Add more controls

Samsung Smart TVs are now another fully supported platform in the Edge-AI Home Monitoring System, bringing the project closer to becoming a complete privacy-first smart home hub.

Full Broadlink Universal Remote Support

The Edge-AI Home Monitoring System now includes complete support for Broadlink Universal Remotes, allowing the platform not only to learn IR commands but also to transmit them directly to infrared-controlled devices.

What’s New

• Added a dedicated Broadlink backend integration for sending stored IR commands.
• Implemented a new /api/broadlink endpoint for controlling Broadlink devices from the dashboard and automation engine.
• Added a Broadlink device class responsible for:
  • Connecting and authenticating with Broadlink RM-series devices.
  • Reading learned IR codes from broadlink_codes.json.
  • Sending IR packets using device.send_data().
• Full support for controlling:

  • TVs
    

  • Air Conditioners
    

  • TV Decoders / Set-Top Boxes
    

• Broadlink devices are now treated like any other supported smart device and can be used throughout the entire system.

Why It Matters

This is a major milestone because the system is no longer limited to devices with official APIs.

Thousands of legacy appliances—including older televisions, air conditioners, and cable/satellite decoders—can now be integrated into the smart home ecosystem using only their existing infrared remote controls.

Users can control these devices from:

• The Web Dashboard
• Automation Rules
• Future Voice Assistant integrations
• Any API client connected to the system

This significantly expands compatibility while maintaining the project’s privacy-first, local-control philosophy.

Challenges

• IR packets must be converted back into raw byte format before transmission.
• Maintaining synchronization between learned codes and registered Broadlink devices remains important as the number of supported devices grows.
• Multiple Broadlink hubs on the same network still require a device selection mechanism, which will be addressed in a future update.

Next Steps

• Add automatic Broadlink hub discovery and selection.
• Integrate Broadlink commands into the Rule Engine as event triggers.
• Add support for macro commands (multiple IR signals with delays).
• Improve the dashboard with command management and testing tools.

Broadlink IR Learning App + Voice Guidance in Setup

The Edge-AI Home Monitoring System now has its own tool for learning IR signals, as well as voice guidance throughout the setup process.

What’s Added

Broadlink IR Learning App

• New desktop application (customtkinter) that connects to a Broadlink device (RM-series) via broadlink.discover() and authenticates automatically.
• Device type selection (TV / AC / Decoder) with dynamic list of commands depending on the type (on/off, channel up/down, volume, direction, menu, etc.).
• “Learn & Save” function: Broadlink enters learning mode, waits for the signal from the real remote control and saves the hex packet.
• The signals are stored in two places:
• broadlink_codes.json — file-library of all codes per room/device/command.
• devices_config.json — so that the device is automatically registered as an entry in the system’s main config, with flag is_broadlink_device: true.

Voice Guidance in Setup Wizard

• Added text-to-speech (via pyttsx4, in a separate thread so as not to block the UI) to each step of the setup wizard: installing dependencies, downloading AI model, creating password, setting Tailscale, Telegram token/chat ID, and registering devices.
• Each device type during registration (Tapo, Philips Hue, Shelly, Broadlink, etc.) now “explains” vocally which fields need to be filled in, making the process more accessible for someone who is setting up the system for the first time without reading documentation.
• Also added broadlink as a new device type in the setup wizard dropdown, which opens the Learning App directly when selected.

Why It Matters

Until now, the system only supported devices with official APIs (Tapo, Hue, Yeelight, LG WebOS, Daikin). Many “dumb” devices at home (old ACs, decoders, TVs without smart features) are controlled only via IR remote — the Broadlink learning app paves the way to integrate virtually any device with an IR remote into the ecosystem, without the need for a special library per brand.

Voice guidance also makes setup more accessible — less need to read README step by step, the system tells you what to do live.

Challenges

• async_connect of the Kasa class has incorrect syntax (await self.device = ... instead of self.device = await ...) — same problem as previous devlog, still pending.
• In the Learning App, the device data is temporarily stored in two separate files (broadlink_codes.json and devices_config.json) — care must be taken to ensure that they do not become out of sync if a write fails in the middle.
• The Broadlink device is always identified as devices[0] by broadlink.discover() — if there are multiple Broadlink devices on the network, the app will always connect to the first one found, without selection.
• TTS requires CoInitialize/CoUninitialize on each thread (Windows-specific), which makes the code less portable to other OSes.

Next Steps

• Full support for sending signals via Broadlink devices (not just learning) — i.e. new /api/broadlink endpoint and Broadlink class in the backend that will read the stored hex codes from broadlink_codes.json and send them via device.send_data().
• Integrate Broadlink devices as actions in the Rule Engine, so they can be used in automations like any other device.
• Wiring emit_device_event to Broadlink sends, so they can also be trigger sources.
• Fix bug in Kasa.async_connect before any other work on Kasa progresses.

Kasa (TP-Link) Support Added

The Edge-AI Home Monitoring System now supports TP-Link Kasa smart plugs/switches, using the python-kasa library’s async Discover API for device communication.

What’s New

• New Kasa class that discovers a device by IP via Discover.discover_single() (with optional username/password for cloud-authenticated devices) and wraps turn_on / turn_off / execute_command behind the same asyncio.run() pattern already used for Daikin and the Tapo devices.
• New /api/kasa endpoint, gated behind @auth.login_required like the rest of the device APIs.
• Registered kasa in DEVICE_ENDPOINTS and wired it into create_device_action, so Kasa devices can already be used as automation actions from day one.
• Added send_kasa and send_daikin helper functions alongside the other device senders, for consistency across the codebase.

Why This Matters

Kasa is one of the most common smart plug ecosystems out there, so this adds another affordable device family to the system without needing a proprietary hub — anything plugged into a Kasa switch can now be turned on/off from the dashboard or from an automation rule, the same way Tapo and Shelly devices already are.

Challenges

• Ran into a syntax issue in async_connect: await self.device = Discover.discover_single(...) isn’t valid — assignment has to come first, i.e. self.device = await Discover.discover_single(...). Needs fixing before this can actually run.
• handle_kasa currently points at a config path (/config/device_config.json) that doesn’t match the filename (devices_config.json) or the relative-path convention used by every other handler — needs to be corrected to "config/devices_config.json".
• Same route also references data["Room"][room][dev_type][number][ip] with ip unquoted (a variable reference instead of the string key "ip"), which will throw a NameError rather than the intended KeyError.
• emit_device_event still isn’t wired into the Kasa (or Shelly) handler, so — same as the other non-TV devices — Kasa state changes can’t yet act as automation triggers.

Next Steps

• Fix the await/assignment order bug in Kasa.async_connect, the config path, and the unquoted "ip" key in /api/kasa before testing against a real device.
• Wire emit_device_event into Kasa, Shelly, and the remaining handlers so any device can act as both an action and a trigger.
• Once event emission is consistent across all device types, move on to energy monitoring for the Tapo smart plugs.

Shelly Support Added

The Edge-AI Home Monitoring System now supports Shelly smart relays, using the ShellyPy library to control relay-based devices (plugs, switches, etc.) over the local network.

What’s New

• New Shelly class wrapping ShellyPy.Shelly, with turn_on_relay / turn_off_relay methods and an execute_command dispatcher for on/off commands.
• New /api/shelly endpoint that reads the device’s IP and relay_number from devices_config.json and forwards the command to the relay.
• Registered shelly in DEVICE_ENDPOINTS and wired it into create_device_action, so Shelly devices can already be used as automation actions (schedule or event triggered), same as every other device type.

Why This Matters

Shelly relays are a common, cheap way to add “dumb” appliances into the automation system — a fan, a boiler, anything behind a relay can now be toggled from the dashboard or from a rule, without needing a dedicated smart-device ecosystem.

Challenges

• Noticed /api/shelly is currently missing the @auth.login_required decorator that every other device endpoint has — since authentication was added, this is the one route that slipped through. Needs to be fixed before this ships anywhere network-exposed.
• Shelly relays are addressed by relay_number rather than an id/model like the other integrations, so the config schema needed a small variation (relay_number field) instead of reusing the existing device pattern exactly.
• emit_device_event isn’t wired into the Shelly handler yet, so — like the other non-TV devices — a Shelly state change can’t currently act as a trigger for other automations.

Next Steps

• Add the missing @auth.login_required to /api/shelly.
• Wire emit_device_event into the Shelly (and remaining) handlers so any device can be both an action and a trigger source.
• Once event emission is consistent across all device types, move on to energy monitoring for the Tapo smart plugs.

Device-Triggered Chaining: Automations Now React to Device State Changes

The Rule Engine already supported schedule triggers and one hardcoded event source (the YOLO “person_detected” signal). This update adds a much more general capability: automations can now trigger off of any device state change, not just security events.

What’s New

• Added emit_device_event(room, dev_type, command), a helper that builds an event name in the form {room}{dev_type}{command} (e.g. living_room_lg_tv_on) and forwards it to AutomationManager.trigger_event.
• Wired this into the /api/tv endpoint: every time a TV command succeeds, the system now emits an event describing exactly what just happened.
• This means a rule can be created like “when the Living Room TV turns on → turn on the Living Room lights” without any new trigger type — it reuses the same event-rule machinery already built for the security pipeline.
• Event names are deterministic and human-readable, so building new automations from the dashboard just means picking the right device + command combo as the trigger.

Why This Matters

Until now, “smart” behavior meant either a fixed schedule or a single special-cased sensor event. This generalizes the pattern: any device action can become a trigger for any other device action, which is the actual foundation of “automation chaining” (TV on → lights on, plug on → AC on, etc.) rather than a one-off integration.

Challenges

• emit_device_event is currently only called from the TV handler. The Tapo, Philips Hue, Yeelight, and Daikin handlers don’t emit events yet, so right now only TV state changes can kick off a chain — everything else is still a dead end as a trigger source. This is the next thing to close.
• Needed to be careful that emitting the event happens after the command actually succeeds, not just after it’s requested, so a failed TV command doesn’t fire a chain based on a state change that never happened.

Next Steps

• Add emit_device_event calls to the remaining device handlers (Tapo light/LED strip, Hue, Yeelight, Daikin) so any device can act as a trigger, not just TVs.
• Once persistence for automation actions is fully solid across restarts, move on to energy monitoring for the Tapo smart plugs.

Automations are Live: Schedules & Event Triggers

The Rule Engine I mentioned as “next steps” in the last few devlogs is now fully working. The Edge-AI Home Monitoring System can now react on its own, without me touching the dashboard.

What’s New

• New AutomationManager class that manages rules independently of the Flask request lifecycle, running on its own background thread
• Schedule triggers: fire any device action at a specific time of day (cron-based under the hood, e.g. “turn off all lights at 23:30”)
• Event triggers: fire actions the moment something happens in the system — currently wired up to the YOLO security pipeline, so a rule can say “when a person is detected, turn on the living room lights”
• Automations are managed straight from the dashboard: add a rule (pick a device, a command, and either a time or an event), see the active list, delete on demand — no need to touch the config files by hand
• Rules persist to disk, so they survive a server restart

Challenges

The trickiest part wasn’t the scheduling logic itself — it was making sure automation actions don’t accidentally go through the same path as normal API requests. Since every device endpoint is gated behind session-based login, having the automation thread call itself over HTTP triggered 401s (a background thread has no browser session, obviously). Fixed by having automation actions call the device classes directly instead of looping back through HTTP — cleaner and faster anyway, since it skips a full request round-trip for something that’s technically internal.

Next Steps

Energy monitoring for the Tapo smart plugs, so automations can eventually be built around actual consumption data too.

Create Autoation Manager class

Added Daikin AC Support

The Edge-AI Home Monitoring System now extends into climate control. I’ve added full support for Daikin air conditioners, using the pydaikin library for asynchronous communication with the device over the network.

What’s New

• New DaikinAC class with full control: on/off, increase/decrease temperature, and mode switching (auto, cool, heat, dry, fan)
• New /api/daikin endpoint for device communication
• Integrated into the same authentication pattern used across the whole system (@auth.login_required), so no new device ecosystem opens a security gap
• Dashboard controls for Daikin devices, including a mode selection dropdown

Challenges

Since pydaikin is fully asynchronous (async context managers), each client call had to be wrapped with asyncio.run() to fit into the rest of the synchronous Flask backend — the same approach used for the Tapo devices.

Next Steps

Rule Engine / Automations system, followed by energy monitoring for the Tapo smart plugs.

Improving the Project Structure

One of the biggest challenges in the project was the lack of organization. As the codebase grew, it became harder to navigate and maintain.

Today I reorganized the project into a much cleaner structure, making the code easier to understand, extend, and maintain. While there’s still room for improvement, this is a solid step toward a more scalable architecture.

What’s Next?

• Add automation features.
• Implement energy consumption monitoring for Tapo devices.

Added Authentication System

Up until now, the dashboard and all API endpoints were completely open to anyone on the network — a serious problem for a system that controls cameras, smart devices, and even has Remote PC Control capabilities.
I implemented a single-admin authentication layer using Flask sessions and password hashing (werkzeug), gating access behind a login page.
Key changes:

• Added a /login route with session-based authentication
• Passwords are hashed (never stored in plain text) using pbkdf2:sha256
• Wrapped every sensitive route with a @login_required decorator — dashboard, device control APIs, camera toggle, and system/remote-PC endpoints
• API routes return a clean 401 Unauthorized instead of redirecting, so the frontend can handle it gracefully
• Sessions auto-expire, requiring re-login after a set period

This closes the biggest security gap in the project — previously, anyone with access to the local network or Tailscale IP could control devices or the PC without any barrier. Now, all of that is locked behind credentials only I know.
Next steps: Rule engine for automations (trigger-based actions), followed by energy monitoring for Tapo smart plugs.