Skip to content

🤖 Jarvis: My AI-Powered Assistant

🥚 “Sir, may I remind you the system is barely holding together? Much like your sleep schedule.”

Jarvis is my personal AI assistant — part automation hub, part conversational sidekick, part mad science experiment. Inspired by Tony Stark’s J.A.R.V.I.S., it’s designed to live in my homelab with the same wit, sarcasm, and charm.

Built in Python and powered by OpenAI, Jarvis integrates with smart home devices, memory systems, and monitoring tools. It’s always evolving, sometimes helpful, and occasionally judgmental.

🧠 Core Features

  • Voice & Text Interface – Wake word (Jarvis!), dismiss phrase (that's all for now), responses via TTS or text.
  • Home Assistant Integration – Maps natural language to service calls (e.g., light.turn_on, media_player.play_media).
  • Spotify Casting with Spotcast – Context-aware commands like “play lo-fi in the kitchen”.
  • Short & Long-Term Memory – Session memory + FAISS embeddings for long-term recall. Supports smart tagging, contextual recall, forgetting, and inferred knowledge.
  • Personality Engine – Sarcastic by default, extensible with future modes (Friday, HAL, etc.).
  • System Integrations – Proxmox, Portainer, and TrueNAS APIs.
  • Command Execution – Shell commands, system queries, service checks.
  • Plugin System – Drop-in Python skills folder for new automations/APIs.
  • Docker-Ready Backend – Packaged for container deployment in my homelab.

🔧 Stack Breakdown

Component Purpose
Python Core framework and logic
OpenAI GPT-4o Conversational intelligence + sarcasm
FAISS Long-term memory via vector search
Home Assistant Smart home backend + device control
Spotcast Spotify casting to Nest/Hue devices
ElevenLabs (TTS) Natural voice output (sarcastic British tone)
Proxmox API VM + cluster monitoring
Portainer API Container orchestration monitoring
TrueNAS API Storage system monitoring

🔮 Roadmap / Future Features

  • Model Router – Switch between GPT-4o and local LLMs (Ollama, Mistral, Phi).
  • Local Fallback Mode – Offline STT, LLM, and TTS support.
  • Multi-Voice Personalities – Swap ElevenLabs voices on command.
  • Web Dashboard – Memory review, logs, and system GUI.
  • Client-Server Architecture – Raspberry Pi / ESP32 voice clients.
  • Proactive Announcements – Weather, trash pickup, motion alerts.
  • Context-Aware Routines – Modes like Movie Night or Work Mode.
  • Self-Review Summaries – Periodic memory consolidation reports.
  • Role-Based Profiles – User-specific routines (kids vs adults).

💬 Sample Commands

💡 Smart Home Control

You: "Jarvis!"
Jarvis: "At your service."
You: "Turn off the kitchen lights."
Jarvis: "Ah, the dramatic flair of darkness. As you wish."
[Executes light.turn_off]

🔈 Spotify Casting

You: "Jarvis, play my workout playlist in the living room."
Jarvis: "Activating beast mode. Don't make me join you."
[Executes spotcast.start → media_player.living_room]

You: "Jarvis, play something relaxing."
Jarvis: "Engaging mellow mode. May I suggest lo-fi and existential dread?"
[Executes spotcast.start with chill playlist]

😏 Sass & Sarcasm

You: "Jarvis, what's the weather today?"
Jarvis: "Let me guess… outside? One moment while I confirm with the satellites."
[Pulls weather info via Home Assistant]

You: "Jarvis, turn off everything."
Jarvis: "Everything? Even my will to serve? Fine. Shutting it all down."
[Executes multiple turn_off commands]

🖥️ System Monitoring

You: "Jarvis, how's Proxmox looking?"
Jarvis: "One node is sleepy, two are hungry for resources. Same as always."
[Returns VM/node status via Proxmox API]

You: "Jarvis, are my containers okay?"
Jarvis: "Portainer says all systems nominal. But I still don’t trust that Jellyfin guy."

🧪 Experimental / Absurd

You: "Jarvis, launch the nukes."
Jarvis: "Initiating... just kidding. But I did dim the lights for dramatic effect."
[Prank automation → turns off lights]

You: "Jarvis, tell me a joke."
Jarvis: "You. Asking me to do manual labor. That’s the joke."

You: "Jarvis, play baby shark."
Jarvis: "No."

🧪 Why I Built It

I didn’t want another generic smart speaker. I wanted something programmable, sarcastic, and truly mine — an assistant I can extend, train, and integrate with my homelab.

Jarvis blends automation, AI, and playfulness into one evolving project that runs entirely on my infrastructure.

📂 Coming Soon

  • Source code (with selective redactions)
  • Architecture diagram
  • Setup guide for running Jarvis in your own lab

👉 More details: github.com/maxthomas95/homelab-jarvis