Andrej Karpathy reveals 'Dobby' AI agent managing his smart home

Andrej Karpathy, a founding member of OpenAI and former director of AI at Tesla, has revealed how he integrated a sophisticated artificial intelligence system to manage his household chores.

Speaking on the No Priors podcast released on Friday, the Eureka Labs founder described his bespoke setup, which he has affectionately named Dobby after the Harry Potter house elf.

This autonomous agent, which Karpathy refers to as a claw, consolidates various home automation tasks into a single interface, effectively replacing six separate applications previously required to monitor his property and utilities.

The system is designed to interact directly with internet-connected devices rather than merely generating text.

Karpathy explained the scope of the agent's responsibilities, stating: “I have a claw basically that takes care of my home, and I call him Dobby the Elf Claw. It controls all of my lights, my HVAC, my shades, the pool, the spa, and also the security system.”

Beyond climate control, the system provides real-time updates on home deliveries via WhatsApp, identifying specific couriers through external camera feeds.

Karpathy’s domestic AI recently received a significant hardware enhancement from Nvidia CEO Jensen Huang, who gifted him a DGX Station powered by a GB300 superchip.

This hardware is specifically engineered for agentic AI workloads, providing a more robust environment for home automation.

Despite the system's extensive control, Karpathy maintains that security remains a priority as the agent operates on a local network.

He highlighted the convenience of the setup, noting: “I can ask it, 'Dobby, it's sleepy time.' That just means all of the lights go off.”