Case Study
Paco: a voice AI that answers WhatsApp calls
Paco answers a normal WhatsApp voice call and holds a real-time conversation. It checks email, calendar, weather and the web while you talk, and it can call you back with a spoken briefing. I built it from an empty folder to a production deployment in a single day.
The problem
I wanted to talk to my assistant the way you talk to a person: by calling it. Not a chat window, not push-to-talk voice notes. A real WhatsApp voice call that gets answered, understood, and responded to with natural latency.
WhatsApp has no public API for answering voice calls, so the interesting engineering problem was building a reliable bridge between a WhatsApp call and a real-time speech model.
Architecture
An inbound WhatsApp call reaches the WaVoIP SDK running inside headless Chrome under Puppeteer. A Node.js bridge captures the call audio, resamples it between the 16 kHz telephony rate and the 24 kHz the model expects, and streams it to the OpenAI Realtime API over WebSocket. The model's spoken reply streams straight back to the caller.
Outbound works too: a POST to the bridge's HTTP API starts a WhatsApp call to my number and delivers a spoken briefing built from live data.
The build, hour by hour
The whole system went from nothing to production on 1 March 2026. The timeline comes straight from the project log:
Running it for real
Paco runs as a daemon on my own Ubuntu server behind Nginx, deployed with PM2 and monitored through a custom operations dashboard. Mid-call, the model calls out to tools for email, calendar, weather and web search, so answers reflect live data rather than training data.