A voice-first AI debugging assistant built on OpenAI Realtime API and AutoGen, upgrading rubber duck debugging into a bidirectional real-time voice conversation system.
DUCK-E (Digitally Unified Conversational Knowledge Engine) is an AI-powered voice assistant that transforms the classic "rubber duck debugging" method into an interactive, bidirectional conversation system. The project is built on FastAPI for the web backend, uses WebSocket for real-time audio streaming, and features a frontend built with Jinja2 templates and vanilla JavaScript, supporting one-click Docker deployment. Current version is 0.2.89 under MIT License.
Core Capabilities#
Voice Interaction: Leverages WebRTC and WebSocket for low-latency real-time voice conversation with natural interruption support. Simply click the microphone icon in the browser to start.
Multi-Model Coordination: Calls OpenAI Realtime API's gpt-realtime model for speech stream processing, with on-demand switching to gpt-5-mini (fast general tasks) and gpt-5 (deep reasoning). Multi-agent orchestration is implemented via Microsoft AutoGen framework's RealtimeAgent.
Tool Calling: Built-in real-time weather queries (WeatherAPI), 3-day weather forecasts, and web search, with extensibility through the @realtime_agent.register_realtime_function(...) decorator for custom tools. Conversation context is maintained throughout the debugging session, and model configuration is auto-generated with only an OpenAI API Key required.
Security & Operations#
Since v0.2.0, security features include built-in rate limiting (WebSocket default 5/minute), cost protection ($5/session cap, $100 circuit breaker), input validation against 10+ attack vectors including SQL injection/XSS/SSRF, OWASP security headers, CORS protection, and optional JWT authentication. Test coverage reaches 92% with 180+ security test cases, and runtime metrics are exposed via Prometheus (/metrics endpoint).
Architecture Overview#
Web Browser (Microphone) → WebSocket (Audio Stream) → FastAPI Backend
├─ RealtimeAgent (AutoGen Framework)
├─ Registered Functions (weather / web_search)
└─ OpenAI Realtime API (gpt-realtime)
Deployment#
Docker (Recommended):
docker pull ghcr.io/jedarden/duck-e:latest
docker run -d -p 8000:8000 \
-e OPENAI_API_KEY=your_key \
-e WEATHER_API_KEY=your_key \
ghcr.io/jedarden/duck-e:latest
Local Install: Python 3.9+ environment, run pip install -r requirements.txt then uvicorn app.main:app --reload --host 0.0.0.0 --port 8000.
Typical Use Cases#
- Verbalizing bug symptoms and receiving guided follow-up questions
- Discussing code concepts and architecture design
- Real-time documentation and API syntax retrieval
- Instant supplementary support in team collaboration and new hire onboarding
- Brainstorming partner for solo developers
Unconfirmed Information#
The model names gpt-5, gpt-5-mini, and gpt-realtime mentioned in the README have not been officially confirmed by OpenAI as of the research date, and may be placeholders or future plans, directly impacting the project's actual runnability. The background of author jedarden is not provided in the repository, and no public live demo address has been found.