Superhuman

Overview

Superhuman is a native Linux desktop app that acts as a silent co-pilot during video calls and meetings. It listens to your meeting audio in real-time, transcribes everything locally, and uses Claude to detect incoming questions and generate concise answers — so you can respond confidently without scrambling through notes.

The app sits in the system tray and surfaces as a compact overlay panel, invisible to screen sharing.

How It Works

Meeting audio (PulseAudio monitor)
        ↓
  WebRTC VAD — utterance boundary detection
        ↓
  faster-whisper — local Whisper transcription
        ↓
  Claude API — question detection + answer generation
        ↓
  WebSocket → Tauri frontend overlay

The Python sidecar handles all audio and AI work. Tauri invokes it as a subprocess and communicates over a local WebSocket, keeping the Rust/frontend layer thin.

Audio Capture

The app targets PulseAudio monitor sources — the loopback sinks that capture system output — so it hears exactly what you hear: the other person's voice through your speakers or headphones. It auto-detects which monitor is attached to your active browser session (e.g., Firefox's audio sink) and pre-selects it.

WebRTC VAD segments the continuous audio stream into utterances. Only complete utterances are sent to Whisper, cutting transcription cost and keeping latency low.

Local Transcription

faster-whisper runs entirely on-device — no audio ever leaves your machine. The Whisper model loads once at session start and processes utterances off the async event loop in a dedicated thread, so the WebSocket stays responsive.

AI Question Detection & Answering

Each transcript is sent to Claude with a tightly constrained system prompt:

• If the utterance is a question, Claude returns a concise 1–3 sentence answer.
• If it's a statement, small talk, or filler, Claude returns SKIP and nothing is shown.

This keeps the UI quiet and focused — you only see responses when they're actually needed.

Meeting Context

Before starting a session, you can paste in meeting notes, an agenda, attendee bios, or any background Claude should know. This context is injected into Claude's system prompt, making answers specific to your meeting rather than generic. The context is persisted in localStorage so it's ready for the next call.

Architecture

Frontend (Tauri + React + TypeScript)

• Compact overlay panel with live status indicator (Disconnected / Listening / Transcribing / Answering)
• Live transcript preview during transcription
• Latest answer card with question echo
• Session history grouped by session with timestamps
• Audio device picker with auto-detection of browser monitor sinks

Backend (Python sidecar)

• audio_capture.py — PulseAudio PCM capture at 16kHz
• transcriber.py — WebRTC VAD + faster-whisper async pipeline
• assistant.py — Anthropic SDK integration
• websocket_server.py — local WebSocket bridge to Tauri
• main.py — orchestrator

Tauri (Rust)

• Spawns/kills the Python sidecar via start_sidecar / stop_sidecar commands
• Detects Wayland vs X11 (global shortcut support differs)
• System tray integration for quick show/hide

Features

• Real-time local transcription with faster-whisper (no cloud)
• Claude-powered question detection — skips statements and filler automatically
• Meeting context injection for domain-specific answers
• Auto-detects browser audio monitor for zero-configuration setup
• Session history with multi-session grouping
• System tray + global keyboard shortcut (Ctrl+Shift+Space on X11)
• Wayland compatible (tray-based workflow)