Call intelligence pipeline

Teams that work by phone — support, sales, clinics, real-estate — generate hours of audio that nobody listens to again. A plain transcript is barely better: it is long and unsearchable by meaning. What is useful is a short structured record per call that can be filtered and searched. This design is a provider-hosted-telephony pipeline that produces exactly that, using synthetic recordings as test data.

• Transcription takes longer than a webhook timeout allows, so the work cannot happen in the request.
• Free-text LLM summaries cannot be filtered, counted or trusted to have the same shape twice.
• Transcription is the expensive step; improving the extraction prompt should not mean paying for it again.

When a call recording is ready, Twilio sends a status callback; the endpoint verifies the signature, stores the event and enqueues a job in BullMQ. The pipeline runs as three independent stages, each persisting its output in PostgreSQL: (1) fetch the recording and transcribe it with Whisper, keeping timestamps per segment; (2) send the transcript to an LLM with a JSON schema — topic, customer intent, objections raised, commitments made, next step, sentiment — and reject any output that fails validation; (3) index the transcript with full-text search. Because each stage stores its result, changing the extraction schema means re-running stage 2 over stored transcripts without touching audio. A React view lists calls with their structured fields, filters by any of them, and jumps from a search hit to the matching moment in the transcript.

The weak point is speaker separation: without it, 'who committed to what' is a guess by the LLM. The next step is dual-channel recordings so each side is transcribed separately, then measuring word error rate and extraction accuracy on a labelled set of synthetic calls.