PROJECT STUDY
Call Quality AI
Check every call, and hear the evidence behind each flag.
A QA workspace for Indonesian contact centres that redacts personal data, scores every recorded call on a rubric calibrated against human QA and links each flag to its moment in the audio.
Explore the build
At a glance
- Industry
- Financial services
- Capabilities
- 8
- Technologies
- 10
- Year
- 2026
The challenge
A contact centre records every call, but its QA team can listen to only a small sample.
Required disclosures, misleading promises, greetings, empathy and script adherence go unchecked on most calls, and a transcript that does exist can leak a customer's personal data.
Call Quality AI is built for the contact centres of banks, insurers, multifinance companies and telcos in Indonesia, and for the outsourced centres that serve them. It transcribes and redacts each recording, scores it against the team's own rubric and takes a reviewer straight to the moment behind every flag.
What we built
Call Quality AI joins a Python speech pipeline to a reviewer workspace.
Speakers from the recording
Stereo WAV and MP3 recordings are split by channel into agent and customer, the way contact-centre recorders keep the two legs, so no diarisation model is needed. Mono files are segmented by an energy-based voice activity detector and marked as lower confidence
Indonesian transcription
An adapter around faster-whisper, which runs on CTranslate2, transcribes the speech in Indonesian with segment timestamps
Redaction before storage
NIK, card numbers checked with the Luhn algorithm, phone and account numbers and OTP codes spoken in Indonesian are masked in the transcript, and the matching audio spans are replaced with a tone before anything is stored
A rubric engine in YAML
Required phrases with Indonesian normalisation and synonyms, forbidden phrases, and order and timing rules, such as a greeting within the first 15 seconds and a disclosure before the closing
Conversation measures
Dead air, talk ratio, interruptions found in overlapping segments, and a lexicon-based Indonesian sentiment trajectory across the call
Evidence-quoting judgement items
Optional items, such as empathy or whether the issue was resolved, run through an LLM adapter and must quote their evidence with timestamps
Calibration against human QA
Automatic scores are compared with human QA scores on a labelled sample of calls, and agreement is reported as Cohen's kappa
A reviewer workspace
A call list with scores, a call view with speaker turns and timestamped flags, an audio player that jumps to the evidence, and reviewer overrides kept in an audit trail
Speakers from the recording
Stereo WAV and MP3 recordings are split by channel into agent and customer, the way contact-centre recorders keep the two legs, so no diarisation model is needed. Mono files are segmented by an energy-based voice activity detector and marked as lower confidence
Indonesian transcription
An adapter around faster-whisper, which runs on CTranslate2, transcribes the speech in Indonesian with segment timestamps
Redaction before storage
NIK, card numbers checked with the Luhn algorithm, phone and account numbers and OTP codes spoken in Indonesian are masked in the transcript, and the matching audio spans are replaced with a tone before anything is stored
A rubric engine in YAML
Required phrases with Indonesian normalisation and synonyms, forbidden phrases, and order and timing rules, such as a greeting within the first 15 seconds and a disclosure before the closing
Conversation measures
Dead air, talk ratio, interruptions found in overlapping segments, and a lexicon-based Indonesian sentiment trajectory across the call
Evidence-quoting judgement items
Optional items, such as empathy or whether the issue was resolved, run through an LLM adapter and must quote their evidence with timestamps
Calibration against human QA
Automatic scores are compared with human QA scores on a labelled sample of calls, and agreement is reported as Cohen's kappa
A reviewer workspace
A call list with scores, a call view with speaker turns and timestamped flags, an audio player that jumps to the evidence, and reviewer overrides kept in an audit trail
How it works
Split and transcribe
The ingest stage reads each recording, splits stereo files into the agent and customer channels and segments mono files with the energy-based detector. The speech recognition adapter transcribes the speech with faster-whisper and keeps the segment timestamps.
Redact before storing
The redactor finds NIK, card, phone, account and OTP numbers in the transcript, masks them, and replaces the matching time spans in the audio with a tone. It runs before anything is written, so the stored transcript and audio are the redacted versions.
Score against the rubric
The rubric engine loads the team's YAML rubric and checks phrases, order and timing, dead air, talk ratio, interruptions and the sentiment trajectory, plus any LLM-assisted items. Every flag keeps the timestamp of its evidence.
Review and calibrate
A FastAPI service backed by SQLite serves the React and TypeScript workspace, where reviewers play the audio from the evidence and override a score with the change recorded in an audit trail. Calibration compares automatic and human scores and reports their agreement as Cohen's kappa.
Engineering insights
The pipeline is built around evidence: every score points to a moment in the call, and personal data is redacted before anything is kept.
Stack by layer
- Frontend
- React
- TypeScript
- Vite
- Backend
- Python
- FastAPI
- Pydantic
- Data
- SQLite
- YAML rubrics
- AI
- faster-whisper
- CTranslate2
- Energy-based VAD
- Lexicon-based sentiment
- LLM judgement adapter
- Cohen's kappa
- Infrastructure
- uv
- pytest
- Ruff
- Biome
Speakers from channels, not a model
Contact-centre recorders usually keep the agent and customer legs on separate channels, so the speaker of every line comes from its channel rather than from a diarisation model. Talk ratio comes from each channel's speech time and interruptions from segments that overlap across the two channels, while mono files are marked as lower confidence
Redaction in text and in audio
A number read out on a call exists twice, in the transcript and in the recording, so the redactor masks the text and overwrites the matching audio span with a tone. Both happen before storage, so a detected NIK, card, phone, account or OTP number never reaches the database or the stored audio
Rubrics that read Indonesian
Required phrases match after Indonesian normalisation and declared synonyms, so informal and variant spellings still count, while forbidden phrases catch promises an agent must not make. Order and timing rules run on the call's timeline, such as a greeting within the first 15 seconds or a disclosure before the closing
Machine judgement with quoted evidence
LLM-assisted items must quote the transcript and give the timestamp of their evidence, so a reviewer can check every machine verdict against the call itself. The adapter keeps the model provider replaceable
Agreement measured as kappa
Calibration compares automatic scores with human QA scores on the same labelled calls and reports Cohen's kappa rather than raw percentage agreement, because kappa discounts the agreement two raters would reach by chance. The team sees how closely the engine agrees with its own reviewers before relying on its scores
A fake speech recogniser fed by fixture transcripts and a fake LLM adapter run the whole test suite offline, without a GPU, a model download or an API key. A script downloads a small faster-whisper model for real transcription.
Design system
QA studio is a calm, readable interface for reviewers who read transcripts all day, in which each colour keeps one meaning.
Colour palette
- Slate white
#f8fafcPage and transcript background - Ink
#0f172aTranscript text, labels and timestamps - Teal
#0e7490Primary buttons, links and focus - Agent blue
#2563ebThe agent's turns, shown with a speaker label - Customer magenta
#be185dThe customer's turns, always labelled and never shown by colour alone - Pass green
#15803dPassed rubric items - Flag red
#b91c1cFailed items and compliance flags - Attention amber
#b45309Items that need a reviewer's attention
Colour
Slate-white surfaces and near-black ink keep long transcripts easy on the eyes, teal is kept for controls, agent blue and customer magenta mark who spoke, and green, red and amber mean only pass, flag and attention
Typography
Inter throughout, with transcript lines held to about 70 characters and timestamps set in tabular numerals, so columns align and digits do not jump during playback
Speakers
Every turn carries a speaker label as well as its colour, so telling the agent from the customer never depends on hue alone
Evidence
Each flag links to its timestamp and the audio player jumps to the evidence, so a reviewer hears the moment before accepting or overriding a score
Where this fits
Contact centres in banking, insurance, multifinance and telecoms can apply the same pipeline to their own recordings and rubrics. Explore our AI solutions, business systems and data dashboards, see our insurance work, or talk to us about reviewing every call.
KEEP EXPLORING
Another problem. A different approach.
Store & Workforce Vision
A privacy-aware store operations platform for occupancy, queue and zone events, with configurable alerts, manager review and aggregate reporting.
Mining Haul Dispatch
Haul dispatch for open-pit mines: a linear program plans haul flows inside the coal-quality band, and each truck goes to the excavator furthest behind plan, with the reason logged.
Let’s connect the dots
Let's put these ideas to work.
Share a process, product, or problem you want to improve. We will work through the possibilities together.