Skip to the case study
Case study01 · OverviewAscendia AI · 2024

From job board to a scored interview report.

An AI recruiting platform that finds and ranks candidates, phones them, schedules interviews, conducts the interview live over voice, and hands recruiters a scored assessment report. I designed and built the AI layer end to end: candidate matching, outreach calling, the real-time voice interviewer, and post-interview analysis.

IndustryHR technology & recruiting
RoleAI Engineer
EmployerAscendia AI

Under 500 ms

End-to-end voice latency, with streaming at every stage

On-method interviews

RAG over the candidate CV and the company's interview rulebook

Comparable scores

Every interview ends in a score and an assessment report

Case study02 · Solution & capabilities2 / 4
02
The solution

Review only candidates who are already interviewed.

Staffing teams screened candidates by hand. Keyword matching on unstructured job descriptions gave weak shortlists, calls and scheduling ate recruiter time, and first-round interviews were slow to run and hard to compare. The goal: a recruiter reviews only candidates who have been matched, contacted, interviewed and scored.

1Matched and contactedRanked candidates get a call, then an SMS if there is no answer
2Interviewed liveA voice agent runs the interview in the company's own method
3Scored and deliveredScore and report reach the recruiter and sync to the ATS

Key capabilities

What I built

Matching engine

Fine-tuned LED-base-16384 summarizer feeding hybrid retrieval on Pinecone.

  • 8-bit quantized, served on ONNX Runtime, CPU only
  • LLM re-ranking scores each match

Custom CV extraction

A fine-tuned image-based model reads candidate CVs.

  • Built when multimodal LLMs were not yet practical
  • Feeds structured text into matching

Outreach and scheduling

A Twilio calling agent reaches candidates without recruiter time.

  • Two-way SMS follow-up when nobody answers
  • LangGraph workflow books the interview slot

Real-time voice interviewer

The full loop, built end to end on LiveKit.

  • Audio transport, VAD tuning, streaming STT, LLM and TTS
  • Barge-in handling and RAG through tool calling

Interview knowledge layer

ChromaDB holds the candidate CV and the company's interview rulebook.

  • Questions follow how that company interviews
  • Node.js ingestion chunks, embeds and loads documents

Analysis layer

A pipeline that turns a full session into a score and a report.

  • Expression, engagement, sentiment and tone
  • AI-assistance signals, skills and role fit
Case study03 · Architecture & build3 / 4
03
Architecture

Five stages, each owned by a separate service.

Python · Node.js · FastAPI · LiveKit · LangGraph · Pinecone · ChromaDB · OpenAI · Twilio · AWS Lambda, S3, DynamoDB

Architecture diagram in five stages. Candidate matching: crawlers on AWS Lambda pull postings from 5+ job boards into S3 and DynamoDB, an LED summarizer and embeddings in Pinecone feed LLM re-ranking and an ATS shortlist. Outreach and scheduling: an outbound call agent on Twilio, SMS follow-up and a LangGraph scheduling workflow. Live AI voice interview: the candidate connects through LiveKit over WebRTC, Silero VAD, streaming STT, GPT-4o-mini reasoning with tool calling against ChromaDB, and streaming TTS, with a barge-in path. Post-interview intelligence: session capture, an analysis service, scoring signals, a compatibility score and an assessment report. Delivery: notifications, human review and an ATS sync.

Open the diagram full size ↗

How it was built

Four services, one pipeline
1MatchCrawl, summarize, rank2ReachCalls, SMS, scheduling3InterviewReal-time voice loop4ReportScore and ATS sync

Challenges

Problem, then fix
Latency broke the conversationEvery stage streams (STT, LLM, TTS overlap) and WebRTC carries small audio chunks, keeping end-to-end latency under 500 ms
Interruptions: a twitchy VAD cut candidates off, a lax one felt sluggishTuned Silero thresholds and silence windows; on a real barge-in the agent stops, flushes the audio buffer and keeps the partial transcript
A bare LLM ran generic interviewsRAG over the rulebook and CV, plus LangGraph state, keeps questions on-method and follow-ups context-aware
CPU inference was too slow and costly8-bit quantization and ONNX Runtime cut VRAM needs by about 70% and allowed CPU-only serving
Millions of postings in one pipelineA microservice split lets scraping and scoring scale independently
Model churnMoved from Whisper STT and ElevenLabs TTS to OpenAI Realtime for faster inference, without rewriting the interview flow
Case study04 · Results & impact4 / 4
04
Results

What it delivered.

<500msend-to-end voice latency
3xfaster job sourcing, 85%+ match accuracy
20K+jobs processed per month from 5+ boards
80%faster candidate response via phone and SMS outreach

Impact

Who it changed things for
For recruiters

A short, scored list

Recruiters review a shortlist instead of a pile of CVs, for example 10 candidates out of 100, each with a comparable score and report.

For candidates

Contacted without waiting

Candidates are called, scheduled and interviewed without recruiter time, and every interview ends in a report instead of loose notes.

For scale

Ready for more volume

The services scale separately, so more volume needs no redesign.

From job board to a scored report: one pipeline that matches, calls, interviews and scores, built end to end.

Muhammad Abdullah · AI Engineer, Ascendia AIStart a similar project