For about a year and a half, alongside two friends, I built Hirestein, an AI recruiter that screens CVs against a job description, schedules interviews, conducts them by voice in real time, and hands the hiring manager a scored report with a skill matrix. I wrote most of the backend, a good chunk of the frontend, the Helm charts, the CI pipeline, the YC application, the Product Hunt launch copy, and the demo video script. In a three-person company, “full stack” covers a lot of ground.

This post is the technical retrospective I wish I’d read before starting. It covers the architecture that actually shipped, the voice pipeline and its latency budget, the decisions I’d defend and the one I wouldn’t, and, because building the thing is only half the job, what applying to YC and launching on Product Hunt taught me. Including the part where our launch got seven upvotes.

What the product had to do

The pitch: a recruiter posts a job, uploads (or collects via a public link) a pile of CVs, and Hirestein ranks them against the JD. The shortlisted candidates get an interview link. At the scheduled time they click it, land in a browser call, and a voice agent interviews them for fifteen minutes, asking role-specific questions, following up on answers, and cutting off ramblers politely. When it ends, the recruiter gets a report: positive and negative feedback, an overall assessment, and a per-skill score matrix, all traceable back to the transcript.

Pricing was credit-based: one credit bought a fifteen-minute AI interview or ten CV screenings. The underlying costs (LLM tokens, speech-to-text minutes, TTS characters) scale with usage, and a flat seat price on top of usage-shaped costs is a fast route to negative gross margins.

One detail that tells you something about startups: the codebase is full of the string mocksmart, not hirestein. The product started life as MockSmart, a mock-interview practice tool for candidates. Candidates loved the idea and had no money. Recruiters had the money and the exact same technical problem, an agent that can hold a structured interview, so we flipped the direction of the product and kept the plumbing. Renaming a Go module path mid-pivot was nobody’s priority.

The shape of the system

Four deployable services, one Kubernetes cluster on DigitalOcean:

                         ┌───────────────────────────┐
  recruiter ───────────► │  website-frontend (React) │
                         └────────────┬──────────────┘
                                      │ REST
                         ┌────────────▼──────────────┐        ┌──────────┐
                         │   Go REST service (Gin)   │◄──────►│ Postgres │
                         │  account / candidate /    │        └──────────┘
                         │  screener / interview /   │        ┌──────────┐
                         │  report / payment / ...   │◄──────►│  Redis   │
                         └───┬───────────────▲───────┘        └──────────┘
                             │ RabbitMQ      │ S3 (transcripts, CVs)
                             ▼ (topic)       │
                         ┌───────────────────┴───────┐
                         │  Go workers (same image)  │  report generation,
                         │  RMQ consumers + crons    │  screening batches,
                         └───────────────────────────┘  status sweeps
                         ┌───────────────────────────┐
  candidate ───────────► │ interview app (Next.js)   │
                         └────────────┬──────────────┘
                                      │ WebRTC (LiveKit)
                         ┌────────────▼──────────────┐
                         │  AI agent (Python)        │  VAD → STT → LLM → TTS
                         │  LiveKit Agents worker    │
                         └───────────────────────────┘

The Go service is a modular monolith organized hexagonal-style. Each domain (account, candidate, screener, interview, report, notification, payment, analytics) gets its own application / core / infrastructure layers with ports and adapters between them. The workers are the same binary in a different mode, consuming from a RabbitMQ topic exchange and running crons, deployed as a separate Helm release so the API pods never do batch work.

Was full hexagonal architecture overkill for a three-person startup? I went back and forth on this, and landed on: no, and the pivot is the proof. When we flipped from mock interviews to recruiting, the domains that survived (interview, report) kept their core logic untouched while their adapters changed. When the CV screener graduated from a Python prototype to a Go implementation, it slotted in as one more domain without touching the others. The ceremony costs an afternoon per domain; the isolation pays it back the first time the product changes direction, and an early-stage product changes direction a lot.

The voice pipeline is the product

Everything else in the system is a fairly standard SaaS backend. The thing that made Hirestein Hirestein was the fifteen minutes a candidate spends talking to a machine. The honest engineering summary of real-time voice AI: you’re chaining four vendors together in sequence, and every problem shows up as a silence somewhere in the chain.

We built the agent on LiveKit Agents. The candidate’s browser joins a WebRTC room; a Python worker joins the same room as the interviewer. The pipeline for every conversational turn:

AgentSession(
    vad=silero.VAD.load(),          # is anyone speaking?
    stt=deepgram.STT(),             # audio → text, streaming
    llm=openai.LLM.with_azure(),    # text → interviewer's reply
    tts=elevenlabs.TTS(),           # reply → audio
    turn_detection=EnglishModel(),  # has the candidate actually finished?
)

Each line in that constructor is a decision with a story behind it.

Turn detection deserves the first story. Naive voice agents decide “the user finished speaking” by silence: if the VAD sees ~700ms of quiet, respond. Interview candidates destroy this heuristic, because people answering hard questions pause mid-thought. “My biggest weakness is… [900ms of careful phrasing]” and the naive agent barges in with a follow-up question while the candidate is still composing. In an interview, where the candidate is already nervous, getting talked over by a robot ruins the experience. LiveKit’s end-of-utterance model is a small transformer that looks at the transcript, not the audio, and predicts whether the sentence is semantically complete. Layering it on top of VAD cut false interruptions dramatically and was the single biggest jump in “this feels like a real interview” we ever shipped. It only worked for English, which was fine for our market.

The LLM slot took the longest to settle, and the git history shows it. The commented-out lines in the real file tell the story:

#llm=openai.LLM(model="gpt-4o"),
#llm=openai.LLM.with_cerebras(model="llama3.3-70b"),
llm=openai.LLM.with_azure(),
#llm=openai.LLM.with_groq(model="llama-3.3-70b-versatile"), #free #not efficient in closing interview

For voice, time-to-first-token dominates everything, which is why the fast-inference providers (Groq, Cerebras) are so tempting: Llama 3.3 70B on their hardware starts streaming almost instantly and costs approximately nothing. And for the middle of an interview it was genuinely good. The problem was the edges. That inline comment, #free #not efficient in closing interview, is nine words summarizing two weeks of pain: the model would not reliably end interviews. It would keep finding one more follow-up question, drift past the time limit, or close so abruptly it felt like a dropped call. Instruction-following on conversation-management rules, not answer quality, is what pushed us to GPT-4o on Azure. That is not a benchmark anyone publishes.

And then we took interview-ending away from the LLM entirely. The original design gave the model a function tool: call close_interview() when you’re done. It was flaky in the way LLM function calls are flaky, sometimes early, sometimes never. The fix was to stop asking. We disabled function calling, put a countdown timer in the Next.js frontend, and had the frontend call an interview-stop endpoint when time expired, which winds down the room. The agent still says a graceful goodbye; it just doesn’t get a vote on when. I now apply this everywhere I work with agents: let the LLM do the language, and let deterministic code do the state machine. Every hard guarantee we tried to get from the model via prompt (“the interview MUST end at 15 minutes”) was one incident report away from becoming code.

The latency budget for a full turn (candidate stops talking to agent starts talking) has to land around a second and a half, because past ~2s humans start saying “hello? can you hear me?” and past 3s they assume the call is broken. Streaming STT and turn detection eat a few hundred milliseconds, LLM time-to-first-sentence a few hundred more, TTS synthesis of that first sentence the rest. Nothing waits for anything to finish: STT streams into the LLM, the LLM streams into TTS sentence-by-sentence. The room name doubles as the interview ID, so when a worker picks up a job it fetches the interview record from the Go service, and refuses to start if the interview is already COMPLETED or past its expiry_at, because candidates will reopen the link three hours later to see what happens.

Never trust a pod with your transcript

The transcript is the most valuable artifact the system produces. The report is derived from it, and the report is what the customer paid for. The agent pod is also, by a comfortable margin, the most likely thing in the system to die: it’s a Python process doing real-time audio, model inference, and third-party API calls, running on spot-priced nodes.

So the transcript never lives in the pod. Every conversation item, candidate utterance or agent reply, is appended to a Redis list keyed by interview ID, with role and timestamp, the moment it exists:

def _handle_user_speech(self, msg: ChatMessage):
    asyncio.create_task(
        self.transcription_service.log_message(self.room_name, "USER", content)
    )

When the candidate disconnects, the agent drains the Redis list, writes the formatted transcript, uploads it to S3, and only then clears Redis. If the pod dies mid-interview, the transcript up to the last utterance survives in Redis and the partial interview is still reportable. The pattern is just a write-ahead log, and “your process can vanish between any two lines” is the right assumption for anything holding user-generated data it can’t recreate.

From there it’s an event, not a request: an rmq.report.generate message goes onto the RabbitMQ topic exchange, and a Go worker picks it up. The worker pulls the transcript from S3 and makes one LLM call through langchaingo with a “judge” system prompt and, critically, OpenAI’s structured output / JSON response format, unmarshalled straight into a typed struct: skill matrix with scores, positive feedback, negative feedback, overall assessment. Report generation is the one LLM call in the system with no human waiting on the other end, so it runs async with worker-level retry, and a failed generation is a redelivered message instead of a lost report.

Splitting the LLM work this way, Python for the real-time agent where the ecosystem lives and Go for the async report pipeline where our infrastructure lived, looks inconsistent on an architecture diagram and was correct in practice. The judge prompt, incidentally, went through more revisions than any code file in the repo. Getting a model to produce consistent scores across candidates, so that a 7/10 on Tuesday means the same as a 7/10 on Friday, is a calibration exercise that’s never really done. When a Product Hunt commenter asked how transparent the scoring was, the honest answer was: every score links back to transcript evidence, because we didn’t trust it either.

The CV screener: an afternoon in Python, a month in Go

The screener’s history is the prototype-to-production pipeline in miniature.

V0 was a single FastAPI file. Upload a JD and a stack of PDF resumes; for each, extract text with pypdf, then three LLM calls with structured output: parse the JD into a Pydantic model (role, domains, required skills, minimum experience), parse the CV into another (name, skills, experience, roles), then ask gpt-4o-mini to compare the two structured objects and emit a match report: experience match, skill match with gaps, domain match, each with a score and a one-line justification, plus an overall 0–100.

The two-step structure, extracting both sides into typed objects first and then comparing the objects, mattered more than any prompt wording. Asking a model to compare two raw documents in one shot produces an impression with a number attached. The intermediate representation forces it to commit to facts (“this JD requires 4 years”) before scoring against them, and gives you something inspectable when a candidate emails asking why they scored 43. It’s cheap (mini-class model, small contexts) and the structured outputs made bad extractions visible instead of silent.

V1 rewrote it as a screener domain in the Go service, because a demo endpoint and a product feature have different requirements: DOCX support alongside PDF, files pulled from S3 instead of a request body, screening jobs queued and processed by workers, a cron sweeping live screening batches, results persisted relationally (skills and domains normalized into their own tables), credits debited per screening, email notifications when a batch finishes, and an LRU cache in front of the public “submit your CV” form pages so candidate traffic never touches the database. None of that is intellectually interesting. All of it is the actual product. The ratio held everywhere: the AI core of a feature was 20% of the code, and the scaffolding that made it sellable was the other 80%.

Infrastructure for a company of three

Everything ran on a DigitalOcean Kubernetes cluster: the four services as Helm charts we wrote, plus RabbitMQ, Redis, ingress-nginx, cert-manager, and an EFK stack from community charts. Postgres was managed (Supabase), because operating a database was the one infrastructure job I didn’t want to own.

The most useful thing in the whole setup was the deploy convention. One GitLab repo, one CI pipeline, and git tags with a service prefix deciding what ships:

rest-v1.4.2        → build + deploy the Go service and workers
agent-v1.2.0       → the Python agent
interview-v1.1.3   → the interview frontend
agent-rest-v1.5.0  → both
all-v2.0.0         → everything

The CI rules just regex-match the tag. It’s a mono-repo with independent deployability and zero platform team. Tagging a release from my laptop rebuilt exactly the affected images, pushed them, and ran helm upgrade. Deploys took minutes and I never once shipped the frontend because I changed a worker.

Was Kubernetes overkill for our traffic? For the traffic, yes. For the shape of the system, no. We had long-lived stateful-ish agent workers, an API deployment, a worker deployment, and four infrastructure dependencies, and I already knew k8s cold from my day-job years. The real cost calculation for a tiny team isn’t “is this tool too big” but “which tool lets the founders think about it the least”, and for me a values.yaml I could diff beat a pile of hand-configured VMs. If none of us had known Kubernetes, the same reasoning would have ruled it out.

I’ll also confess what didn’t exist: no staging environment (feature flags and careful tag ordering instead), smoke tests over unit-test coverage in the Go service, and a turn_detector.py experiment file that shipped in the agent image for a year because it hurt nothing. Early-stage engineering is deciding which corners are load-bearing. Some of these weren’t. The transcript durability path was, and that one we built properly.

The other half of the job: YC, demos, and launch day

While all of the above was being built, I was also the person filling out the Y Combinator application, recording the demo video, and running the Product Hunt launch. If you’re an engineer thinking about founding something, this section is for you.

The YC application is a systems-design exercise for your company. The questions are short and brutal: what do you make, who wants it, how do you know, why you. Writing “an AI voice interviewer” is easy; defending why recruiters would trust a machine’s judgment with their pipeline in two sentences took more drafts than the report generator’s judge prompt. We didn’t get in. When I went digging through YC’s startup directory for the batches around ours, India-based teams were conspicuously absent, and whatever the official position, the revealed preference in that directory said our geography was a harder filter than our product. I’m not bitter about it (okay, mildly), but it reframed the rejection usefully: it wasn’t feedback on the thing we built. The application was still worth the hours, because it forced us to articulate what we’d been avoiding: our demo was magical and our distribution plan was a shrug. I’d tell any technical founder to write the application even if you never submit it.

Demos expose your latency budget. Recording the demo video taught me more about the product’s rough edges than a month of building it, because a demo has no mercy: every 2.5-second pause the agent took to respond, invisible in my own testing because I knew it was thinking, looked like a bug on video. We tightened the turn-detection thresholds and re-recorded. If you want to find out whether your real-time product is actually good, record yourself using it and watch the recording. The pauses you’ve stopped noticing will be obvious.

Launch day: we got seven upvotes. I’m putting the number in writing because launch-day posts that only report successes have skewed everyone’s expectations. We launched on Product Hunt (“Smart Hiring, Powered by AI”, three makers listed, LiveKit and OpenAI in the built-with) and the internet responded with seven points, one genuinely nice review, and one sharp question about scoring transparency that I answered at length. That question turned out to be worth more than fifty upvotes: it told us exactly which objection every future sales conversation would open with, and the transcript-linked evidence in our reports went from a nice-to-have to the centerpiece of the pitch.

The honest lesson: a Product Hunt launch is not distribution, it’s a photograph of the distribution you already have. We had none. No audience, no waitlist, no launch-day network primed to show up. The product worked; the interviews ran; the reports were good. Nobody was standing in the room where we announced it. If I do this again, the “audience” workstream starts the same week as the git repo, not the same week as the launch.

What I’d keep, and what I’d change

Keep: the deterministic state machine around the LLM; the frontend timer ending interviews instead of a function call was the single best reliability decision we made. The Redis write-ahead transcript. Structured outputs on every LLM boundary, with typed models on both sides. The tag-based deploy convention. The two-step extract-then-compare screener design. Hexagonal domains, which turned a pivot into a refactor instead of a rewrite.

Change: I’d spend the Groq/Cerebras evaluation weeks earlier and more systematically. We discovered “fails to close interviews” in production-shaped testing when a scripted conversation-management eval would have caught it in a day. I’d build the multilingual story sooner; English-only turn detection capped our market. And I’d trade one of the four services’ worth of polish for an audience, because the hardest scaling problem we hit was not in the cluster.

Building Hirestein compressed about five years of ordinary engineering lessons into eighteen months: real-time systems, LLM reliability, event-driven pipelines, Kubernetes on a shoestring, and the humbling discovery that the market does not attend your launch just because the latency budget closes. Seven upvotes, one great question, and a system I’m still proud of. I’d do it again, and the next post will be about whatever “again” turns out to be.