Voice Input in the Browser: The Gotchas Nobody Warns You About

Why Voice Is the Part Tutorials Skip

An interview you type isn't really interview practice. The whole point of this app is answering out loud, so this part adds browser voice input with speech recognition — the feature that makes it real, and the one most tutorials quietly avoid. In part 7 the pipeline generated questions; now the candidate speaks their answers.

The browser gives us two APIs for this: MediaRecorder to capture the actual audio, and the Web Speech API (SpeechRecognition) to transcribe it live. Each is simple on its own. Running them together, reliably, on real devices, is where the sharp edges are — and this part is mostly about those edges, because they're the difference between "works in the demo" and "works for real users."

Capturing Audio and Speech Together

The goal is to keep both a real audio recording (for later, more accurate transcription) and a live transcript (for instant on-screen feedback). We start the microphone first, then speech recognition:

async function startAnswerCapture() {
  let micOk = false;
  try {
    const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
    mediaRecorder = new MediaRecorder(stream, { mimeType: 'audio/webm' });
    chunks = [];
    mediaRecorder.ondataavailable = (e) => chunks.push(e.data);
    mediaRecorder.start();
    micOk = true;
  } catch (err) {
    mediaRecorder = null; // no mic — we'll fall back to transcript only
  }

  const SR = window.SpeechRecognition || window.webkitSpeechRecognition;
  if (SR) {
    recognition = new SR();
    recognition.lang = 'en-US';
    recognition.continuous = true;
    recognition.interimResults = true;
    // ... handlers below
    recognition.start();
  }
  return micOk;
}

That ordering is not cosmetic. Acquire the microphone before starting recognition. If speech recognition is already running when you call getUserMedia, Chrome aborts the recognition session — and it does so silently, leaving you with an empty transcript and no error. Settling the mic permission first avoids that entirely. This is gotcha number one, and it's invisible until it bites you in production.

Gotcha: Interim Results Aren't Optional

It's tempting to only keep "final" transcript results and ignore interim ones — final sounds like the clean, done version. That's a trap. Chrome only finalizes a result after it detects a silence gap. If a candidate talks right up until they hit submit, with no trailing pause, that last stretch never finalizes — and the answer arrives completely blank.

recognition.onresult = (e) => {
  interimTranscript = '';
  for (let i = e.resultIndex; i < e.results.length; i++) {
    if (e.results[i].isFinal) {
      finalTranscript += e.results[i][0].transcript + ' ';
    } else {
      interimTranscript += e.results[i][0].transcript + ' ';  // keep these!
    }
  }
};

So we track interim text separately and never throw it away — it's shown live, and folded into the final transcript when the session ends. The fix is small; finding out you need it usually means first losing a bunch of real answers to blank submissions.

Gotcha: Recognition Stops Itself

Even in continuous mode, SpeechRecognition ends its own session after a stretch of silence. A candidate who pauses to think mid-answer would have recognition quietly die on them. So we restart it — but only while the answer is still active — and we rescue any interim words from the session that just ended, since they'll never get a chance to finalize:

recognition.onend = () => {
  if (interimTranscript.trim()) {
    finalTranscript += interimTranscript.trim() + ' ';
    interimTranscript = '';
  }
  if (recognitionActive) {
    try { recognition.start(); } catch (err) { /* already restarting */ }
  } else if (onRecognitionEnd) {
    onRecognitionEnd();
  }
};

The recognitionActive flag is the guard: while the answer is live we auto-restart; once we've deliberately stopped, we let it end and signal whoever is waiting.

Gotcha: Stopping Is a Race Condition

The nastiest edge is at the finish line. When you call recognition.stop(), its last results don't arrive synchronously — they fire after stop, a moment later. Read the transcript immediately and you lose the final words of every answer. So stopping has to wait for the session to actually end, with a bounded timeout so a browser that never fires onend can't hang the app:

function stopRecognitionAndWait() {
  return new Promise((resolve) => {
    recognitionActive = false;
    const rec = recognition;
    if (!rec) { resolve(); return; }
    let settled = false;
    const finish = () => {
      if (settled) return;
      settled = true;
      clearTimeout(fallback);
      rec.onresult = rec.onend = rec.onerror = null; // stop late events leaking
      recognition = null;
      resolve();
    };
    const fallback = setTimeout(finish, 2000); // never hang forever
    onRecognitionEnd = finish;
    try { rec.stop(); } catch (err) { finish(); }
  });
}

Two defensive details make this safe: the 2-second fallback guarantees we always move on even if the browser misbehaves, and detaching the handlers stops a late-firing event from one question leaking into the next. Only after this resolves do we stop the recorder and read the final transcript. Timing, not the API, is the hard part.

Sending the Answer to Django

With both the audio blob and the transcript in hand, we send them together. The audio is dropped if it's missing or over the backend's 5MB cap — the transcript alone still carries the answer:

const formData = new FormData();
if (capturedBlob && capturedBlob.size > 0 && capturedBlob.size <= 5_000_000) {
  formData.append('audio', capturedBlob, 'answer.webm');
}
formData.append('transcript', (finalTranscript + ' ' + interimTranscript).trim());
formData.append('question_id', q.id);
await interviewAPI.submitAnswer(interviewId, formData);

On the server, the view validates the audio by its real content — not its filename, which is always answer.webm regardless of the true codec — then stores the recording and transcript on the question:

class SubmitAnswerAPIView(APIView):
    parser_classes = [MultiPartParser]

    def post(self, request, *args, **kwargs):
        audio = request.FILES.get('audio')
        transcript = request.data.get('transcript')

        if audio and audio.size > 5_000_000:
            return Response({'err': 'Max file size is 5MB'}, status=400)

        if audio and not is_valid_audio_file(audio):
            return Response({'err': 'Audio format not supported!'}, status=400)

        question = InterviewQuestion.objects.get(
            id=request.data.get('question_id'), interview=interview,
        )
        if question.status == 2:            # already answered — ignore duplicates
            return Response({'msg': 'success'}, status=200)

        question.record = audio
        question.user_answer = transcript
        question.status = 2                 # completed
        question.save(update_fields=['record', 'user_answer', 'status', 'updated_at'])
        return Response(status=201)

Notice the question.status == 2 check: the timer can auto-submit at the same moment the user clicks, so the endpoint has to be idempotent — a second submission for an answered question is accepted quietly rather than double-processed. Keeping the raw audio around is deliberate too: browser transcription is convenient but imperfect, so storing the recording leaves the door open to re-transcribe it later with something more accurate.

What This Gets You

With voice in place, the app finally does what it was built for: it reads each question aloud, listens to a spoken answer, shows a live transcript, and stores both the recording and the text. That's real interview practice, not a typing exercise.

Almost none of the work here was the "happy path" — it was the timing: acquiring the mic in the right order, keeping interim results, restarting on silence, and waiting out the stop race. That's genuinely the hard part of browser voice, and it's why so many demos work once and fail for real users. If you'd rather start from the finished, production-tested version of all of this, it's part of the full source in the starter kit. Otherwise, part 9 is next, where the AI closes the loop: scoring each answer and turning the whole session into a final interview report.