Voice Memo & Dictation

A dual-gesture microphone button that taps for live speech-to-text dictation and holds for audio memo recording. All processing is on-device — SFSpeechRecognizer with requiresOnDeviceRecognition, AVAudioRecorder saving to a local Documents folder.

Preview

Meridian · Backend Enginterviewing

Tap: dictate

Tap the mic to demo live dictation · click "Demo: hold to record" to add a memo entry

Anatomy

The pattern has three distinct layers that work in parallel.

LayerGestureServiceOutput
Live DictationTap mic buttonVoiceDictationService (SFSpeechRecognizer)Streaming transcript → appended to notes draft
Audio MemoLong press (0.5s)AudioRecorderService (AVAudioRecorder)UUID.m4a saved to Documents/AudioMemos/
PlaybackTap memo rowAVAudioPlayerIn-app audio playback with scrubber

Design notes

Single button, two modesAvoid separate buttons for 'dictate' and 'record'. A single floating mic with dual gesture keeps the UI uncluttered. The tap/hold distinction is learnable after one use.
Visual state is essentialDictation mode: animated waveform bars (5–9 bars, heights randomised every ~100ms). Recording mode: red fill + pulsing ring at 0.9s ease-out repeat. Idle: accent fill, static mic icon. Never leave the user uncertain about which mode is active.
On-device only — declare itSet requiresOnDeviceRecognition = true on SFSpeechAudioBufferRecognitionRequest. This prevents the request from being routed to Apple's servers. Declare NSSpeechRecognitionUsageDescription in Info.plist stating that recognition happens on-device.
File naming is idempotentSave audio files as UUID.m4a relative to Documents/AudioMemos/. Reconstruct the full path at playback time so the file survives app reinstall of the database while the Documents folder persists (for TestFlight / device backup).
Transcription is asyncApply SFSpeechRecognizer to the finished .m4a after AVAudioRecorder stops, not during recording. This avoids contention between the audio engine input node and the speech recogniser. Store the transcript in the AudioMemo SwiftData model once available.

Implementation

VoiceMemoList.tsx
tsx
// Web reference implementation using MediaRecorder API
"use client";

import { useState, useRef, useEffect } from "react";

interface AudioMemo {
  id: string;
  blob: Blob;
  duration: number;   // seconds
  transcript?: string;
  createdAt: Date;
}

export function VoiceMemoList() {
  const [memos, setMemos]       = useState<AudioMemo[]>([]);
  const [recording, setRecording] = useState(false);
  const [elapsed, setElapsed]   = useState(0);
  const mediaRef = useRef<MediaRecorder | null>(null);
  const chunksRef = useRef<BlobPart[]>([]);
  const timerRef  = useRef<ReturnType<typeof setInterval> | null>(null);

  const start = async () => {
    const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
    const mr = new MediaRecorder(stream);
    chunksRef.current = [];
    mr.ondataavailable = e => chunksRef.current.push(e.data);
    mr.onstop = () => {
      const blob = new Blob(chunksRef.current, { type: "audio/webm" });
      setMemos(prev => [
        { id: crypto.randomUUID(), blob, duration: elapsed, createdAt: new Date() },
        ...prev,
      ]);
      stream.getTracks().forEach(t => t.stop());
    };
    mr.start();
    mediaRef.current = mr;
    setElapsed(0);
    setRecording(true);
    timerRef.current = setInterval(() => setElapsed(e => e + 1), 1000);
  };

  const stop = () => {
    mediaRef.current?.stop();
    if (timerRef.current) clearInterval(timerRef.current);
    setRecording(false);
  };

  const fmt = (s: number) =>
    `${Math.floor(s / 60)}:${String(s % 60).padStart(2, "0")}`;

  return (
    <div className="space-y-3">
      <button
        onClick={recording ? stop : start}
        className={`px-4 py-2 rounded-full text-sm font-medium transition-all
          ${recording
            ? "bg-red-500 text-white"
            : "bg-[rgb(var(--accent))] text-white"}`}
      >
        {recording ? `Stop · ${fmt(elapsed)}` : "Record Memo"}
      </button>

      {memos.map(m => (
        <div key={m.id} className="p-3 rounded-xl bg-[rgb(var(--surface-raised))] border
                                   border-[rgb(var(--border))] space-y-1">
          <div className="flex justify-between text-[11px] text-[rgb(var(--text-tertiary))]">
            <span>{m.createdAt.toLocaleTimeString()}</span>
            <span className="font-mono">{fmt(m.duration)}</span>
          </div>
          <audio controls src={URL.createObjectURL(m.blob)} className="w-full h-7" />
        </div>
      ))}
    </div>
  );
}

Privacy & entitlements

Key / EntitlementPlatformNotes
NSMicrophoneUsageDescriptioniOS + macOSRequired for AVAudioSession. State on-device only.
NSSpeechRecognitionUsageDescriptioniOS + macOSRequired for SFSpeechRecognizer. State on-device only.
com.apple.security.device.audio-inputmacOS entitlementHardened runtime entitlement for mic access on macOS.

Accessibility

  • The mic button must declare accessibilityLabel that changes with mode: 'Start dictation', 'Stop dictation', 'Recording in progress — tap to stop'.
  • The pulsing animation ring should carry accessibilityHidden(true) — it is decorative.
  • Audio memos should expose accessibilityLabel with creation time and transcription preview: 'Voice memo from 2:41 PM, 47 seconds. Need to follow up on system design question.'
  • Ensure AVAudioSession category .playAndRecord does not interrupt VoiceOver. Set .mixWithOthers option or pause session when VoiceOver is active.