Most writing about AI streaming interfaces is about the transport: SSE versus WebSockets, backpressure, reconnection. All real concerns, and none of them are why one streaming UI feels alive and another feels like a dot-matrix printer. The difference is in decisions that happen after the token arrives.
This is the demo I keep coming back to when I think about it:
AI stream
'use client'
import { useState, useEffect, useRef, useCallback, useSyncExternalStore } from 'react'
const PROMPT = 'Explain the tradeoffs of micro-frontend architecture'
const RESPONSE_TOKENS = (
'Micro-frontends let teams own and deploy UI slices independently, ' +
'which reduces coordination overhead and enables polyglot tech stacks. ' +
'The cost is real: every boundary adds a network round-trip, a separate JavaScript bundle, ' +
'and a shared-state contract that can drift. ' +
'Consistent design systems become hard to enforce when each team ships its own component library. ' +
'The pattern pays off at scale where deployment independence outweighs the integration tax. ' +
'Below that threshold, a well-structured monorepo is almost always simpler.'
).split(/(?<=\s)|(?=\s)/).filter(Boolean)
const FULL_RESPONSE = RESPONSE_TOKENS.join('')
const REDUCED_MOTION_QUERY = '(prefers-reduced-motion: reduce)'
function subscribeToMotionPreference(onStoreChange: () => void): () => void {
const media = window.matchMedia(REDUCED_MOTION_QUERY)
media.addEventListener('change', onStoreChange)
return () => media.removeEventListener('change', onStoreChange)
}
function usePrefersReducedMotion(): boolean {
return useSyncExternalStore(
subscribeToMotionPreference,
() => window.matchMedia(REDUCED_MOTION_QUERY).matches,
() => false,
)
}
function jitter(token: string): number {
const base = 30 + Math.random() * 50
const isPunctuation = /[.,!?;:]$/.test(token.trim())
return base + (isPunctuation ? 150 + Math.random() * 70 : 0)
}
type Status = 'idle' | 'streaming' | 'done'
export default function AiStreamComponent() {
const [status, setStatus] = useState<Status>('idle')
const [displayed, setDisplayed] = useState('')
const reducedMotion = usePrefersReducedMotion()
const outputRef = useRef<HTMLDivElement>(null)
const timeoutRef = useRef<ReturnType<typeof setTimeout> | null>(null)
const indexRef = useRef(0)
// Use a ref to hold the recursive scheduler so useCallback deps stay stable.
const schedulerRef = useRef<(() => void) | null>(null)
const clearTimers = useCallback(() => {
if (timeoutRef.current) clearTimeout(timeoutRef.current)
}, [])
// Build the scheduler and store in ref to avoid circular useCallback deps.
useEffect(() => {
schedulerRef.current = () => {
const i = indexRef.current
const token = RESPONSE_TOKENS[i]
if (i >= RESPONSE_TOKENS.length || token === undefined) {
setStatus('done')
return
}
timeoutRef.current = setTimeout(() => {
setDisplayed((prev) => prev + token)
indexRef.current = i + 1
schedulerRef.current?.()
}, jitter(token))
}
}, [])
const handleStream = useCallback(() => {
if (status === 'streaming') return
clearTimers()
if (reducedMotion) {
indexRef.current = RESPONSE_TOKENS.length
setDisplayed(FULL_RESPONSE)
setStatus('done')
return
}
setDisplayed('')
indexRef.current = 0
setStatus('streaming')
schedulerRef.current?.()
}, [status, clearTimers, reducedMotion])
const handleReset = useCallback(() => {
clearTimers()
setDisplayed('')
indexRef.current = 0
setStatus('idle')
}, [clearTimers])
useEffect(() => {
if (outputRef.current) {
outputRef.current.scrollTop = outputRef.current.scrollHeight
}
}, [displayed])
useEffect(() => () => clearTimers(), [clearTimers])
return (
<div className="flex w-lg max-w-full flex-col overflow-hidden rounded-(--radius-card) border border-(--color-border) bg-(--color-surface) font-mono text-sm">
<style>{`
@keyframes ai-stream-blink {
0%, 100% { opacity: 1; }
50% { opacity: 0; }
}
.ai-stream-cursor { animation: ai-stream-blink 1060ms steps(1) infinite; }
@media (prefers-reduced-motion: reduce) {
.ai-stream-cursor { animation: none; }
}
`}</style>
<div className="flex items-center gap-2.5 px-5 py-4">
<span className="select-none text-(--color-fg-subtle)">{'>'}</span>
<span className="flex-1 truncate text-(--color-fg-muted)">{PROMPT}</span>
{status === 'idle' && (
<button
type="button"
onClick={handleStream}
className="shrink-0 rounded-md border border-(--color-border-strong) bg-(--color-surface-hover) px-3 py-1 text-xs text-(--color-fg) transition-colors hover:border-(--color-accent)"
>
Send
</button>
)}
</div>
{status !== 'idle' && (
<div
ref={outputRef}
aria-live="polite"
// While streaming, aria-busy suppresses per-token announcements (~every
// 50ms = spam); flipping to false on 'done' announces the finished
// response once as a single polite update.
aria-busy={status === 'streaming'}
aria-label="AI response"
className="scrollbar-thin max-h-48 overflow-y-auto border-t border-(--color-border) bg-(--color-bg) px-5 py-4 leading-relaxed text-(--color-fg)"
>
<span>{displayed}</span>
{status === 'streaming' && (
<span aria-hidden="true" className="ai-stream-cursor text-(--color-accent)">
|
</span>
)}
</div>
)}
<div className="flex items-center justify-between gap-3 border-t border-(--color-border) px-5 py-3">
<span className="text-[10px] tracking-[0.14em] text-(--color-fg-subtle) uppercase">
{status === 'idle' && 'Ready'}
{status === 'streaming' && 'Streaming\u2026'}
{status === 'done' && `${RESPONSE_TOKENS.length} tokens`}
</span>
{status === 'done' && (
<button
type="button"
onClick={handleReset}
className="rounded-md border border-(--color-border) px-3 py-1 text-xs text-(--color-fg-muted) transition-colors hover:text-(--color-fg)"
>
Reset
</button>
)}
</div>
</div>
)
}
Pacing is a rendering decision
Tokens arrive from the network in bursts: nothing for 300ms, then forty tokens at once. Render them as they arrive and the text stutters and lurches. The fix is a small buffer between the network and the DOM that meters tokens out at a readable rhythm, draining faster when it gets behind so the buffer never grows unbounded.
Once you own the rhythm, you can shape it. Uniform delay reads as mechanical, so the demo above jitters each token's delay and adds a longer pause after sentence-ending punctuation. It is a small thing that changes the texture completely: the output reads as writing rather than printing.
Layout has to hold still
The reader is trying to read while you mutate the document under them. Two rules keep that tolerable. First, nothing that has already rendered may move: no reflowing earlier paragraphs, no re-measuring containers mid-stream. Second, auto-scroll must be polite. Stick to the bottom while the user is at the bottom, but the moment they scroll up to re-read something, stop following. Yanking the scroll position away from someone mid-read is the single most hostile thing a streaming UI can do.
Markdown adds a wrinkle: half-arrived syntax. An unclosed code fence will restyle the entire rest of the message as code for a few frames. If you render markdown mid-stream, parse defensively (auto-close open fences before rendering) or hold formatting back until the block completes.
Screen readers hear this too
An aria-live region that announces every token is unusable, and one that announces nothing
leaves a non-visual user staring at silence. The pattern I like: aria-live="polite" on the
message container so it announces in coherent chunks when the user is idle, plus an explicit
announcement when generation completes. The "done" signal is the one that matters most;
sighted users get it from the cursor disappearing, so everyone else deserves an equivalent.
The stop button is part of the typography
Every streaming interface needs a visible way to interrupt, and it needs to work instantly even though the backend will take a moment to actually stop. Cancel the rendering loop first, flush what has arrived, then tell the server. Users judge responsiveness by the pixels, not the socket.
None of this touches the model or the protocol. It is all interface craft, which is good news: it means the feel of a streaming UI is fully in our hands.