DocA-Streaming Response

• StreamingResponse essentials
  • It’s not “background monitoring.” It’s one HTTP request that keeps the connection open.
  • The server yields chunks (e.g., yield "..."), and the client can receive and render those chunks as soon as they arrive.
  • The stream ends when the generator finishes (no more chunks) or when the client disconnects/aborts, then the connection closes.
  • Ensure proxies don’t buffer streaming responses, otherwise chunks won’t reach the client in real time.
• Streaming mainly reduces TTFB (Time To First Byte) and perceived latency

Two streaming way

1. Chunked text streaming (most common)
   • Use media_type="text/plain" or application/octet-stream.
   • The server yields chunks (e.g., yield "..."), and the client uses fetch to read the response body as a stream (response.body).
2. SSE (Server-Sent Events)
   • Use media_type="text/event-stream".
   • Each yield must follow SSE format: data: xxx\n\n.
   • Great for event-like updates (progress, tokens, status). However, the browser’s native EventSource is mainly GET-only, so it’s less convenient when you need a POST with a complex request body.

Example

• Backend

# main.py
import asyncio
from fastapi import FastAPI
from fastapi.responses import StreamingResponse
from fastapi.middleware.cors import CORSMiddleware
from fastapi import Request
 
app = FastAPI()
app.add_middleware(
    CORSMiddleware,
    allow_origins=["http://localhost:3000"],
    allow_methods=["*"],
    allow_headers=["*"],
)
 
 
@app.get("/stream")
async def stream(request: Request):
    async def gen():
        for i in range(1, 21):
            if await request.is_disconnected():
                break
            yield f"chunk {i}: hello\n"
            await asyncio.sleep(0.3)
        yield "done\n"
 
    return StreamingResponse(gen(), media_type="text/plain")
 

• Frontend
  • React (a UI library)
    • it focuses on how you build components and render the interface.
  • Next.js (a framework)
    • it wraps React and provides the full set of tools to build a complete web app.

"use client";
 
import { useRef, useState } from "react";
 
export default function Page() {
  const [text, setText] = useState("");
  const [busy, setBusy] = useState(false);
  const controllerRef = useRef<AbortController | null>(null);
 
  const start = async () => {
    setBusy(true);
    setText("");
 
    const controller = new AbortController();
    controllerRef.current = controller;
 
    const res = await fetch("http://localhost:8000/stream", {
      method: "GET",
      signal: controller.signal,
    });
 
    if (!res.body) throw new Error("No response body");
 
    const reader = res.body.getReader();
    const decoder = new TextDecoder();
    let acc = "";
 
    try {
      while (true) {
        const { value, done } = await reader.read();
        if (done) break;
 
        const chunk = decoder.decode(value, { stream: true });
        acc += chunk;
        setText(acc);
      }
    } catch (e: any) {
      if (e?.name !== "AbortError") throw e;
    } finally {
      reader.releaseLock();
      setBusy(false);
      controllerRef.current = null;
    }
  };
 
  const stop = () => controllerRef.current?.abort();
 
  return (
    <main style={{ padding: 24 }}>
      <h1>Simulate Streaming</h1>
 
      <div style={{ display: "flex", gap: 8, marginTop: 12 }}>
        <button onClick={start} disabled={busy}>
          {busy ? "Streaming..." : "Start"}
        </button>
        <button onClick={stop} disabled={!busy}>
          Stop
        </button>
      </div>
 
      <pre style={{ marginTop: 16, color: "#111", background: "#f6f6f6", padding: 12 }}>
        {text}
      </pre>
    </main>
  );
}