Coverage for src/local_deep_research/web/dependencies/csrf.py: 96%
118 statements
« prev ^ index » next coverage.py v7.16.2, created at 2026-10-22 21:20 +0000
1"""
2CSRF protection for FastAPI.
4Implements a session-based CSRF check:
5- Generate a random token, store in session
6- Include token in forms as hidden field, and expose via /auth/csrf-token
7- The frontend sends it back as the X-CSRFToken header (or, for
8 urlencoded bodies only, a csrf_token form field)
9- CSRFMiddleware validates on every state-changing request
11This replaces Flask-WTF's CSRFProtect.
13DECISION RECORD — "why not fastapi-csrf-protect?" (evaluated for PR #3299,
14in response to a maintainer review comment preferring an established
15library over hand-rolled security code, a reasonable default instinct):
17 Evaluated github.com/aekasitt/fastapi-csrf-protect v1.0.7 (PyPI, Sep
18 2025 release; actively maintained, no known CVEs) against this file's
19 actual requirements and found it a poor fit on four independent axes:
21 1. Enforcement model. It is a per-route `Depends()` dependency, not
22 ASGI middleware — there is no middleware class in the package at
23 all. Adopting it means adding `csrf_protect: CsrfProtect = Depends()`
24 plus an explicit `await csrf_protect.validate_csrf(request)` call to
25 every one of this app's ~135 POST/PUT/PATCH/DELETE route handlers
26 (17 router files), each a manual opt-in. `CSRFMiddleware` here is a
27 single fail-closed choke point: every unsafe-method request is
28 checked unless explicitly listed in `_SKIP_EXACT_PATHS` /
29 `_SKIP_PATH_PREFIXES` (itself covered by
30 tests/security/test_csrf_hardening.py), and any *future* route is
31 protected automatically. A per-route dependency is fail-OPEN by
32 omission — one forgotten `Depends()` on any current or future
33 mutating route silently ships with no CSRF check, and nothing
34 catches it short of a bespoke "every mutator has the dependency"
35 meta-test that would have to be built and maintained anyway.
36 2. Session coupling. It uses its own signed double-submit cookie
37 (itsdangerous `URLSafeTimedSerializer`, a second secret key, a
38 second cookie) with no integration with Starlette's
39 `SessionMiddleware`. Our token lives in the session payload
40 (`request.session["_csrf_token"]`) — i.e. inside the app's existing
41 signed, HttpOnly, SameSite=Lax session cookie, not a second
42 cookie under a second secret. To be precise about the taxonomy:
43 that makes this a *session-bound signed double-submit*, not a
44 server-side synchronizer store — the token rides in the same signed
45 cookie the session does. The properties that matter follow from the
46 binding, not from where the bytes are parked: a cross-origin
47 attacker cannot read the cookie through JavaScript (HttpOnly) or
48 read the token endpoint/page across origins (same-origin policy),
49 nor forge a signed session (SECRET_KEY). Explicit SameSite=Lax also
50 keeps the cookie off cross-site unsafe requests while permitting
51 incoming top-level GET links. The token dies with its session —
52 `request.session.clear()` at login (session-fixation defence) and at
53 logout drops it, so the next mint is a fresh token. The library's
54 cookie is independent of auth state; its own README has route
55 handlers call `unset_csrf_cookie()` by hand "to prevent token
56 reuse" — invalidation becomes a per-handler responsibility instead
57 of falling out of existing session lifecycle.
58 3. Header + form-field support together (what this middleware needs
59 for the no-JS form fallback) is only available via the package's
60 `fastapi_csrf_protect.flexible` sub-package, which is materially
61 less mature: introduced in 1.0.4, that release was pulled from
62 PyPI as a "FAILED ROLLOUT ... WIP code" and iterated through
63 1.0.5-1.0.7 to reach a working state. Its base (non-flexible) mode
64 is single-location only (header XOR body), which this app's mixed
65 JS/no-JS forms need to be XOR-able across the whole app, not one
66 endpoint.
67 4. DoS shape. The library's `get_csrf_from_body()` buffers the full
68 request body via `await request.body()` with no size cap of its
69 own — the exact synchronous-parse-on-the-single-event-loop hazard
70 the 256 KB cap below (`_MAX_CSRF_FORM_BODY`) was added to close;
71 this app's general body-size ceiling is upload-sized (tens/hundreds
72 of MB), far above what's safe to `parse_qs` synchronously per
73 request. That this is a real, not theoretical, hazard for this
74 library is corroborated by its own issue tracker/changelog: GH
75 issue #23 ("CSRF Token from body can cause 'Stream consumed'
76 Exception with Form data") and the 1.0.6 changelog entry fixing a
77 `Stream consumed` bug in that same code path.
79 Verified against the published 1.0.7 wheel (not just its docs), Aug
80 2026: the distribution contains only `core.py`, `csrf_config.py`,
81 `load_config.py`, `exceptions.py` and the `flexible/` sub-package —
82 no middleware module of any kind (1); `validate_csrf()` reads
83 `request.cookies[cookie_key]` and unsigns it with its own
84 `URLSafeTimedSerializer`, touching `request.session` nowhere (2);
85 base `LoadConfig.token_location` is `Literal["body", "header"]`,
86 header-XOR-body, and `flexible/` is absent from the 1.0.3 wheel and
87 present in 1.0.5, matching the changelog's "1.0.4 ... FAILED ROLLOUT
88 ... Rolled out with WIP code; immediately deleted version from PyPI"
89 (1.0.4 is indeed missing from the PyPI release index) (3); and
90 `get_csrf_from_body(await request.body())` buffers unbounded (4).
91 Two things found in that read that the review comment did not raise
92 and that cut *toward* this file: the library compares tokens with a
93 plain `token != signature`, not a constant-time compare, where the
94 code below uses `secrets.compare_digest`; and its body extraction is
95 `data.decode().replace("&", '","').replace("=", '":"')` fed to a
96 pydantic model — a hand-rolled urlencoded parser that mangles any
97 value containing `&` or `=`. "Use a library" is the right default;
98 it is not automatically the more careful option.
100 None of this is a case against ever using a library for CSRF — the
101 underlying pattern here (session-bound token, compared with
102 `secrets.compare_digest`, carried in the app's existing signed session
103 cookie) is the standard OWASP-documented approach, not novel
104 crypto. It's an assessment that *this particular* library's
105 architecture (per-route opt-in, decoupled cookie) doesn't fit *this*
106 app's shape (single ASGI choke point, session-bound tokens, one
107 event loop to protect). Re-evaluate if a library ships a pure-ASGI,
108 session-integrated CSRF middleware with an equivalent exemption
109 model — none was found as of this review (Aug 2026).
111 Coverage this file must keep preserving: tests/security/
112 test_csrf_hardening.py, test_csrf_protection.py, test_csrf_e2e_flow.py;
113 tests/web/test_csrf_middleware_edges.py,
114 tests/web/dependencies/test_csrf_body_cap.py; and the browser-level
115 proof in tests/ui_tests/test_download_and_csrf_flows_ci.js (enforcement
116 in both directions, JSON and multipart).
117"""
119import hashlib
120import secrets
122from fastapi import Request
123from loguru import logger
124from starlette.responses import JSONResponse
125from starlette.types import ASGIApp, Receive, Scope, Send
128def _log_path_preview(path: str) -> str:
129 """Bounded preview of the request path for the rejection log lines.
131 (#6938: the path is client-chosen and logged before authentication,
132 so it goes through the shared value preview. Imported at call time so
133 this middleware never imports the routers package at import time.)
134 """
135 from ..routers.notes import _log_value_preview
137 return _log_value_preview(path)
140def generate_csrf_token(request: Request) -> str:
141 """Generate or retrieve CSRF token for the current session.
143 Stores the token in the session so it persists across requests.
144 Malformed legacy/session state is rotated instead of being returned to a
145 template or API caller as a token that validation can never accept.
146 """
147 token = request.session.get("_csrf_token")
148 if not isinstance(token, str) or not token or not token.isascii():
149 token = secrets.token_hex(32)
150 request.session["_csrf_token"] = token
151 return token
154def get_auth_context(session) -> str | None:
155 """Fingerprint a login for retry fencing, without exposing its session ID.
157 This value grants no authority. CSRF validation and authentication still
158 apply independently; the fingerprint only detects a different login.
159 """
160 session_id = session.get("session_id")
161 if (
162 not session.get("username")
163 or not isinstance(session_id, str)
164 or not session_id
165 ):
166 return None
167 return hashlib.sha256(
168 f"ldr-csrf-context:{session_id}".encode("utf-8")
169 ).hexdigest()
172def _tokens_match(session_token: object, provided_token: object) -> bool:
173 """Compare usable string tokens without letting malformed state raise."""
174 if not isinstance(session_token, str) or not isinstance(
175 provided_token, str
176 ):
177 return False
178 if not session_token.isascii() or not provided_token.isascii():
179 return False
180 return secrets.compare_digest(session_token, provided_token)
183def validate_csrf_token(request: Request, token: str) -> bool:
184 """Validate a CSRF token against the session token.
186 Args:
187 request: The current request.
188 token: The token from the form submission.
190 Returns:
191 True if the token matches, False otherwise.
193 Not on any request path: `CSRFMiddleware` below is the single
194 enforcement point and does its own check. This exists for handlers
195 that need to validate a token out-of-band (and for direct unit
196 testing of the comparison); if you reach for it in a route, that is
197 a sign the route should be relying on the middleware instead.
198 """
199 session_token = request.session.get("_csrf_token")
200 if not session_token or not token: 200 ↛ 201line 200 didn't jump to line 201 because the condition on line 200 was never true
201 return False
202 return _tokens_match(session_token, token)
205# Methods that mutate state and require CSRF validation.
206#
207# Enforcement is keyed on the METHOD and nothing else, so a route that
208# mutates state has to be declared on one of these verbs to be covered.
209# That is deliberate: the method arrives in its own ASGI scope key and
210# has exactly one spelling, whereas a per-path allowlist has to agree
211# with the router about how a path is written, and disagreeing fails
212# OPEN (the middleware misses, the router still matches, the handler
213# runs). Keep state-changing routes on an unsafe verb rather than
214# re-introducing a path allowlist to cover a mutating GET.
215_UNSAFE_METHODS = frozenset({"POST", "PUT", "PATCH", "DELETE"})
217# Routes that bootstrap auth or are token-authenticated and so cannot
218# carry a session-bound CSRF token. These bypass CSRF validation.
219# Exact paths that bootstrap auth and so cannot carry a session-bound
220# CSRF token yet. Prefix matching is too permissive (e.g. /auth/login
221# would also match /auth/login-attacker-route) so we match exactly.
222_SKIP_EXACT_PATHS = frozenset(
223 {
224 # `/auth/csrf-token` is the token-mint endpoint itself — can't
225 # require a token to fetch one.
226 "/auth/csrf-token",
227 }
228)
229# NOTE: `/auth/validate-password` is deliberately NOT listed here. It was
230# exempt under Flask as an "idempotent strength check called before any
231# session exists", but the register/change-password forms that call it
232# already render a CSRF token via the same template injection used by
233# `/auth/login`, so the exemption bought nothing and left an unauthenticated
234# POST that accepts a password outside the middleware.
235# NOTE: `/auth/login` and `/auth/register` are intentionally NOT listed
236# here. Both forms render a CSRF token via template injection and POST
237# it back in a hidden field; the middleware validates it normally.
238# Listing them here re-opens a login-CSRF (OWASP A07) vector where an
239# attacker-controlled form silently logs the victim into the attacker's
240# account.
242# Prefixes that route to non-cookie-authenticated subsystems. /api/v1
243# is NOT in this list: those endpoints currently use require_auth
244# (session cookies), so CSRF applies.
245_SKIP_PATH_PREFIXES = (
246 # Socket.IO ASGI handles its own auth handshake. Include both
247 # `/ws/` and `/ws` because the app mounts at `/ws` (no trailing
248 # slash) and a POST to the bare mount path would otherwise
249 # miss the prefix match.
250 #
251 # Caveat this bare entry carries, and the reason the exact-match
252 # rule above exists: `startswith("/ws")` also matches a
253 # hypothetical `/wsearch` or `/wsomething`. Nothing routes there
254 # today — `app.mount("/ws", socket_app)` serves only `/ws` and
255 # `/ws/...`, and no router prefix begins with `/ws` — so this is
256 # currently inert, but it is a standing constraint: do NOT register
257 # a route whose path starts with `/ws` but is not the Socket.IO
258 # mount, or it ships CSRF-exempt. Tighten to an exact `/ws` entry
259 # plus the `/ws/` prefix if that ever stops being obvious.
260 "/ws/",
261 "/ws",
262)
265class CSRFMiddleware:
266 """ASGI middleware enforcing CSRF on state-changing requests.
268 Validates the `X-CSRFToken` header (preferred, set by frontend JS)
269 or, for `application/x-www-form-urlencoded` bodies only, the
270 `csrf_token` form field, against the per-session token stored in
271 `request.session["_csrf_token"]`.
273 Multipart and JSON bodies are deliberately NOT parsed for a token:
274 those callers must send the header, and a multipart/JSON request
275 without one fails closed with a 403 (pinned by
276 tests/web/test_csrf_middleware_edges.py::
277 test_multipart_csrf_field_is_not_honored and
278 ::test_json_body_token_field_is_not_honored). The no-JS fallback
279 only ever needs the urlencoded path, since a plain HTML form
280 without JS posts urlencoded.
282 Runs INSIDE SessionMiddleware so request.session is populated.
283 """
285 def __init__(self, app: ASGIApp) -> None:
286 self.app = app
288 async def __call__(
289 self, scope: Scope, receive: Receive, send: Send
290 ) -> None:
291 if scope["type"] != "http":
292 await self.app(scope, receive, send)
293 return
295 method = scope.get("method", "GET").upper()
296 if method not in _UNSAFE_METHODS:
297 await self.app(scope, receive, send)
298 return
300 path = scope.get("path", "")
301 if path in _SKIP_EXACT_PATHS or any(
302 path.startswith(p) for p in _SKIP_PATH_PREFIXES
303 ):
304 await self.app(scope, receive, send)
305 return
307 session = scope.get("session") or {}
308 session_token = session.get("_csrf_token") if session else None
310 # Read headers before token validation: a replacement login may not
311 # have rendered a page (and minted its CSRF token) yet.
312 headers = {
313 k.decode("latin-1").lower(): v.decode("latin-1")
314 for k, v in scope.get("headers", [])
315 }
316 # Bind opt-in writes to the login rendered into their page, including
317 # a change between CSRF refresh and retry. This header grants no auth.
318 expected_context = headers.get("x-ldr-auth-context")
319 if expected_context is not None:
320 context = get_auth_context(session)
321 if not _tokens_match(context, expected_context):
322 response = JSONResponse(
323 {
324 "error": "Your sign-in changed. Reload this page before trying again."
325 if context
326 else "Authentication required"
327 },
328 status_code=409 if context else 401,
329 )
330 await response(scope, receive, send)
331 return
333 # Fail closed: an unsafe request MUST carry a session-bound CSRF
334 # token. Endpoints legitimately reachable without one (login,
335 # register, csrf-token fetch, password-strength check) are in
336 # _SKIP_EXACT_PATHS above. Everything else needs a token, whether
337 # the caller is authenticated or not — an attacker can forge an
338 # unauthenticated POST just as easily, and an empty-session
339 # bypass makes the middleware pointless for any future public
340 # mutator endpoint.
341 if not session_token:
342 logger.warning(
343 "CSRF rejected: request lacks session _csrf_token ({} {})",
344 method,
345 _log_path_preview(path),
346 )
347 response = JSONResponse(
348 {"error": "CSRF token missing: fetch /auth/csrf-token first"},
349 status_code=403,
350 headers={"X-LDR-CSRF-Rejected": "1"},
351 )
352 await response(scope, receive, send)
353 return
355 provided = headers.get("x-csrftoken") or headers.get("x-csrf-token")
357 # Only buffer the body when we actually need to read the
358 # `csrf_token` form field — i.e., when the header is missing AND
359 # the request is form-urlencoded. Doing this unconditionally
360 # forced every file upload (multipart) and every JSON API POST
361 # into memory before the handler could stream it.
362 body = b""
363 needs_body_replay = False
364 # Lowercase the value: media types are case-insensitive (RFC 9110),
365 # so a spec-valid "Application/X-WWW-Form-Urlencoded" must still match
366 # — otherwise the form-body token extraction is skipped and a request
367 # carrying a valid csrf_token FIELD is wrongly rejected with 403.
368 content_type = headers.get("content-type", "").lower()
369 if not provided and "application/x-www-form-urlencoded" in content_type:
370 # Cap the buffered form body. This handler runs on the single
371 # event loop, so buffering + `parse_qs` here is synchronous work
372 # that stalls EVERY request/WebSocket for its duration. Without a
373 # cap, an authenticated caller could POST a multi-hundred-MB
374 # urlencoded body (up to BodySizeLimit's upload-sized ceiling)
375 # with no X-CSRFToken header and force a multi-second parse_qs on
376 # the loop. A legitimate CSRF-bearing form (the no-JS fallback)
377 # is a few KB; well-behaved clients send the token in the header.
378 # If the form body exceeds the cap we fail closed: leave
379 # `provided` unset so the token check below rejects with 403.
380 _MAX_CSRF_FORM_BODY = 256 * 1024 # 256 KB
381 body_chunks: list[bytes] = []
382 more_body = True
383 buffered = 0
384 overflowed = False
385 while more_body:
386 message = await receive()
387 if message["type"] == "http.request": 387 ↛ 399line 387 didn't jump to line 399 because the condition on line 387 was always true
388 chunk = message.get("body", b"")
389 buffered += len(chunk)
390 if buffered > _MAX_CSRF_FORM_BODY:
391 overflowed = True
392 # Keep draining so the connection isn't left with an
393 # unread body, but stop accumulating.
394 more_body = message.get("more_body", False)
395 continue
396 body_chunks.append(chunk)
397 more_body = message.get("more_body", False)
398 else:
399 more_body = False
400 body = b"".join(body_chunks)
401 needs_body_replay = True
402 if overflowed:
403 logger.warning(
404 "CSRF rejected: urlencoded body over the form-parse cap "
405 "with no X-CSRFToken header ({} {})",
406 method,
407 _log_path_preview(path),
408 )
409 response = JSONResponse(
410 {
411 "error": (
412 "CSRF token missing: send it in the X-CSRFToken "
413 "header for large form submissions"
414 )
415 },
416 status_code=403,
417 )
418 await response(scope, receive, send)
419 return
420 try:
421 from urllib.parse import parse_qs
423 parsed = parse_qs(body.decode("utf-8", errors="replace"))
424 csrf_values = parsed.get("csrf_token") or []
425 if csrf_values:
426 provided = csrf_values[0]
427 except Exception:
428 provided = None
430 # Both sides are untrusted runtime state. Header bytes are latin-1
431 # decoded, form values may contain U+FFFD, and a stale/forged session
432 # payload may hold a non-string value. ``compare_digest`` raises for
433 # non-ASCII strings and mismatched operand types, so validate both
434 # operands and fail closed with the normal 403 instead of a 500.
435 if not provided or not _tokens_match(session_token, provided):
436 logger.warning(
437 "CSRF validation failed: {} {} (token present: {})",
438 method,
439 _log_path_preview(path),
440 bool(provided),
441 )
442 response = JSONResponse(
443 {"error": "CSRF token missing or invalid"},
444 status_code=403,
445 headers={"X-LDR-CSRF-Rejected": "1"},
446 )
447 await response(scope, receive, send)
448 return
450 # Replay the buffered body to the inner app. If we never buffered
451 # (header-provided token, or multipart/JSON upload), pass
452 # `receive` through unchanged so the handler reads from the wire.
453 if not needs_body_replay:
454 await self.app(scope, receive, send)
455 return
457 body_replayed = False
459 async def replay_receive() -> dict:
460 nonlocal body, body_replayed
461 if not body_replayed:
462 body_replayed = True
463 chunk = body
464 body = b""
465 return {
466 "type": "http.request",
467 "body": chunk,
468 "more_body": False,
469 }
471 # The buffered body has been handed over. Defer to the real
472 # transport from here on instead of manufacturing an endless
473 # stream of empty http.request messages.
474 #
475 # This is load-bearing, not tidiness. Below ASGI spec_version
476 # 2.4 — uvicorn advertises "2.3" — Starlette's StreamingResponse
477 # races the body iterator against listen_for_disconnect(), which
478 # is `while True: await receive()` until it observes an
479 # http.disconnect. Repeating http.request never ends that loop,
480 # and because this coroutine had no await point it never yielded
481 # to the event loop either. A single form-token POST to a
482 # streaming route (POST /library/api/download-all-text is one
483 # today) therefore pinned the loop at 100% forever: with
484 # workers=1 that is the whole instance — HTTP, Socket.IO and the
485 # health endpoint included — until the process is killed.
486 return await receive()
488 await self.app(scope, replay_receive, send)