enrichment: Haiku flat details + image gallery on expand

apply service
- POST /internal/fetch-listing: headless Playwright fetch of a listing URL,
  returns {html, image_urls[], final_url}. Uses the same browser
  fingerprint/profile as the apply run so bot guards don't kick in

web service
- New enrichment pipeline (web/enrichment.py):
  /internal/flats → upsert → kick() enrichment in a background thread
    1. POST /internal/fetch-listing on apply
    2. llm.extract_flat_details(html, url) — Haiku tool-use call returns
       structured JSON (address, rooms, rent, description, pros/cons, etc.)
    3. Download each image directly to /data/flats/<slug>/NN.<ext>
    4. Persist enrichment_json + image_count + enrichment_status on the flat
- llm.py: minimal Anthropic /v1/messages wrapper, no SDK
- DB migration v5 adds enrichment_json/_status/_updated_at + image_count
- Admin "Altbestand anreichern" button (POST /actions/enrich-all) queues
  backfill for all pending/failed rows; runs in a detached task
- GET /partials/wohnung/<id> renders _wohnung_detail.html
- GET /flat-images/<slug>/<n> serves the downloaded image

UI
- Chevron on each list row toggles an inline detail pane (HTMX fetch on
  first open, hx-preserve keeps it open across the 3–30 s polls)
- CSS .flat-gallery normalises image tiles to a 4/3 aspect with object-fit:
  cover so different source sizes align cleanly
- "analysiert…" / "?" chips on the list reflect enrichment_status

Config
- ANTHROPIC_API_KEY + ANTHROPIC_MODEL wired into docker-compose's web
  service (default model: claude-haiku-4-5-20251001)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
EiSiMo 2026-04-21 14:46:12 +02:00
parent 2609d3504a
commit eb66284172
11 changed files with 688 additions and 44 deletions

View file

@ -195,6 +195,13 @@ MIGRATIONS: list[str] = [
);
CREATE INDEX IF NOT EXISTS idx_rejections_user ON flat_rejections(user_id);
""",
# 0005: LLM enrichment — extracted details + downloaded image count per flat
"""
ALTER TABLE flats ADD COLUMN enrichment_json TEXT;
ALTER TABLE flats ADD COLUMN enrichment_status TEXT NOT NULL DEFAULT 'pending';
ALTER TABLE flats ADD COLUMN enrichment_updated_at TEXT;
ALTER TABLE flats ADD COLUMN image_count INTEGER NOT NULL DEFAULT 0;
""",
]
@ -447,6 +454,31 @@ def get_flat(flat_id: str) -> Optional[sqlite3.Row]:
return _conn.execute("SELECT * FROM flats WHERE id = ?", (flat_id,)).fetchone()
def set_flat_enrichment(flat_id: str, status: str,
enrichment: Optional[dict] = None,
image_count: int = 0) -> None:
with _lock:
_conn.execute(
"""UPDATE flats SET enrichment_status = ?,
enrichment_json = ?,
enrichment_updated_at = ?,
image_count = ?
WHERE id = ?""",
(status,
json.dumps(enrichment) if enrichment is not None else None,
now_iso(), image_count, flat_id),
)
def flats_needing_enrichment(limit: int = 100) -> list[sqlite3.Row]:
return list(_conn.execute(
"""SELECT id, link FROM flats
WHERE enrichment_status IN ('pending', 'failed')
ORDER BY discovered_at DESC LIMIT ?""",
(limit,),
).fetchall())
# ---------------------------------------------------------------------------
# Applications
# ---------------------------------------------------------------------------