moritz/lazyflat
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Packages Wiki Activity Actions

enrichment: drop LLM for structured info, dedup images by sha + phash

Browse source 
Per user request, the LLM is no longer asked to extract rooms/size/rent/WBS —
those come from the inberlinwohnen.de scraper which is reliable. Haiku is now
used for one narrow job: pick which <img> URLs from the listing page are
actual flat photos (vs. logos, badges, ads, employee portraits). On any LLM
failure the unfiltered candidate list passes through.

Image dedup runs in two tiers:
1. SHA256 of bytes — drops different URLs that point to byte-identical files
2. Perceptual hash (Pillow + imagehash, Hamming distance ≤ 5) — drops the
   "same image at a different resolution" duplicates from srcset / CDN
   variants that were filling galleries with 2–4× copies

UI:
- Wohnungsliste falls back to scraper-only display (rooms/size/rent/wbs)
- Detail panel only shows images + "Zur Original-Anzeige →"; description /
  features / pros & cons / kv table are gone
- Per-row "erneut versuchen" link + the "analysiert…/?" status chips were
  tied to LLM extraction and are removed; the header "Bilder nachladen (N)"
  button still surfaces pending/failed batches for admins

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
EiSiMo 2026-04-21 15:29:55 +02:00
parent 374368e4af
commit 0aa4c6c2bb
6 changed files with 137 additions and 233 deletions
Download patch file Download diff file Expand all files Collapse all files

54
web/templates/_wohnungen_body.html
View file

@ -90,8 +90,8 @@
hx-post="/actions/enrich-all" hx-target="#wohnungen-body" hx-swap="outerHTML">
<input type="hidden" name="csrf" value="{{ csrf }}">
<button class="btn btn-ghost text-xs" type="submit"
hx-confirm="Altbestand jetzt durch Haiku nachträglich anreichern? Kann einige Minuten dauern.">
Anreichern ({{ enrichment_counts.pending + enrichment_counts.failed }})
hx-confirm="Bilder für ausstehende Wohnungen nachladen? Kann einige Minuten dauern.">
Bilder nachladen ({{ enrichment_counts.pending + enrichment_counts.failed }})
</button>
</form>
{% endif %}
@ -133,52 +133,16 @@
{% elif item.last and item.last.success == 1 %}<span class="chip chip-ok">beworben</span>
{% elif item.last and item.last.success == 0 %}<span class="chip chip-bad">fehlgeschlagen</span>
{% endif %}
{% if f.enrichment_status == 'pending' %}<span class="chip">analysiert…</span>
{% elif f.enrichment_status == 'failed' %}<span class="chip chip-warn" title="Detail-Analyse fehlgeschlagen">?</span>
{% endif %}
</div>
<div class="text-xs text-slate-500 mt-0.5">
{% if f.enrichment_status == 'pending' %}
Infos werden abgerufen…
· <span data-rel-utc="{{ f.discovered_at|iso_utc }}" title="{{ f.discovered_at|de_dt }}">…</span>
{% elif f.enrichment_status == 'failed' %}
{% set err = (item.enrichment or {}).get('_error') or 'unbekannt' %}
<span title="{{ err }}">Fehler beim Abrufen der Infos</span>
{% if is_admin %}
<form method="post" action="/actions/enrich-flat" class="inline"
hx-post="/actions/enrich-flat" hx-target="#wohnungen-body" hx-swap="outerHTML">
<input type="hidden" name="csrf" value="{{ csrf }}">
<input type="hidden" name="flat_id" value="{{ f.id }}">
<button type="submit" class="underline text-slate-600 hover:text-slate-900 ml-1">erneut versuchen</button>
</form>
{% endif %}
· <span data-rel-utc="{{ f.discovered_at|iso_utc }}" title="{{ f.discovered_at|de_dt }}">…</span>
{% else %}
{# LLM first, scraper as fallback. The scraper data
from inberlinwohnen.de is reliable; we only
replace it when the LLM has a concrete value. #}
{% set e = item.enrichment or {} %}
{% set rooms = e.rooms if e.rooms is not none else f.rooms %}
{% set size = e.size_sqm if e.size_sqm is not none else f.size %}
{% set rent = e.rent_total or e.rent_cold or f.total_rent %}
{% if e.wbs_required is sameas true %}
{% set wbs_label = 'WBS: ' ~ (e.wbs_type or 'erforderlich') %}
{% elif e.wbs_required is sameas false %}
{% set wbs_label = 'ohne WBS' %}
{% elif f.wbs == 'erforderlich' %}
{% set wbs_label = 'WBS: erforderlich' %}
{% elif f.wbs == 'nicht erforderlich' %}
{% set wbs_label = 'ohne WBS' %}
{% else %}
{% set wbs_label = '' %}
{% endif %}
{% set parts = [] %}
{% if rooms %}{% set _ = parts.append('%g Z'|format(rooms)) %}{% endif %}
{% if size %}{% set _ = parts.append('%.0f m²'|format(size)) %}{% endif %}
{% if rent %}{% set _ = parts.append('%.0f €'|format(rent)) %}{% endif %}
{% if wbs_label %}{% set _ = parts.append(wbs_label) %}{% endif %}
{{ parts|join(' · ') }}{% if parts %} · {% endif %}<span data-rel-utc="{{ f.discovered_at|iso_utc }}" title="{{ f.discovered_at|de_dt }}">…</span>
{% set parts = [] %}
{% if f.rooms %}{% set _ = parts.append('%g Z'|format(f.rooms)) %}{% endif %}
{% if f.size %}{% set _ = parts.append('%.0f m²'|format(f.size)) %}{% endif %}
{% if f.total_rent %}{% set _ = parts.append('%.0f €'|format(f.total_rent)) %}{% endif %}
{% if f.wbs == 'erforderlich' %}{% set _ = parts.append('WBS: erforderlich') %}
{% elif f.wbs == 'nicht erforderlich' %}{% set _ = parts.append('ohne WBS') %}
{% endif %}
{{ parts|join(' · ') }}{% if parts %} · {% endif %}<span data-rel-utc="{{ f.discovered_at|iso_utc }}" title="{{ f.discovered_at|de_dt }}">…</span>
</div>
</div>
<div class="flex gap-2 items-center">