moritz/lazyflat
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Packages Wiki Activity Actions

enrichment: size floor + LLM fallback for opaque CDN URLs

Browse source 
Two issues surfaced on HOWOGE and similar sites:

1. Tiny icons/1x1 tracking pixels leaked through (e.g. image #5, 1.8 KB).
   Added MIN_IMAGE_BYTES = 15_000 and MIN_IMAGE_DIMENSION = 400 px on the
   short side; files below either threshold are dropped before saving.
   Pillow already gives us the dims as part of the phash pass, so the
   check is free.

2. Listings whose image URLs are opaque CDN hashes
   (.../fileadmin/_processed_/2/3/xcsm_<hash>.webp.pagespeed.ic.<hash>.webp)
   caused the LLM URL picker to reject every candidate, yielding 0 images
   for legit flats. Fixes: (a) prompt now explicitly instructs Haiku to
   keep same-host /fileadmin/_processed_/ style URLs even when the filename
   is illegible, (b) if the model still returns an empty set we fall back
   to the unfiltered Playwright candidates, trusting the pre-filter instead
   of erasing the gallery.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
EiSiMo 2026-04-21 16:44:47 +02:00
parent 0aa4c6c2bb
commit 83db8cd902
2 changed files with 38 additions and 13 deletions
Download patch file Download diff file Expand all files Collapse all files

27
web/llm.py
View file

@ -36,11 +36,18 @@ TOOL_SCHEMA = {
SYSTEM_PROMPT = (
"Du bekommst eine Liste von Bild-URLs einer Wohnungsanzeige. Wähle nur "
"die URLs aus, die ein Foto der Wohnung zeigen (Innenraum, Außenansicht "
"des Gebäudes, Grundriss). Verwerfe Logos, Icons, Banner, Ads, "
"Bewertungs-Sterne, Karten/Stadtpläne, Mitarbeiter-Portraits, Tracking-"
"Pixel. Behalte die Reihenfolge der Input-Liste bei. Antworte "
"ausschließlich über den Tool-Call."
"die URLs aus, die ein Foto der Wohnung zeigen (Innenraum, Außenansicht, "
"Grundriss). Verwerfe Logos, Icons, Banner, Ads, Bewertungs-Sterne, "
"Karten/Stadtpläne, Mitarbeiter-Portraits, Tracking-Pixel.\n\n"
"WICHTIG: Viele Wohnungsbaugesellschaften liefern ihre Fotos über ein "
"CDN mit kryptischen Dateinamen aus (z.B. "
"fileadmin/_processed_/2/3/xcsm_abc123def.webp.pagespeed.ic.xyz.webp). "
"Solche URLs sind IMMER zu behalten, wenn sie vom selben Host wie die "
"Anzeige kommen und in /fileadmin/_processed_/ oder /assets/media/ "
"oder ähnlichen Media-Pfaden liegen. Lehne nichts nur wegen eines "
"unleserlichen Dateinamens ab.\n\n"
"Behalte die Reihenfolge der Input-Liste bei. Antworte ausschließlich "
"über den Tool-Call."
)
@ -92,5 +99,13 @@ def select_flat_image_urls(candidates: list[str], page_url: str,
# Constrain to the original candidate set so the model can't
# invent URLs (it sometimes lightly rewrites them otherwise).
allowed = set(candidates)
return [u for u in urls if u in allowed]
kept = [u for u in urls if u in allowed]
if not kept and candidates:
# The model rejected everything — usually because the URLs
# are opaque CDN hashes it couldn't identify positively.
# Trust the Playwright pre-filter instead of returning [].
logger.warning("image-select returned 0 of %d — falling back to unfiltered list",
len(candidates))
return candidates
return kept
return candidates