Combined flat-alert + flat-apply with authenticated web UI
Find a file
EiSiMo 83db8cd902 enrichment: size floor + LLM fallback for opaque CDN URLs
Two issues surfaced on HOWOGE and similar sites:

1. Tiny icons/1x1 tracking pixels leaked through (e.g. image #5, 1.8 KB).
   Added MIN_IMAGE_BYTES = 15_000 and MIN_IMAGE_DIMENSION = 400 px on the
   short side; files below either threshold are dropped before saving.
   Pillow already gives us the dims as part of the phash pass, so the
   check is free.

2. Listings whose image URLs are opaque CDN hashes
   (.../fileadmin/_processed_/2/3/xcsm_<hash>.webp.pagespeed.ic.<hash>.webp)
   caused the LLM URL picker to reject every candidate, yielding 0 images
   for legit flats. Fixes: (a) prompt now explicitly instructs Haiku to
   keep same-host /fileadmin/_processed_/ style URLs even when the filename
   is illegible, (b) if the model still returns an empty set we fall back
   to the unfiltered Playwright candidates, trusting the pre-filter instead
   of erasing the gallery.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-21 16:44:47 +02:00
alert guard double-apply, hide error msg, wohnungen polish, bitwarden block 2026-04-21 14:20:31 +02:00
apply enrichment: Haiku flat details + image gallery on expand 2026-04-21 14:46:12 +02:00
web enrichment: size floor + LLM fallback for opaque CDN URLs 2026-04-21 16:44:47 +02:00
.env.example lazyflat: combined alert + apply behind authenticated web UI 2026-04-21 09:51:35 +02:00
.gitignore lazyflat: combined alert + apply behind authenticated web UI 2026-04-21 09:51:35 +02:00
docker-compose.yml enrichment: Haiku flat details + image gallery on expand 2026-04-21 14:46:12 +02:00
README.md lazyflat: combined alert + apply behind authenticated web UI 2026-04-21 09:51:35 +02:00

lazyflat

Combined deployment of flat-alert (reliable scraper) and flat-apply (experimental autoapplier) behind a single authenticated web UI.

Architecture

Three isolated containers on one internal Docker network:

┌─────────┐   POST /internal/flats    ┌─────────┐   POST /apply     ┌─────────┐
│  alert  │ ────────────────────────► │   web   │ ────────────────► │  apply  │
│ scraper │                           │  UI+DB  │                   │ browser │
└─────────┘                           └─────────┘                   └─────────┘
                                          ▲
                                          │  HTTPS (Coolify / Traefik)
                                          │
                                       user (auth)
  • alert/ — scrapes inberlinwohnen.de (unchanged logic) and posts each discovered flat to web. Zero external dependencies on apply, so apply crashes can never bring the alerter down.
  • apply/ — FastAPI wrapper around the experimental Playwright applier. Only accepts requests with a shared X-Internal-Api-Key. Not exposed publicly.
  • web/ — FastAPI + Jinja + HTMX dashboard. The only public service. Owns the SQLite database, auth, and orchestration.

Safety / isolation

Because apply/ is still experimental, the system is hardened around it:

Control Behavior
Separate container A crashed apply does not take alert or web with it.
Internal-only network apply is not reachable from the internet; requires internal API key.
Default mode manual New flats are just shown in the UI; apply runs only on click.
Circuit breaker N consecutive apply failures auto-disable further apply calls.
Kill switch One-click button in the UI that blocks all apply activity.
SUBMIT_FORMS=False Default for apply — runs the full flow without final submit.
Audit log Every auth event, mode change, and apply is recorded.

Web security

  • Argon2id password hashes (argon2-cffi), constant-time compare.
  • Session cookie: signed with itsdangerous, HttpOnly, Secure, SameSite=Strict.
  • CSRF: synchronizer token bound to the session on every state-changing form.
  • Login rate limit (in-memory, per-IP).
  • Strict Content-Security-Policy, X-Frame-Options: DENY, noindex.
  • /internal/* endpoints gated by a shared INTERNAL_API_KEY and never exposed by Coolify.

Deployment on Coolify

  1. Create repo: push this monorepo to ssh://git@git.moritz.run:2222/moritz/lazyflat.git.
  2. New Coolify resourceDocker Compose → point it at this repo. Coolify will read docker-compose.yml and deploy all three services on one network.
  3. Domain: set flat.lab.moritz.run on the web service only. Coolify (Traefik) handles TLS. Do not set a domain on alert or apply.
  4. Secrets: paste the environment variables from .env.example into Coolify's env UI. At minimum you need:
    • AUTH_USERNAME, AUTH_PASSWORD_HASH
    • SESSION_SECRET, INTERNAL_API_KEY
    • GMAPS_API_KEY, BERLIN_WOHNEN_USERNAME, BERLIN_WOHNEN_PASSWORD
    • personal info + WBS for apply
  5. Generate the password hash:
    python -c "from argon2 import PasswordHasher; print(PasswordHasher().hash('<your-password>'))"
    
  6. Generate secrets:
    python -c "import secrets; print(secrets.token_urlsafe(48))"   # SESSION_SECRET
    python -c "import secrets; print(secrets.token_urlsafe(48))"   # INTERNAL_API_KEY
    
  7. First apply launch should stay in manual mode with SUBMIT_FORMS=False until each provider is verified end-to-end.

Local development

cp .env.example .env
# fill in AUTH_PASSWORD_HASH, SESSION_SECRET, INTERNAL_API_KEY, creds
docker compose up -d --build
# open http://localhost:8000  (set COOKIE_SECURE=false for plain http!)

To also expose the web port locally, add ports: ["8000:8000"] under web: in docker-compose.yml (the Coolify production compose doesn't publish host ports).