feature/oembed-detection-fix #3

Merged

patillacode merged 2 commits from feature/oembed-detection-fix into main

2026-04-09 15:01:51 +02:00

patillacode commented

2026-04-09 14:55:53 +02:00

Owner

What

Two separate improvements bundled in one branch:

README & project cleanup

Docker Compose examples now use the published registry image (no local build required)
Branch strategy simplified to main → feature/* (dropped develop)
just pr now detects the current branch automatically
Fixed a silent bug: detect_short was swallowing exceptions, so the error tracker in app.py never fired on
YouTube failures

Short detection rewrite (production bug fix)

The previous approach fetched youtube.com/shorts/{id} and checked if the final redirect URL still contained
/shorts/.
This is broken for EU deployments!
YouTube redirects all requests to consent.youtube.com first, so /shorts/ is never found and every video gets cached as is_short=0.

Replaced with the oEmbed API:

GET https://www.youtube.com/oembed?url=https://www.youtube.com/shorts/{id}&format=json

Shorts are portrait (height > width); regular videos are landscape. Bypasses consent walls entirely.

Edge case handling:

Non-200 (age-restricted, 404): return False without caching, retried on next feed refresh
Missing dimensions in response: raise ValueError so the error tracker fires and the video passes through uncached

After deploying

Clear the poisoned video cache (all entries are wrong is_short=0):

docker exec yt-shorts-proxy python -c "
import sqlite3; conn = sqlite3.connect('/data/cache.db')
conn.execute('DELETE FROM videos'); conn.commit(); print('cache cleared')
"

What Two separate improvements bundled in one branch: 1. `README` & project cleanup - Docker Compose examples now use the [published registry image](forgejo.patilla.es/patillacode/yt-shorts-proxy:latest) (no local build required) - Branch strategy simplified to `main` → `feature/*` (dropped `develop`) - `just pr` now detects the current branch automatically - Fixed a silent bug: `detect_short` was swallowing exceptions, so the error tracker in `app.py` never fired on YouTube failures 2. Short detection rewrite (production bug fix) The previous approach fetched `youtube.com/shorts/{id}` and checked if the final redirect URL still contained `/shorts/`. This is broken for EU deployments! YouTube redirects all requests to `consent.youtube.com` first, so `/shorts/` is never found and every video gets cached as `is_short=0`. Replaced with the `oEmbed API`: ``` GET https://www.youtube.com/oembed?url=https://www.youtube.com/shorts/{id}&format=json ``` Shorts are portrait (height > width); regular videos are landscape. Bypasses consent walls entirely. Edge case handling: - Non-200 (age-restricted, 404): return False without caching, retried on next feed refresh - Missing dimensions in response: raise `ValueError` so the error tracker fires and the video passes through uncached After deploying Clear the poisoned video cache (all entries are wrong `is_short=0`): ``` docker exec yt-shorts-proxy python -c " import sqlite3; conn = sqlite3.connect('/data/cache.db') conn.execute('DELETE FROM videos'); conn.commit(); print('cache cleared') " ```

patillacode added 2 commits

2026-04-09 14:55:53 +02:00

Simplify branch strategy, fix README, and patch silent error swallowing 800362c10a

- README: use published image in docker-compose examples, simplify branch
  strategy to main→feature, update standalone setup instructions
- docker-compose.yml: switch from build: . to published registry image
- justfile: just pr now detects current branch automatically
- detect.py: remove dead DB_PATH var; let detect_short exceptions propagate
  so app.py error tracker actually fires on YouTube failures
- tests: update detect_short error test to expect propagation; remove
  redundant detect.DB_PATH patches (cache.DB_PATH already patched by fixture)

Fix Short detection: replace redirect-follow with oEmbed API

Tests / test (pull_request) Successful in 18s

Details

c3e8f7695f

The previous approach fetched youtube.com/shorts/{id} and checked if the
final redirect URL still contained /shorts/. This breaks for EU deployments
because YouTube redirects all requests to consent.youtube.com first, so
/shorts/ is never found and every video is cached as is_short=0.

The oEmbed API bypasses consent entirely and returns reliable portrait/
landscape dimensions. Shorts are portrait (height > width); regular videos
are landscape.

Behaviour:
- Non-200 response (age-restricted, 404, etc.): return False without caching
- Missing dimensions in response: raise ValueError so the error tracker fires
  and the video passes through uncached (better to let a Short slip through
  than break a feed permanently)
- Network exceptions: propagate as before so error tracker fires

After deploying, clear the poisoned video cache:
  docker exec yt-shorts-proxy python -c "
  import sqlite3; conn = sqlite3.connect('/data/cache.db')
  conn.execute('DELETE FROM videos'); conn.commit(); print('cache cleared')
  "