Skip to content

Architecture

Guiding principle: heavy client, light server

What drives the cost of a video product is whoever processes the frames. Here the phone assembles the vlog and the server never sees a raw frame: it only signs URLs, keeps rows in PostgreSQL and decides whose turn it is.

The premiere spike — the whole group opening the vlog at once — is soaked up by the CDN. The origin never notices. The topology is drawn below, and the exchange that makes it work in Storage and network.

The pieces

flowchart LR
    subgraph phone["Phone (does the work)"]
        CAM[Camera] --> MAN[(Manifest<br/>on disk)]
        MAN --> EDL[EDL]
        EDL --> NAT[Native engine<br/>MediaCodec + OpenGL]
    end

    subgraph fly["Fly.io - one machine, Amsterdam"]
        API[FastAPI]
        SCH[Scheduler<br/>every 15 min]
    end

    NEON[(Neon Postgres<br/>Frankfurt)]
    R2[(Cloudflare R2)]
    CDN{{Public bucket domain}}
    EXPO[Expo Push] --> FCM[FCM] --> phone

    NAT -->|"presigned PUT"| R2
    phone <-->|"JSON only"| API
    API --- NEON
    SCH --- NEON
    API -->|"signs, never carries"| R2
    R2 --- CDN
    CDN -->|"playback"| phone
    API --> EXPO

    style R2 fill:#f38020,color:#fff
    style NEON fill:#00e599,color:#000
    style fly fill:#7b3fe4,color:#fff

The arrow that is missing is the point: no video ever passes through the API.

Backend (backend/)

Module Responsibility
app/api/ HTTP routers: auth, circles, turns, media (dev only)
app/api/deps.py The current user (token, revocation, deleted account) and the membership check
app/api/ratelimit.py The in-memory rate limiter, as middleware -- see Rate limiting
app/api/serializers.py ORM rows to response shapes: names, viewers, reactions, comments, the history summary
app/services/turns.py Turn life cycle, the re-roll vote and the deferred push queue. The heart of the loop
app/services/members.py Leaving, being removed, deleting the account: the turn and ownership handovers
app/services/voting.py Re-roll arithmetic: how many votes a turn needs. Pure
app/services/draw.py The draw: weighted RNG on top of a CSPRNG
app/services/messages.py Every push and email the server writes, in both languages
app/services/users.py What name to show for an account, deleted ones included
app/services/storage.py Object abstraction: LocalStorage (dev) / S3Storage (R2)
app/services/push.py Expo Push Service, with a null backend for development
app/services/email.py SMTP, with a null backend that only logs
app/services/reset.py Short single-use codes: password resets and email verification
app/services/invites.py Invite codes, with no characters that get misread aloud
app/services/edl.py The edit algorithm (mirror of the client's)
app/services/render.py Fallback FFmpeg render (optional, off)
app/services/render_gate.py One server render at a time, with a bounded wait
app/services/staging.py Raw takes waiting for a server render
app/core/i18n.py Which language a request wants, and the Spanish for every error detail
app/workers/scheduler.py APScheduler: one tick() per interval (900 s live)

The checks every turn route makes are FastAPI dependencies rather than lines repeated at the top of each handler (#155), so a route says in its signature what it needs and cannot skip a step: MemberTurn (the turn exists and the caller is in its circle), VloggerTurn (...and it is theirs), OpenVloggerTurn (...and it is still open, deadline not passed), VotableTurn (it can still be voted on). SERVER_RENDER sits on the route itself, so a server with the fallback off answers 501 before looking up any turn. The owner-only circle routes take OwnedCircle the same way.

Client (mobile/)

Module Responsibility
src/edit/edl.ts Edit Decision List: cuts and transitions. Pure
src/edit/vad.ts Silence detection (FFmpeg's silencedetect). Unreachable -- see below
src/edit/commands.ts FFmpeg command construction. Pure
src/edit/renderer.ts Orchestrates VAD -> EDL -> normalisation -> assembly (FFmpeg). Unreachable -- see below
src/edit/nativeRenderer.ts The path a device actually takes: the same job through the native engine
src/edit/engine.ts Picks between ffmpeg, native and server, and says what each gives up
src/edit/nativeFallback.ts Retries on the server when the native engine throws mid-render
src/edit/renderQueue.ts One render at a time; a newer edit supersedes the one running
src/edit/rebuild.ts Rebuilding the preview from the review screen
src/edit/trimming.ts TRIM_SILENCES, the one constant that switches silence trimming (off)
modules/vlog-editor/ The native engine: MediaCodec + GL on Android
src/edit/serverRender.ts Compatibility path: assemble on the server
src/edit/publish.ts Upload with retries, confirmation and purge of raw clips
src/api/client.ts Every call to the API, with the token refresh behind it
src/api/fileUpload.ts Streaming a file off disk, failing after 120 s with no progress
src/api/retry.ts The one retry-with-backoff loop both upload paths share
src/storage/clipVault.ts Clip vault in the sandbox plus a persistent manifest
src/state/ Session, circles, recording, recovery after a power cut
src/state/turnPages.ts The history screen's turns, a page at a time
src/state/writeQueue.ts Serialises the saves of a camera flip, so no segment is lost
src/state/saveVlog.ts "Guardar en el dispositivo": download a vlog into the gallery
src/state/observability.ts Sentry, only when a DSN is configured
src/state/notifications.ts Push registration, a no-op inside Expo Go
src/ui/Splash.tsx Boot screen shown while the session and any pending render resume
src/ui/Sheet.tsx Bottom sheet behind the circle's overflow menu
src/ui/Roulette.tsx The draw reveal: names spin past and land on the vlogger
src/storage/seenDraws.ts Which reveals this device has already played
src/i18n/ Every string on screen, in Spanish and English, and which one this phone gets
app/ Screens (expo-router)

Screens

app/ is laid out the way expo-router reads it, one file per route:

Route Screen
/ (index.tsx) Splash, then off to /circles or /login depending on the session
/login, /reset, /verify-email Sign in or register, reset a password with a code, confirm the address (skippable)
/circles, /circles/new, /circles/join Your circles; create one; join one by invite code
/circle/[id] The circle: the roulette reveal, whose turn it is and how it is going, the premiere and its comments, the re-roll vote
/manage/[circleId] Owner settings: name, cadence, lengths, premiere hours, invite code, members
/history/[circleId] Past vlogs, by month
/account, /account/delete Display name, email verification, push test, signing out here or everywhere; closing the account
/record/[turnId] The camera and the take list
/render/[turnId] The edit running, with its progress
/review/[turnId] Watching the result before publishing it, with the editing controls below
/watch/[turnId] Playing a vlog: reactions, saving it to the gallery

The vlogger's path through them is circle -> record -> render -> review -> watch: publishing happens on the review screen, and it lands on the premiere. A server render skips review -- the server publishes the moment it has assembled the vlog -- and goes straight from render to watch.

The API

Every route the API serves, in one place, bar /media/..., which only exists in local storage mode. Who may call each one is the dependency it takes (above); the sections further down explain the ones with a story behind them.

Route What it does
POST /auth/register, POST /auth/login A token pair. Registration also sends a verification code
POST /auth/refresh A new pair from a refresh token
POST /auth/logout-all Ends every session and clears every push token of the account
POST /auth/password Change the password; ends every session
POST /auth/password-reset/request, /confirm A code by email; the code and a new password
POST /auth/email-verification/resend, /confirm Another code; mark the address verified
GET, PATCH, DELETE /auth/me Who you are; change your display name; delete the account (asks for the password)
PUT /auth/me/language What your pushes and emails are written in
PUT, DELETE /auth/me/push-token Register this device's push token; forget it on sign-out
POST /auth/me/push-test A test notification to every device of yours
POST /circles, GET /circles, POST /circles/join Create, list, join by code
GET, PATCH /circles/{id} The circle with its members and current turn; owner settings
POST /circles/{id}/invite-code A new invite code (owner)
PATCH /circles/{id}/mute Silence this circle on your own phone
POST /circles/{id}/draw Force a draw (owner); 409 while a turn is open
GET /circles/{id}/turns/current The open turn, or the one premiering
GET /circles/{id}/turns The turn history, newest first, a page at a time
DELETE /circles/{id}/members/me, /{user_id} Leave; remove somebody (owner)
GET /turns/{id} One turn
POST /turns/{id}/heartbeat The vlogger's phone reporting status and progress
POST /turns/{id}/upload-url, /complete A signed upload; confirm it and premiere
POST, DELETE /turns/{id}/reroll-vote Vote to re-roll or end the premiere; take it back
PUT /turns/{id}/clips/{filename}, DELETE /turns/{id}/clips, POST /turns/{id}/render-fallback The server render: stage takes, clear them, assemble
POST /vlogs/{id}/view Count a view
PUT, DELETE /vlogs/{id}/reactions/{emoji} React; take it back
POST /vlogs/{id}/comments Comment
GET /health Liveness, and whether the server render is on

The heartbeat is the only thing the server hears from a turn before the vlog exists. The phone sends it after every saved take (recording) and when the edit starts (rendering), with the take count and seconds of footage; the server stores both on the turn and sets its status from it. A figure the heartbeat leaves out keeps its last value rather than falling to zero (#149).

The history is paged by keyset. GET /circles/{id}/turns takes limit (1-100, default 20) and before, the id of the last turn of the previous page; the next page is everything older than that turn by (created_at, id). An id that is not a turn of this circle is a 404. Keyset rather than offset so a draw landing while somebody scrolls does not shift every later page by one. The history screen walks it through src/state/turnPages.ts, and its "N vlogs · X min" line comes from history_count and history_seconds on the circle, counted in SQL across the whole history rather than from whatever pages happen to be loaded (#159).

The draw

app/services/draw.py. Not a flat random.choice, because in a group of four that produces streaks that kill the fun:

  1. Active members only.
  2. The previous vlogger is excluded (unless they are the only option).
  3. Each candidate's weight is (group_max_turns - their_turns) + 1. Whoever has come up least gets more tickets.
  4. The tie-break uses secrets.SystemRandom, not random.

The tests (backend/tests/test_draw.py) check that across 60 rounds with five people everybody comes up, and that nobody lands outside 5..19 turns against a fair share of 12. The bounds are deliberately loose: a tight assertion on a random draw is a flaky test, and an earlier version asserting a spread of 2 failed about one run in six. See decisions.md.

The re-roll vote

A turn can be re-drawn on demand if the circle agrees. POST /turns/{id}/reroll-vote records a vote; the moment a strict majority is in, the re-roll happens inside that same request and the response is already the new turn. Nobody waits for the scheduler.

Design decisions worth knowing:

  • Strict majority, n // 2 + 1. With 2 members that means both of them — a majority of two people is two people. With 1 it means 1, so a solo circle can re-roll itself.
  • During a premiere the same vote needs everyone. Re-rolling an open turn costs nothing; ending a premiere early takes the vlog away from whoever has not watched it, so that one is unanimous. The turn goes to archived, exactly where it would have landed once the circle's premiere_hours ran out.
  • Not while the vlog is being built. rendering is not in VOTABLE_STATUSES (app/services/turns.py), so voting on a turn in that state, or taking a vote back, is a 409. The vote opens again only if the turn drops back to recording (see Turn states).
  • No strike. Unlike a missed deadline, this is the group deciding, not the vlogger failing. The turn ends as skipped, not missed.
  • The vlogger may vote too. Often the person who most wants out is the one holding the turn.
  • Votes from people who left stop counting. Otherwise someone could leave the circle and leave behind a vote they can no longer withdraw, holding the turn hostage.
  • Votes belong to the turn, so they vanish with it — there is no counter to reset when the next turn opens.

Administering a circle

Owner-only, and all of it added after the fact: a circle used to be frozen at the moment it was created.
PATCH /circles/{id} Name, cadence, length, premiere hours, and handing the circle to somebody else. Absent fields keep their value.
POST /circles/{id}/invite-code A new code; the old one stops working at once.
DELETE /circles/{id}/members/{user_id} Remove somebody. The owner cannot remove themselves — that is leaving, which is a different route.
DELETE /circles/{id}/members/me Leaving, unchanged, but now with the handovers below.
POST /circles/{id}/draw Spin the roulette now rather than waiting for the tick. A 409 while a turn is open, or when the circle has nobody active to draw.

Every owner-only route takes the OwnedCircle dependency, which loads the circle and refuses anybody but its owner with a 403; leaving is the one route here that is not owner-only.

A cadence change applies from the next draw. Moving the deadline of a turn somebody is already recording changes the deal halfway through it.

Three routes end in the same place — leaving, being removed, and deleting the account — and all three have the same two loose ends, which is why the handover lives in app/services/members.py rather than in any one of them:

  • An open turn of theirs is handed on. Status missed, no strike: leaving is not the same as being given a turn and dropping it.
  • A circle they owned is handed to whoever has been in it longest. Before this, an owner who left took the only account that could force a draw or change a setting, and there was no way to get it back.

Reactions, and what comes back to the vlogger

The product moves in one direction — record, edit on the phone, everybody watches — and for a long time nothing at all came back the other way. The return path is now four things:

  • Five emoji, PUT/DELETE /vlogs/{id}/reactions/{emoji}, idempotent, and both answer with the whole turn so the screen redraws from the server's counts rather than its own guess. Which five is REACTIONS in app/schemas/schemas.py; anything else is a 400.
  • The viewers by name. vlog_views always held them and the API returned only the count.
  • Short comments, POST /vlogs/{id}/comments. At most 200 characters, trimmed before they are checked, and a comment that is only whitespace is a 422 (CommentBody in app/schemas/schemas.py). Same reach as a reaction: anyone still in the circle, before or after the premiere closes. The vlogger gets a push about it (notify_comment), unless the comment is their own. Comments sit next to the reactions rather than replacing them, and are kept short so they stay a quick "I saw this" rather than a thread.
  • A summary when the premiere closes. close_premiere archives the turn and tells the vlogger how many watched and how many reacted. Both paths go through it — the scheduler's tick and the unanimous vote to end early — so cutting a premiere short does not skip the one report the vlogger gets.

Push: one token per device, muted per circle

Every push in this product — the draw, the deadline reminder, the premiere, the comment that just landed — goes through the same function, _notify in app/services/turns.py. It sends nothing itself: it queues the push, and the queue is only sent once the commit it describes has landed — by _committing in the scheduler's tick, and by send_after_commit in an endpoint, which also hands the Expo round trip to FastAPI's BackgroundTasks so it runs after the response has gone out (#141). When the queue is sent (_render_deferred), two decisions are made first.

Who, of the people who should hear about this, has muted the circle it happened in. Membership.muted is a per-(user, circle) flag, checked and filtered out before a single message is built. Muting is self-service and invisible to the rest of the circle — nobody's push list shrinks because somebody else stopped listening, and the muted member is drawn, records and votes exactly as before. It silences a phone, not a membership.

How many devices each remaining recipient has live. push_tokens holds one row per app install, not one per account (docs/data-model.md), so _render_deferred builds one PushMessage per token. An account signed into a phone and a tablet at once gets pushed on both; before this table existed, users held a single token and the second device to register silently evicted the first. POST /auth/me/push-test — the "Enviar notificación de prueba" button — sends to every token on the account, not just the phone that pressed it, because the device silently not getting pushes is just as likely to be the other one. It sends synchronously and answers 502 as soon as any one of them fails, with no token in the log.

Muting and multiple tokens are independent knobs, and both stop at delivery. Neither one is checked anywhere near the draw, the deadline, or who can publish — a circle you have silenced still runs exactly as if you had not.

How a token stops being pushed to. Three ways, each for a different reason:

  • Signing out of one device names that device's token in DELETE /auth/me/push-token before the app forgets its credentials (#148). Only that token, and only if it belongs to the caller; best effort, since a failure costs a stray notification, not access.
  • logout-all and deleting the account clear every token at once, the same all-or-nothing as the token_version bump that ends every session.
  • Expo saying the install is gone. deliver sends in batches of 100 and reads the ticket Expo returns for each message; a DeviceNotRegistered ticket marks that token dead, and forget_dead_tokens deletes it -- but only if its row was last registered before the send started. An install that re-registered the same token in the meantime has just proved it is alive, and PUT /auth/me/push-token bumps updated_at when it does (#147). Any other error in a ticket is logged and the token is kept: a bad credential is not the device's fault.

Language

The app speaks Spanish on a phone set to Spanish (any region) and English on any other -- pickLanguage in mobile/src/i18n/language.ts, decided once at startup, since changing the system language restarts the app anyway. Every string on screen is in src/i18n/es.ts and en.ts; anything with a name or a count in it is a function there, because the two languages do not put the pieces in the same order. EXPO_PUBLIC_LANGUAGE pins a build to one language regardless of the phone, which the Maestro build uses.

The server writes four things of its own, and each takes its language from where it can:

  • Error details follow the request. The app sends Accept-Language on every call; each HTTPException in the code is written in English, and localized_http_exception in app/main.py swaps in the Spanish from ERRORS_ES (app/core/i18n.py) on the way out. A request that sends no header gets Spanish, because every build released before this spoke only Spanish and none of them send it.
  • Pushes and emails have no request to read, so they follow users.language. Registration sets it from that request's header; after that the app corrects it (PUT /auth/me/language) whenever it stops matching the phone. _render_deferred renders each push per token, in that token's owner's language -- a circle can hold people on both, and each gets their own. The text itself lives in app/services/messages.py.
  • The name of a deleted account follows the request, like an error. A closed account's vlogs, comments, reactions and views stay with the circle, and wherever they show up the name is "Cuenta eliminada" or "Deleted account" in the reader's language (DELETED_ACCOUNT_NAME in app/core/i18n.py, via shown_name in app/services/users.py). It is chosen per response and never stored: the row's display_name is blanked instead, because storing the Spanish showed it to every phone (#157).
  • Validation errors (422) are left alone: their detail is a list written for developers, and the app shows its own sentence instead. So do a timeout and a dropped connection, which never reach the server at all.

What the phone keeps, and for how long

Raw takes are the heaviest thing the app touches, so the sandbox is swept from three directions:

  • On publish. purgeRawClips deletes the takes and purgeTurn removes the turn's directory outright, finished mp4 included. Same on both render paths, and the server clears its staging area too.
  • On every render. The normalised segments live in a render/ working directory cleared in a finally. They are as heavy as the takes themselves, and a failed render used to leave them behind until a publish that was never going to happen.
  • At startup. Takes for a turn that has since closed — someone else was drawn, the circle re-rolled, the deadline passed — are purged by state/recovery.ts. Nothing revisits an idle manifest during normal use, so before this they simply accumulated. It only deletes on a definitive answer from the server: a network error means "unknown", and footage is not worth deleting on a guess.

Clips the edit drops are not deleted early, and cannot be: which takes get dropped is only known once the EDL is computed — a take can fall below the minimum length, or land past the total the vlog can hold — and until then they are the user's only copy.

Where it is on disk

Everything lives under the app's own documentDirectory, never the camera roll:

Path What
vlogroulette/turns/<turnId>/manifest.json The turn's manifest: clips, order, render state
vlogroulette/turns/<turnId>/clip-NNN.<ext> One take (or segment, or still) per file, numbered in the order it was shot
vlogroulette/seen-draws.json Which roulette reveals this device has already played, the last 50

The session's tokens are not files: they live in SecureStore under vlogroulette.token and vlogroulette.refresh, never in AsyncStorage.

The manifest is written so that a crash cannot lose the footage (#150). Every write goes to manifest.json.tmp first and is then moved over the real one, and every read and write of one turn's manifest is serialised through a per-turn queue (exclusive in clipVault.ts) -- writes because they share that temporary file, reads because on iOS the move is a delete followed by a rename, and a read in between would find nothing. On read, a newer .tmp wins over the manifest it was replacing. If neither file can be read but take files are there, the broken manifest is set aside as manifest.corrupt-<timestamp>.json rather than deleted, the take list is rebuilt from the clip-NNN files, and the event goes to Sentry. Before this, an unreadable manifest meant an empty list, and the next sweep deleted every take as unreferenced.

The render state

The manifest's renderState is what lets the app pick up after being killed. Six values are declared (clipVault.ts); this is how today's code moves between them:

stateDiagram-v2
    [*] --> idle
    idle --> review : native engine built it
    review --> idle : the edit changed, preview discarded
    review --> uploading : publish, a ticket is issued
    idle --> uploading : server render, takes being staged
    uploading --> done : confirmed with the server
    done --> [*] : turn directory deleted

rendering is written only by the FFmpeg client pipeline, which no build can reach, and nothing writes rendered at all; both stay in the type, and the resume below still honours rendered.

At startup state/recovery.ts walks every manifest on the phone. done is deleted. Anything whose turn the server definitively says is over (closed, not the caller's, or a 404) is deleted too; a network error is not definitive, and the takes stay. idle is left alone. review is left strictly alone: the vlog is built and waiting on its author, and publishing it from here would take that decision away from them. rendered and uploading resume the publish through resumePending in src/edit/publish.ts -- see Getting the vlog up for why uploading asks the server before uploading again. A manifest still in rendering is put on a "needs a render" list, but nothing in the app reads that list yet, so an interrupted render is only redone when the vlogger opens it again (#168).

Signing in and staying signed in

An access token lasts an hour and a refresh token sixty days (decisions.md), so a 401 is an ordinary event. ApiClient in src/api/client.ts handles it on its own: it calls POST /auth/refresh and repeats the original request once -- a second 401 with a token minted seconds ago is a real refusal. Several requests failing at the same moment share the one refresh in flight rather than each starting its own. The renewed pair is handed to the session store (src/state/session.ts), which writes it to SecureStore; without that, a refresh would work for the rest of the session and be lost on the next launch. If the refresh itself fails, the session is over -- revoked, or sixty days old -- and the app signs out. stageClip, which streams off disk instead of going through fetch, uses the same headers and the same refresh, since a long server render outlives an access token.

Circles with a single member

A one-person circle is a real, supported case, not a degenerate one: it is how most people will try the app before inviting anyone.

The draw excludes the previous vlogger unless they are the only option, so the roulette simply keeps landing on the same person. The premiere notification goes to "everyone except the vlogger", which is nobody, so no push is sent and nothing errors. You can watch your own premiere and it counts as a view. test_reroll.py covers the full solo loop end to end.

The auto-edit engine

Which engine renders a given vlog is decided by src/edit/engine.ts, and it is the one place that knows what each one cannot do:

flowchart TD
    START([Assemble this vlog]) --> FF{Native FFmpeg<br/>linked into the build?}
    FF -->|yes| FFE[["ffmpeg: everything,<br/>on the device"]]
    FF -->|no| NAT{Native module present<br/>and reporting available?}
    NAT -->|"no: Expo Go, a build older<br/>than it, or no usable encoder"| SRVQ
    NAT -->|yes| MAT{Does it declare it can<br/>carry this material?}
    MAT -->|yes| DEV[["native: on the device"]]
    MAT -->|"no: e.g. stills on an<br/>engine without an encoder"| SRVQ{Is the server<br/>fallback switched on?}
    SRVQ -->|yes| SRV[["server: FFmpeg, same EDL"]]
    SRVQ -->|no| NONE[["none: nothing can render it"]]

    style DEV fill:#2d7d46,color:#fff
    style FFE fill:#2d7d46,color:#fff
    style SRV fill:#8a6d1f,color:#fff
    style NONE fill:#8a2f2f,color:#fff

Two things the shape does not show.

The FFmpeg branch is unreachable today. It is checked first because it would be the best of both — on the device and the full treatment — but ffmpeg-kit-react-native is deprecated and its binaries are gone from Maven Central, so nothing sets that flag.

"Present and reporting available" and "can it carry this material" are both asked of the device, not assumed. On Android, getCapabilities() is answered by EngineCapabilities.kt (#161): it walks MediaCodecList(REGULAR_CODECS) once, looking for an AVC encoder that can take 1080x1920 at 30 fps and an AAC encoder, and checks the API level is at least 21. Either missing, and every flag -- available, supportsPhotos, supportsCrossfade and the rest -- goes off together, because they all come from the same decode/composite/encode pass; the reason is logged. The answer is cached for the life of the process, since the codec list cannot change under it and the edit screen asks on every visit. It is written this way so a device or a build that cannot still produces a vlog -- through the server, or an honest none -- rather than a failure halfway through.

The chooser reports what it gave up rather than silently degrading, and distinguishes "this phone has no engine" from "this material needs one", so nobody goes hunting for a fault in a device that is working fine. describeCompromise() returns a key (noTrimming, hardCuts...) rather than a sentence, so engine.ts stays free of anything Expo; the render screen (app/render/[turnId].tsx) translates it through t.engine. What is kept with the vlog is a separate thing, render_notes: the native path builds it in src/edit/nativeRenderer.ts (cuts, takes dropped by the EDL or rejected by the engine, the source formats, and what was not done), the server render builds its own in render_fallback (app/api/turns.py), and either way it is stored on the vlog, so after the fact you can tell which path a vlog took and why.

The edit is split in two halves so the hard part is testable:

1. The EDL (the decision). Given each clip and its silence spans, it computes the exact cuts. It is a pure function — no files, no FFmpeg — and it exists twice, in TypeScript (mobile/src/edit/edl.ts) and Python (backend/app/services/edl.py), with the same semantics and the same tests. Rules:

  • Silence trimming is off (src/edit/trimming.ts). The spec called for it and it is built, tested and working on both the device and the server -- but on real footage it reads as the app editing over you rather than for you. A pause someone left before speaking is not necessarily dead air, and nothing here can tell the difference. Kept behind one constant, because it is a judgement about feel rather than a defect.
  • When it is on, leading and trailing silence goes with a 120 ms cushion, so the cut does not bite into the first syllable.
  • Internal silences are left alone: a pause in the middle of a sentence is information, not noise.
  • A clip that drops below 0.6 s after trimming is discarded.
  • A clip that is entirely silent (a cutaway, a landscape) is kept whole: the safe failure is not to cut.
  • Transitions overlap, so they add nothing to the total duration.

The cap belongs to the finished vlog, and this is where it is enforced: the last take is trimmed to fit and anything past it is dropped. The camera works against a slightly larger recording allowance (recording_allowance_factor, 1.05x), so that the cap can actually be reached: transitions overlap, taking a quarter of a second off at every cut, so 120s of footage makes a vlog a few seconds short of 120s.

It was 1.5x while the edit trimmed silences, which reclaimed roughly that much on ordinary speech. With trimming off, that headroom became a minute of footage somebody shoots and then loses to the cap, so it is now sized to what the edit actually removes — under five seconds even at twenty takes. A test derives the figure from the transition length rather than asserting it, so changing one and not the other is caught.

The two numbers are shown side by side on the camera screen and the edit screen says how many takes did not fit, so nothing is ever lost silently.

2. The execution. Two implementations run the same EDL: the native engine on the device, and FFmpeg on the server. What follows is the FFmpeg one, which the tests below cover; the native engine does the equivalent through MediaCodec and OpenGL. Every cut is normalised to the same profile (1080x1920, 30 fps, 48 kHz stereo) before chaining. Skip that step and mixing a portrait 30 fps take with a landscape 60 fps one drifts the result's audio out of sync. The segments are then folded together pairwise — clip 0 and 1 merge, that result merges with clip 2, and so on — each merge chaining xfade/acrossfade between exactly two streams, with the music bed (-18 dB) and loudnorm applied once, at the final merge. Not one filter_complex with every segment as a simultaneous input: see Known failure modes and safety nets for why that shape does not survive the machine this actually runs on.

Two details that break quietly, both covered by tests on either side (commands.test.ts, test_render.py):

  • xfade measures its offset against the already-chained stream, not the standalone clip. You have to accumulate the duration while subtracting each fade's overlap.
  • The pixel format has to be pinned. Left alone, xfade negotiates yuv444p and libx264 encodes "High 4:4:4 Predictive". Desktop players decode that without complaint, which is exactly why it survived review — on a phone, whose hardware decoder only does 4:2:0, it plays as perfect audio over a garbled picture. Every encode ends in format=yuv420p with an explicit -pix_fmt, -profile:v high and -level:v 4.0. loudnorm needs the same treatment on the audio side: it works at 192 kHz internally and leaves the output there unless a trailing aresample pulls it back to 48 kHz.

Known failure modes and safety nets

Two gaps the engine diagram above does not show, because both are about what happens after a choice is made, not the choice itself.

getCapabilities() on Android can only rule out the devices that were never going to work. Since #161 it checks the API level and that the encoders for the output profile exist (see The auto-edit engine), and it used to be a hardcoded true for everything. What it still cannot see is a chipset that lists the right encoder and then fails while encoding: MediaCodec has real, documented device-specific failures on some manufacturers and chipsets that only surface once assemble() is actually asked to run — and until this was noticed, nothing caught that: a runtime failure on the native path just lost the turn.

src/edit/nativeFallback.ts is the catch: if VlogEditor.assemble() throws at runtime, and the backend's server-side render is switched on (isServerRenderAvailable(), asked fresh rather than trusting whatever currentEngine decided when the screen opened), the same manifest is retried through renderOnServer and the original failure is reported to Sentry — tagged render.fallback: native-to-server — so it is visible instead of silent. If the server cannot take it, the original error still surfaces unchanged: there is no safety net to catch it, so it must not be swallowed. The capability check and this fallback cover different failures, and both are needed: the first keeps a device without the encoders off the native path before any work is wasted, the second catches a device that has them and fails anyway.

The transition assembly bounds itself to two decoded streams, and still has a last-resort fallback. A vlog with more than two clips used to build one filter_complex that opened every segment as a simultaneous FFmpeg input to chain xfade/acrossfade across all of them — decoding N 1080x1920 streams at once inside a single process. On the production machine (limits.md: one shared-cpu-1x, 1 GB at the time, 512 MB now, and that memory is for FFmpeg, not the API) this OOM-killed the render for any vlog with more than one clip, with no concurrency required to trigger it — a single normal-sized vlog, alone, was enough. The pairwise/rolling merge described above is the fix: no ffmpeg invocation ever decodes more than two streams, regardless of how many clips the vlog has.

Even that can still fail on a machine tight enough — loudnorm's own memory scales with total duration, not stream count, so the final merge of a long enough vlog can still run out of room. render() catches exactly that and degrades once to _concat_simple: a plain -c copy concat with hard cuts, no crossfades, no loudnorm. Worse quality, same compromise the app already accepts for iOS devices without crossfade support (see the diagram above) — a finished vlog with hard cuts beats no vlog at all.

Only one render runs at a time, even alone that memory budget has no room for two. render_fallback is a sync FastAPI route, so Starlette runs each request in its own threadpool thread — nothing stopped two concurrent requests from each shelling out to ffmpeg at once. A single render already sits close to a 1 GB ceiling (production is now 512 MB, a deliberate cost trade-off -- see limits.md); measured against the real machine at 1 GB, two concurrent renders there die to OOM even with the smallest possible clips — the ceiling is exactly one render at a time, not "however many users the machine can serve." app/services/render_gate.py wraps the actual render() call in a threading.Semaphore sized by settings.max_concurrent_renders (default 1), so a second request waits for the first instead of racing it for memory. The wait itself is bounded — settings.render_queue_timeout_seconds (default 300s) — so a queued request cannot tie up its threadpool worker forever; past that it gives up and the endpoint answers 503 rather than hanging.

Storage and network (zero egress)

Two jobs live here that look like one, and separating them is most of the design. The server is a small program that has to be awake at all times -- it holds who is in which circle, draws turns, and counts down the hours of a premiere -- but it only ever moves text. The video is 20-50 MB written once and read by everyone in the circle at roughly the same moment, which is transport, not computation.

Putting both on one machine breaks at exactly the wrong time: every viewer would be taking bandwidth and CPU from the process that also has to answer the API, during the premiere, which is when they all arrive at once. So the bytes never touch the server:

sequenceDiagram
    participant P as Phone
    participant A as API (Fly)
    participant R as R2
    participant O as Other phones

    P->>A: vlog is ready, where do I put it?
    A-->>P: presigned URL (signed, expiring)
    P->>R: PUT the file, straight past the server
    P->>A: uploaded
    A->>R: does it exist, and how big is it really?
    A-->>O: push: something premiered
    O->>A: anything new?
    A-->>O: playback URL
    O->>R: watch it from Cloudflare's edge

Step 3 is the one that matters. The presigned URL is a signed, expiring permit: the phone hands it to R2 and R2 accepts the upload on its own. The server authorises the transfer without carrying it, which is what lets the machine stay small enough to cost a few dollars a month.

The permit is narrow on purpose (#143): it names the key, the type (video/mp4, the only one the API will sign) and the exact byte count the phone declared, so R2 refuses any other body. And when the phone says it is done, the API asks the bucket for the object's real size and type rather than believing the phone: that is the size stored on the vlog, and an object over the upload limit or of another type is deleted instead of premiered. In status codes: upload-url answers 413 to a declared size over max_upload_bytes (200 MB) before signing anything, and complete answers 409 when the object is not there, and 413 or 415 when the bucket's own head() reports it too big or not video/mp4.

Getting the vlog up

The upload is the step most likely to meet a bad connection, so src/edit/publish.ts is built around surviving one:

  • Resuming asks first. A manifest left in uploading may belong to a PUT that finished just before the app died. So a resume calls complete before anything else; only a 409 means "upload it after all". Not asking cost the whole vlog again over mobile data for a file the bucket already had.
  • Only what can change is retried. No answer at all, a stall, 408, 429 and 5xx are worth waiting out; any other answer from the bucket is final. The loop is the shared retry() in src/api/retry.ts (four attempts, backoff from 1 s doubling), the same one the server render's take uploads use.
  • A 403 gets a new ticket, once. It means the signed URL expired -- a slow upload, or a resume the next morning, outlives it -- and asking the same URL again cannot help.
  • A stall fails after 120 s without progress, not after a fixed total. uploadFile in src/api/fileUpload.ts streams the file with FileSystem.createUploadTask and restarts its clock every time the upload reports progress, so a slow upload is left alone and a dead one is not waited on forever.

R2 rather than S3 for one reason: egress. Most object stores charge per gigabyte downloaded, and this app's whole shape is one upload read by every member, every week, forever. R2 charges nothing for it.

STORAGE_BACKEND=s3 points at any S3-compatible service; the configuration is written for Cloudflare R2:

S3_ENDPOINT_URL=https://<ACCOUNT_ID>.r2.cloudflarestorage.com
S3_BUCKET=vlogroulette-media
CDN_BASE_URL=https://media.yourdomain.com

With CDN_BASE_URL set, playback URLs point at the CDN domain and are not signed: the object caches at the edge. Without it, a read URL is signed against the bucket (handy for testing without your own domain).

STORAGE_BACKEND=local writes to disk and serves under /media/.... That router is only mounted in local mode: in production the API has no route that moves video bytes at all.

What happens to a vlog afterwards

A premiere lasts the circle's premiere_hours (24 unless the owner changed it) and then the turn is archived — but the object stays in the bucket, and archived turns still serialise a playback URL. So the history was always there in the API; for a long time nothing in the app asked for it, and a vlog simply vanished from view while continuing to cost storage forever.

app/history/[circleId].tsx is what closes that. The rules about what counts as history are pure and tested (src/storage/history.ts):

  • Only turns that produced a playable vlog. A missed or re-rolled turn is not history; a row whose only content is that somebody let the group down is worse than no row.
  • Not the current premiere, which the circle screen already shows — twice on screen, saying different things about the time left, reads as a bug.
  • Grouped by month. A flat list of dates reads like a log; months are how people remember when something happened.

This also changes an open question rather than answering it: keeping every vlog forever is defensible now that the group can watch them. It was not, while they were being stored and never shown.

None of that survives the account, the circle, or the object itself going away, though, which is what "Guardar en el dispositivo" is for: a plain download of playback_url straight into the phone's own gallery, reachable from the premiere and from any watchable row in the history. It asks for write-only gallery access — nothing here ever reads it back — and it is a snapshot, not a subscription: it copies whatever is on screen the moment it is pressed, and nothing tracks whether it was.

Rate limiting

app/api/ratelimit.py, a middleware in front of every route, and deliberately small: a fixed window of timestamps per client per rule, held in the process's memory. Only routes where abuse is cheap and does damage are limited -- guessing passwords and codes, farming accounts, mail to somebody else's inbox, pushing bytes, running FFmpeg. Reads are not, since a member refreshing a circle is not an attack. The numbers are in limits.md.

The rules come in two kinds. RULES are exact (method, path) pairs, for the routes with no id in them -- login, register, the reset and verification codes. PATTERN_RULES are named regular expressions for the three that do: stage-clip (PUT /turns/{id}/clips/{filename}, 200 an hour), render-fallback (10 an hour) and upload-url (30 an hour). A pattern rule is counted by its name, not by the path (#158). These used to be two prefix rules counted per full path, so every take staged under a new filename opened a fresh bucket and the upload limit never bit -- while the heartbeat, the re-roll vote and complete, which nobody meant to limit, all fell under it. Anything no rule matches -- the heartbeat, votes, complete included -- is not limited at all.

Who "the client" is comes from the header the proxy in front writes (CLIENT_IP_HEADER: Fly-Client-IP on Fly), never from X-Forwarded-For, whose first entry is whatever the caller chose to send; with nothing configured, the socket address. Over the limit, the answer is a 429 whose detail is translated like any other error and whose Retry-After says how many seconds until the oldest hit leaves the window. Keys nobody has used for longer than the longest window are swept every five minutes, so the map does not grow forever.

The counters living in memory is the second reason the deployment is exactly one machine (see Open decisions).

The system clock

app/workers/scheduler.py runs turns.tick() on an interval: SCHEDULER_INTERVAL_SECONDS, 60 s by default for development and 900 s in production (backend/fly.toml). The difference is Neon: its free tier suspends the compute after five idle minutes, and a query a minute would keep it awake around the clock -- see limits.md. One tick:

  1. Reminds vloggers whose deadline is close (remind_expiring): one push per turn, to the vlogger only, once fewer than reminder_hours_before (24) hours are left on an open turn. turns.reminded_at records it, so a scheduler running every fifteen minutes does not send it ninety-six times. It runs first so that a turn expiring on this same tick gets its strike, not a reminder that there is still time.
  2. Closes open turns whose deadline passed -> missed, a strike for the vlogger and an immediate draw for the next one. "Open" includes rendering: a vlog still being built when the deadline lands is missed like any other.
  3. Closes expired premieres -> archived plus a fresh draw.
  4. Starts the first turn of circles that do not have one yet.

Each turn or circle is committed on its own, and its pushes go out only after that commit (_committing), so one failure does not roll back the rest of the tick and nothing is announced that did not happen. tick() is idempotent: you can call it a thousand times without duplicating turns or notifications. That does not make it safe to run on two machines at once -- see the scheduler entry under Open decisions: the deployment is one machine, and SCHEDULER_ENABLED=false exists for local tooling such as the load test, not for sharing the tick out.

Turn states

The whole product is a state machine over one row, and it lives in data-model.md with a diagram, so there is one copy of it rather than two that drift apart.

The distinction worth carrying here: missed and skipped both hand the turn on immediately, and only missed can carry a strike -- when the deadline ran out, not when the vlogger left the circle.

Server-side fallback render

SERVER_SIDE_RENDER_ENABLED=true enables POST /turns/{id}/render-fallback, which takes the raw clips and assembles them on the server with the same algorithm.

It exists for two specific reasons, and it is off by default because it cuts against the project's economics:

  1. A build with no native module in it -- Expo Go, or one older than the module -- has no engine on the device at all. Without this path the product cannot be tried without compiling a binary.
  2. It lets the assembly pipeline be tested in CI without a device.

The client only uses it as a reserve: engine.ts returns server when the device has no usable engine, or when the engine it has says it cannot carry the material, and the edit screen goes through serverRender.ts. If the backend has it off it answers 501 and the app explains that it must be enabled, or a development build used.

Takes go up one request each, streamed off disk by expo-file-system and staged server-side (app/services/staging.py) until the assembly runs. The first version sent every take in a single multipart body; that body has to be built in JS memory on the phone, and a mobile connection drops the request long before fifty megabytes have made it across — which showed up as a bare "network connection failed" with nothing to act on. Each upload retries with backoff, except against a server rejection, which retrying cannot fix. DELETE /turns/{id}/clips clears the staging area so a retry does not assemble the same take twice.

Turning it on in production means sizing CPU and inbound bandwidth for it: the light server becomes a video server — see Known failure modes and safety nets for what that cost looked like in practice, and how the assembly step now bounds it.

Open decisions

  • Playback used to be expo-av. Its Video component is deprecated and rendered 1080x1920 H.264 as colour smears on Android — the server render was provably clean, the player was mangling it. The app now uses expo-video.
  • Stills are letterboxed, video is cropped. A video take can be reframed to fill the screen and lose little; cropping a landscape photo into a vertical frame throws most of the picture away, and unlike a take there is no shooting it again. So stills are shown whole with black bars, video covers the frame.
  • Stills from the gallery. A photo becomes a segment through -loop 1 -t, held for a duration the client chooses — nothing in the file can say how long a still should stay on screen, so it rides along as a photo_seconds field on the upload. JPEG colour is full-range and camera footage is TV-range, so the range is converted explicitly; leaving it to ffmpeg produced a brightness jump at the cut.
  • Music bed. The pipeline supports one (musicUri, -18 dB, mixed with amix), but no file ships with the project: licensing has to be settled before bundling music into the app.
  • Scheduler across instances. Not a preference any more but a hard constraint: the scheduler and the rate limiter both count in memory, so the deployment is pinned to exactly one machine. A second one would run every tick twice over -- the one-open-turn index (#142) stops a second draw, but not the same turn being closed and announced twice -- and every rate limit would silently double. Lifting it means the schedule in the database behind a lock and the limiter in something shared. See decisions.md.
  • Deleting the vlog after archiving. Nothing in the app ever deletes one, so R2 holds a circle's whole history and only grows. Trivial at today's size and a real product decision eventually: keep forever, expire, or archive. Whatever is decided also has to reach scripts/backup.py, which deliberately keeps local copies of objects that have gone from the bucket.
  • The FFmpeg client pipeline is dead code. renderer.ts, commands.ts and vad.ts are written against ffmpeg-kit-react-native, which is deprecated on npm and whose native binaries have been withdrawn: every version of com.arthenica:ffmpeg-kit-* now 404s from Maven Central. It was never added as a dependency, so engine.ts never reaches that branch. The architecture's central claim is not waiting on it — modules/vlog-editor answered it, on real hardware — but the modules are kept, because they are the reference the server render mirrors and because an in-process FFmpeg would use them unchanged. roadmap.md has the history.
  • Remote push in Expo Go. SDK 53 removed it, so notifications.ts detects Expo Go and skips registration entirely rather than logging an error nobody can act on. Real notifications need a development build.
  • expo-file-system legacy API. SDK 54 introduced a new File/Directory API and moved the old one to expo-file-system/legacy, which is what the client still imports — it is the only one with a streaming upload. Uploads go through FileSystem.createUploadTask inside uploadFile (src/api/fileUpload.ts) rather than uploadAsync, because a task reports progress and can be cancelled, which is what the stall deadline needs (#151). Porting is separate work.