Wyze Vision · engineering notes

Recording every camera on one PC, and finding the few seconds worth seeing

Home cameras record far more video than anyone will watch, and the usual answer is to send it all to the vendor's cloud. I wanted the opposite: the cameras as plain video sources, and one PC doing everything else. That turned out to be two problems pulling against each other. Recording must never stop, and the detection that finds the interesting moments is exactly the part most likely to slow down, leak or crash. These notes are about keeping those two apart, and about the mistakes that taught me how.

The short version
66commits since June 2026
15,168lines of Python
20test scripts, run before every commit
13 sfor recording to return after every recorder process was killed
0.16 sfor one detection scan, down from about two seconds
1vision-model call per event, never one per frame

01Why I built it

I wanted my cameras recorded around the clock, and I wanted to know when something worth seeing happened: a person, an animal, a package, a vehicle arriving. I did not want the video to leave the house to get that. A vendor's cloud does the job, but it means the household's video lives on someone else's computers, and every decision about what is interesting is made somewhere I cannot see.

So the design starts from one position. The cameras are plain video sources. One PC records them, watches them, decides what matters, and lets me search and play back the result. The vendor's cloud is switched off, and nothing in the video path depends on it.

That puts every hard problem on one machine. Recording has to be continuous, so nothing else on the PC may ever interrupt it. Detection has to catch someone crossing the picture in a few seconds, at night as well as by day, without reporting every headlight sweep and every object that looks a little like a person. And the PC is shared with other work, so the expensive parts have to earn their place.

02Architecture at a glance

The system is two programs and a keep-alive. The recorder owns the cameras; the app owns everything that thinks. They meet at one local channel that never waits.

RecorderA supervisor and a worker. One ffmpeg per camera copies the stream, without decoding it, into an archive, a rolling clip buffer and a UDP feed to the app.
Keep-alive and launcherRestarts the app if it exits or stops answering, and keeps the recorder running. One launcher script is the only way to start or stop either.
Stream readersThe app decodes the UDP feed into frames for live view and detection, and measures from the files on disk how much of every hour was recorded.
Motion watch and detectorA cheap motion watch on every camera, then an object detector on the GPU only where something moved, a tracker, and a gate that decides what is an event.
Vision checkAn optional local vision model, called once per event to say whether it is real and describe it in a sentence.
Footage reviewRe-checks each closed hour from the recording and adds anything the live check missed.
Learning from ratingsTurns thumbs up and down into explainable rules and a small model of which kinds of report are wrong.
Sightings and searchThe short daily list of what is worth seeing, and search by meaning or by words, every result linked to the recording.

The shape follows one rule: the part that must never fail should depend on nothing, and the parts that are allowed to fail should fail without touching it. Everything clever lives in the app, and the app can be restarted at any moment without losing a frame of recording.

03From a bridge to plain video streams

The first version, in June 2026, reached the cameras the way most hobby projects do: through a community bridge that spoke the vendor's own protocol, running in a container inside a Linux environment on Windows, with a proxy in front of it. The app pulled snapshots from the bridge and analysed them.

It worked, but every layer had its own way of failing quietly. The bridge could report a camera as connected while replaying a cached frame. The Linux environment shut itself down when it thought nobody was using it. The proxy could exit and leave a blank screen. I spent much of that month building watchdogs for infrastructure I did not control.

In November I replaced all of it. The cameras can serve a standard video stream directly, so the system now reads that and nothing else. The rebuild is 53 of the project's 66 commits. The old launchers, container files and bridge code were deleted rather than left switched off, so nothing could restart the old stack by accident.

04Recording that nothing else can stop

In the first November design, one ffmpeg process per camera wrote the recording and also handed frames to the app through a pipe. Then a thread leak slowed the app down. The pipe filled, ffmpeg blocked on it, and the recording stalled with it. For two hours, between 37 and 60 per cent of the video was recorded, while the dashboard still showed every camera as live.

The fix was to stop sharing anything that can wait. The recorder is now its own program. It copies each camera's stream into five-minute MP4 files with sound, into four-second pieces used to cut event clips, and onto a UDP port on the same machine. UDP never makes the sender wait. If the app is busy, frozen or gone, its packets are dropped and the recording carries on.

The recorder is a supervisor and a worker, each holding a lock so a second copy exits at once. The supervisor restarts a worker whose heartbeat stops. The worker restarts any camera whose recorded time has not advanced for 20 seconds, with a delay that grows from two seconds to a minute. Every database write goes through a queue, so a locked database cannot delay recording either.

It is also started so that it belongs to nobody. Once, an update to another program closed the group of processes the dashboard had been started in, and the dashboard stayed down all night. The recorder had been started outside that group, and it recorded straight through. The keep-alive is now started the same way.

Measured on the running system. Restarting the dashboard left no gap in recording. Killing the recorder's worker had recording back in 11 seconds, and killing every recorder process had it back in 13.

05Knowing what is actually on disk

The pipe incident taught me that a tile saying live is not evidence of recording. So the app now measures the recording itself. For every closed hour and every camera, it reads the archive files, takes each file's start from its name and its length from ffprobe, and works out the share of the hour on disk and the longest gap. An hour under 98 per cent raises a warning, whatever the cause.

Cameras fail in their own ways too. A camera's video service can stop listening, or accept a connection and send nothing, while the camera itself stays on the network. Reconnecting cannot fix that. The app probes a silent camera locally and names the cause: moved, service down, no video, unreachable, or a login problem. With the vendor's cloud off, a frozen camera is reported for a power-cycle rather than restarted remotely.

The commonest cause turned out to be the simplest. After a power cut the router hands out new addresses. Every two minutes the app checks that the device at each camera's address still has that camera's hardware address. If not, it sweeps the local network for the video port and finds each camera again, by hardware address, or failing that by matching one picture against the camera's usual view.

06Cheapest check first

Early on, the detector looked at each camera about once every eleven seconds. Someone crossing the picture in five seconds was seen in one picture or none. Running the detector more often on every camera would have cost far too much, so the work is now ordered by cost.

A motion watch looks at every camera five times a second, on a grey copy shrunk to 480 by 270 pixels. Each look takes a few milliseconds and is plain arithmetic: no model at all. It keeps a running background and masks out the camera's own clock. It drops changes in light alone. When headlights or a cloud brighten a patch, the patch keeps its texture, so a correlation test can tell light from a thing that moved. A spot must move on two looks in a row.

Only a camera with movement is scanned by the object detector, on the GPU, about once a second until the movement stops. Where the motion watch saw something and the detector found nothing in the full picture, the detector takes a zoomed second look around the moving spot. That makes a small animal several times larger in the detector's input.

The detector moved to the GPU. On the CPU it averaged about 2.6 cores, with peaks of 7.5. It now picks its graphics card by name, falls back to the CPU and says why, and never uses the card another service depends on.

07Three looks, and somewhere to go

A single confident detection is not good evidence. At night the detector will read a reflection, a shadow or a still object as a person, with high confidence, on one look. So a tracker joins looks into one track per living thing, letting the labels vote, and a person or animal is reported only after three looks of the same track, or two if the detector is very sure.

I measured that rule against every event I had rated before adopting it. Real people and animals were detected again two or three times in the next three seconds. Twelve of the fourteen rejected look-alikes were never detected again at all.

The second half of the rule is that a living thing goes somewhere. A track must have moved at least a fifth of its own size from where it started. One look-alike that kept firing moved a tenth of its size; real people near it moved between a third and a half. Spots where still look-alikes live are learned into a fixture map, rebuilt hourly, and a report on one of them needs to move much further before it counts.

08One vision call per event

A local vision model can say much more than a detector: what is in the box, whether it is real, what it is doing. It is also slow, and it is shared with other work on the same PC. So it is asked once per event, never once per frame, and everything works without it.

The event's picture goes in with the object boxed in red, and a short structured answer comes out: real, false or unclear, what it is, and one plain sentence of description. That one call replaced up to three separate calls per event. Two settings made it fast enough: thinking switched off, and the image shrunk to 640 pixels. A yes or no check went from 6.3 seconds to 0.9.

Shrinking has a cost, and I found it the hard way: small objects get lost. Anything whose box is under 12 per cent of the picture's width is now sent as a crop around the box instead.

What the answer is allowed to do depends on what it costs to be wrong. A false verdict can mute a vehicle or a package report. It never mutes a person: the report is kept with the disagreement noted, because a false alarm is cheap next to a missed visitor. And one switch pauses every AI call while recording and detection carry on.

09A second look at every hour

The live check works under a time budget, so it will sometimes miss something. Fifteen minutes after each hour ends, the footage review reads that hour back from the recording and looks again, with no AI call of its own.

It is built to be cheap. The live motion watch logs, for every camera and minute, how many looks it took and whether anything moved, so the review decodes only the minutes with movement, plus any minute nobody was watching. It decodes key frames only, on the GPU, which measured 2.3 times faster than the CPU for that job. A small, fast detector looks first, and the main one only where the small one sees a person or animal.

Some things the detector cannot name at all, such as something small and dark at night. The review also follows moving spots across key frames, and keeps one that lasts ten seconds and travels well beyond its own size. It rejects short, always-brighter spots as headlight flashes, and many spots moving at once as a lighting change. Anything the live check did not report becomes an event marked found on review, and the vision model checks it like any other.

10Learning from a thumbs-down without going deaf

Ratings are the obvious way to teach the system what matters, and the obvious design is wrong. My first version learned where: enough thumbs-down at a spot in the picture muted that kind of report there. Then a run of thumbs-down on night-time look-alikes muted a real person walking through the same spot. A thumbs-down had been read as "ignore this area", which is never what it means.

Now ratings teach three explainable things. Spots still learn, but for people and animals a spot only flags a report; the vision check on that picture decides. A fixture map records where still look-alikes live. And a small logistic-regression model learns what kind of report gets rejected, from named signals every event already carries: the kind, night, how many looks, how far it travelled, whether the vision check agreed.

The model is retrained a few seconds after each rating. It mutes nothing until it has twenty ratings with at least three of each kind, and then only when it is very sure. It reports how often it would have agreed with me, leave-one-out, so it is never a black box. It also lists the sightings it is least sure about, because rating those teaches it most.

A thumbs-down can carry a word or two. Those words are a reason, never a relabel: a person report that names some object as the culprit stays a rejected person report, and is never relabelled as that object.

Sightings is the short list of what is worth knowing, by day: people, animals and packages always, vehicles only when they are unusual. Each card has the picture zoomed to the object, the clip and a link into the recording. A review mode plays each sighting's recording from a few seconds before it, with large rating buttons beside Next, so a day can be worked through on a phone.

Search works two ways and fuses them. Each event becomes one descriptive sentence, embedded by a local model, so a search can match by meaning. A full-text index matches exact words. The two rankings are combined with reciprocal-rank fusion, so a name is found as well as an idea. The cut-off for a meaning match was calibrated on the system's own events rather than picked.

The search index also held one of the slower bugs. Every start re-indexed every capture, deleting each old row with a lookup the index could not do quickly. At about seven captures a second it would have needed some sixteen hours, so it never finished, and every live save paid the same cost. Index rows are now keyed by the capture's own id, and a rebuild in batches took about two and a half minutes.

12How a change is proven

Most of what went wrong here looked correct in the code. So a change counts as done only when one check passes. It compiles every Python file, runs all twenty test scripts against simulated cameras and throwaway data, and parses every script on every page. Against the running app, it calls every API the pages read and opens every page in a headless browser at desktop and phone width, failing on any script error, console error or page that draws nothing. A pre-commit hook runs the fast part.

Detection has its own proof. A replay tool runs recorded footage through the live logic at live timing, with a simulated clock and throwaway data, against known cases that must and must not be reported. Every detection change since mid-November was checked against that list, which grew to twenty cases.

A test is only worth having if it can fail. When the check was added, I put that day's dashboard bug back in, and both the syntax stage and the browser stage failed at the exact line. The thread-leak test fails on the old code with more than five hundred extra threads.

13Twelve lessons

Each of these is something the system did wrong, usually quietly, and the rule it left behind. Most of them share a shape: a signal that looked healthy while the thing it was meant to describe had failed.

01Connected is not live

In the bridge era, a camera could report itself connected while the bridge replayed a cached frame. The dashboard showed pictures twenty minutes old as live.

Now: Liveness comes from the content itself: the recorder's media time must keep advancing, and 20 seconds without new video means a restart.

a status flag reports what a component believes; check the output it is supposed to produce.

02The uptime figure described something else

The bridge died about once a minute. The uptime I checked looked healthy for hours, because it described the shared kernel underneath, not the Linux environment that was being shut down when idle. The system journal said so plainly.

Now: The bridge is gone, and every liveness check measures the unit that actually fails.

a metric about the parent says nothing about the child.

03The latest release was not patched

A published flaw named the video tool and the transport this system uses. I updated to the newest tagged release and assumed that was the fix. The fix existed only in unreleased code, and two copies of the tool were still exposed.

Now: A binary built before the disclosure date cannot contain the fix, whatever its version says. The recorder also limits ffmpeg to the protocols it needs.

check the build date against the disclosure, not the version against the list.

04A silent except hid a dead component

In June, a reconnector referenced a key that no longer existed, and a bare except swallowed the error on every pass. Cameras sat waiting for recovery for sixteen hours with no action taken. In November, the vision worker's outer handler did the same and hid every failure it had.

Now: Every background loop logs its errors to the system log, once per ten minutes per place.

an error handler that says nothing turns a crash into a component that is merely absent.

05A pipe let the app stop the recording

One ffmpeg per camera fed both the recording and the app. When the app slowed down, the pipe filled and the recording stalled with it, for two hours, while every tile said live.

Now: The recorder is its own process and talks to the app only over UDP, which never waits.

if one part must never stop, give it nothing that can make it wait.

06A thread pool per scan

Each scan built a fresh pool of threads and called the detector from them. The library underneath keeps worker threads for every thread that calls it, so each scan left some behind. After a day the app held 3.7 GB, and every camera feed dropped about once a minute.

Now: All inference runs on one long-lived thread, and a test fails on the old code.

a library's per-thread state turns short-lived threads into a leak.

07Database setup on every save

Every save re-ran the schema and a dozen migrations, each needing the write lock, so a save took most of a second and sometimes over eight. A moving camera was looked at every few seconds instead of every second, and a person walking past was seen once, never the three times the rule needs.

Now: Migrations run once per process, saving happens on one writer thread, and the live check fails if a scan's median goes over 0.6 seconds.

a slow write can look like a detection bug; time every stage before tuning the detector.

08A thumbs-down silenced a person

Eleven thumbs-down on night-time look-alikes at one spot muted every person report there, including a real person walking through.

Now: For people and animals a spot only flags a report, and the vision check on that picture decides.

a rating says this report was wrong, not that nothing there will ever matter.

09The busiest hours were never reviewed

The footage review asked ffmpeg for its frames with one condition per frame. Past about a hundred frames the expression was too long, ffmpeg returned nothing, its error output was hidden, and the fallback used the same list. The hours with the most movement were exactly the ones skipped.

Now: Frames are requested as runs, and a test asks for 140 scattered frames and checks every one comes back.

a failure that scales with how interesting the input is will hide in your best data.

10Shrinking the picture hid the animal

To make the vision check fast, the whole picture was shrunk to 640 pixels. At night that turned small, distant things into shapeless dark patches, and the model described them as exactly that. Zoomed around the box, the same model described them correctly.

Now: Objects under 12 per cent of the picture's width are sent as a crop.

a speed setting is a decision about what the model may no longer see.

11The address watch never ran

After a router restart renumbered the cameras, the app kept the old addresses for about an hour and a half: one camera's video was filed under another's name, and a working camera was restarted three times for nothing. The code meant to follow a moved camera had never matched an address with a login in it. The rewrite then shipped calling a function on the wrong object, so every tick raised an error and nothing was followed.

Now: The watch reports when it last ran, and the live check fails if that is missing or more than ten minutes old.

a safety net needs a check that it is actually running, not only a test that it would work.

12Better on one measure, worse on the one that mattered

I fine-tuned the detector on the system's own footage. False alarms on look-alikes fell from 17 per cent to none. Real people and animals found fell from 64 per cent to 59, and vehicles from 98.5 to 95.

Now: A tuned model is switched on only if it has fewer false alarms and finds at least as many real visitors, so this one stays off until there are more ratings to train on.

gate a model change on everything it must keep, not on the number it was built to move.

14What I would do differently

Most of these are things the system eventually taught me. Knowing them at the start would have saved a few weeks.