Video freezes, audio continues: detecting a stall that libvlc still calls Playing

displayed +25   read +50,000   demux +48,000   audio +40   engine state: Playing
displayed  +0   read +50,000   demux +48,000   audio +40   engine state: Playing

Two adjacent one-second samples. One number changed. Bytes still arrive at the same rate, the demuxer still consumes them, audio buffers still reach the sound card, and the engine reports Playing on both lines and will keep reporting it for as long as you leave the channel open. What is on screen is the last frame that reached the video output. Those are the constants our test fixture feeds the detector, not a capture off a real channel: a 4 Mbit/s stream reads about 500,000 bytes per second, an order of magnitude above the fixture, and its read and demux numbers track far more closely than a sustained 4 percent gap. The shape is what transfers. Nothing in the pair is an error, no error event follows it, and those four deltas are the whole body of evidence a detector gets.

Everything below is measured against libvlc 3.0.21 through LibVLCSharp 3.9.7 on our Windows desktop app. The statistics API moved between libvlc majors, so read the counter claims as claims about 3.x.

libvlc reports Playing through a frozen picture#

The engine state answers a narrower question than people read into it. Playing means the playback thread has not been stopped, paused or ended. It carries no claim that a picture reached the screen in the last four seconds, and no claim that the picture which did reach the screen had anything in it.

Our watchdog specification states the premise the whole subsystem rests on: libvlc sits in Playing indefinitely through both a freeze and a black feed, and EncounteredError is reserved for hard open and protocol failures, so it almost never fires for the two failure modes viewers actually report. That sentence is our reading of the engine rather than a line lifted from its documentation, and it is the reason none of the rules below ask the engine how it is doing.

The observed consequence is the size of the taxonomy we had to build. Nine classes are defined: no bytes at the access layer, a silent end-of-stream on a live channel, the engine's own error, bytes arriving while the demuxer produces nothing, video stalled with audio alive, both stalled while data flows, black frames on a healthy pipeline, a stopped input clock, and repeated rebuffering inside a rolling window.

Three of those nine ride engine events. One is the engine's error. The other two are ordinary events that are not errors at all: an end-of-stream or a silent stop on a channel nobody stopped, and a cache-level report we count into a rebuffering loop. Six are inferred from counters, though the stopped clock among them also leans on the engine's clock event. And nine overstates what the system can say, because the stopped-clock class is never declared anywhere. It exists to corroborate and has no taxonomy code of its own.

The failure card used to throw all of that away and show one sentence for every case, so a viewer whose source refused the connection and a viewer whose device could not decode the video read identical words. Each class now maps to a code, and the same class means different things on either side of the first frame, so the mapping carries both. No bytes inside the connect grace is a stream that never began. The identical stall ten minutes in is one that stopped.

Video freeze detection is four counter deltas in a fixed order#

A sample is eleven fields: one monotonic timestamp and ten cumulative counters, read from the engine in a single pass. Eight of the ten are differenced against the previous sample. The other two, late pictures dropped and audio buffers dropped, are differenced by nothing; they were collected in case a rule wanted them and no rule ever has. Four deltas carry the verdicts:

• Bytes read, at the access layer. Zero means nothing is arriving from the network.
• Bytes consumed by the demuxer. Nonzero read with zero demux means bytes arrive that contain no program.
• Pictures displayed, at the video output. This is the primary video liveness signal, and the next section is about why it cannot be believed on sight.
• Audio buffers played, at the audio output. This is what separates a video freeze from a total one.

Decoded video is the only support counter with authority: it stands in when the displayed-picture counter turns out to be unusable. Decoded audio has no matching role. It is read once, to prove an audio stream exists. Corrupted units and discontinuities are the remaining pair, and a spike in either arms shortened freeze windows for a while.

Order matters more than any individual threshold, and the code says why in place:

// F5 / F6 presentation freezes. Suppressed while demux starvation is suspected: starvation
// stalls presentation too, and F4's longer window owns the classification (its ladder entry
// differs). F1 needs no such guard; it shares the shorter window and is checked first.

Access is tested first, the demuxer second, presentation last. A dead access layer starves everything downstream of it, so a picture that stops four seconds after the bytes stop is a symptom rather than a diagnosis, and reporting it as a freeze sends the recovery ladder in at the wrong rung. Our current windows are four seconds of zero bytes, six of a starved demuxer, four of stalled presentation. Those are tuning values that will move the next time somebody measures them; the ordering is what carries the meaning.

Suspicion is stored as an instant rather than a flag. Four tags each hold the timestamp at which their condition began, and a verdict is a tag whose age has passed its window. One good tick sets the tag back to null and the age is gone, so the window is a subtraction rather than per-tick bookkeeping. The corruption spike is the exception that proves the shape: it stores an expiry instead of an onset, nothing clears it early, and no verdict reads its age.

One scope note before the thresholds start to look universal. All of this is calibrated on raw MPEG-TS. We carry the source kind as ts or hls, but only for telemetry and for per-kind thresholds nobody has written yet. On HLS the adaptive demuxer runs its own downstream fetches, so the top-level access and demux split does not mean what this section says it means, and no threshold we ship is HLS-specific.

Why does the picture counter say a healthy channel keeps freezing?#

Some video output and hardware-decode paths underreport displayed pictures, and there is no way to ask which one you are on. Our own code comment blames the access module for this, which cannot be right: the access layer sits upstream of demux and decode and never touches the counter that the video output core increments. The comment is wrong and the behavior it describes is real.

So for the first thirty seconds after a stream starts presenting, the machine watches instead of judging. At the end of the window it commits to one of three positions across five branches: the displayed counter advanced and is trusted; the displayed counter is dead but decoded video advanced and becomes the fallback signal; or every video counter is dead while the input clock advances and a video output exists, so the path underreports and freeze rules switch off for the session. A fourth branch catches a mislabeled channel, where nothing video-side moved and no video output appeared but audio plays, and audio liveness takes over. A fifth defaults to trusting the counter.

One guard sits above all of that. No freeze verdict is possible until a video counter has advanced at least once this session, because a counter that has never moved is indistinguishable from a counter that underreports, and the genuinely-dead-from-the-start case belongs to the startup classifier, which has different evidence and a different ladder entry. Forty-five one-second ticks with audio playing, the clock advancing and a video output up, every video counter flat, declare nothing.

The edge case that shaped the calibration is the low-frame-rate slate, a station identity card or a radio visual updating at about half a picture per second. On a one-second sampler that channel spends every other tick at zero displayed pictures, the exact signature of a freeze. Calibration detects the rate and widens the freeze window from four seconds to fifteen, and the widening is the counterintuitive half: the instinct is to judge sooner on a channel with less to show, and that produces a false verdict on every slate you own. A full stop of the slate is still caught, on the wider window.

There is a cost to arming a suspicion no rule can consume, and we paid it. An audio stall beside healthy video used to raise suspicion on a video channel, where no verdict can ever read it, so the session sat in that state for its whole life: sampling twice as fast and never emitting the stability signal that clears the recovery ledger. Every later fault in that session inherited a ladder whose attempts were already spent and went straight to the give-up card. A suspicion that cannot become a verdict is not conservative. It is a leak.

A pause looks exactly like a freeze#

Every counter in the sample stops on a user pause, in precisely the pattern a total freeze produces.

There is no way to tell the two apart from the numbers, so the pause is not timed at all: it masks verdicts indefinitely, and ten minutes of frozen counters behind a pause declare nothing. A seek gets a bounded mask of five seconds instead, because a seek resolves or it does not. Buffering suspends the presentation verdicts but leaves the access rule armed, on the reasoning that zero bytes during buffering is a dead source rather than a slow link, since a slow link still shows bytes arriving.

Reopens are the other place the arithmetic bites. The byte counters are 32-bit and scoped to one media, so they restart at zero when new media is applied and they wrap on a long enough session. Either way the delta comes back negative, and a negative delta on any of the six liveness counters resets the baseline and skips evaluation for that tick rather than reporting a stall of the whole pipeline.

Black frame detection has to look at pixels#

A black feed decodes, displays, plays audio and pulls bytes at the ordinary rate. Every rule so far sees a healthy stream. This is the one path in the detector that has to look at pixels, and it needs a real frame captured off the video output at 96 by 54.

One luma threshold is not enough, and the reason is content. The verdict takes two axes and repetition:

• Fraction of black pixels at or above 0.98, where a pixel counts as black at luma 24 or below.
• Luma standard deviation at or below 4.0, which is what spares a dark-but-structured scene.
• Four consecutive black samples, which is what a fade cannot satisfy.

Both content failures are pinned by test rather than argued. A night scene, mostly near-black with scattered moonlit structure, fails on both axes at once. Letterboxed content comes out under half black, because the bars are a minority of the frame even on a wide aspect.

The fade argument has a branch I should not skip. For ten seconds after a recovery confirms, the required run drops from four samples to two, about three seconds at the configured floor and longer in practice, which a slow fade can hold. That is a deliberate trade: the evidence that condemned the source moments ago still stands, so a source that comes back broken is re-condemned faster, and the fade guard is what we spend to buy it. The 1500 ms interval is a floor rather than a spacing, too. The probe re-arms on it and fires on the next message that arrives, so realized spacing is longer: seven seconds and then nine, in the run below.

When the probe runs is a decision with a visible cost, and the honest description is narrower than the name suggests. It runs once as a sweep right after a stream first presents, and again on the transition into suspicion when a video stall raised it. Once the opening sweep closes it does not run again while the session stays healthy. Mid-play black on an otherwise healthy stream therefore produces no verdict, reproducibly, and we chose not to fix it: catching it costs a snapshot on every session of every stream, to detect what the person watching sees instantly, and zapping away and back re-runs the sweep anyway.

A second limitation I cannot close with evidence. The capture goes through the engine's own snapshot call, and readback from a hardware-decoded surface is a well-known place for a player to hand back a blank frame for reasons unrelated to what is on screen. The snapshot that never arrives, we handle: it is evidence of a wedged video output, corroborating a counter verdict but never declaring alone. The one that arrives and is wrongly black, we do not, and I have no measurement of how often that happens on our path.

The part I am least comfortable with is what the probe measures and then discards. Every sample computes a 64-bit average hash of the luma plane and a mean luma, both travel in the message, and the rule engine reads neither. A stream frozen on a bright static frame, on a channel whose picture counter calibrated as untrusted, is what two equal hashes across spaced samples would catch. We do not compare them, and whether the case is common enough to be worth the rule is a question we cannot answer, for a reason the last two sections get to. One detail from the hash surprised me enough to keep: a flat frame hashes to all ones, every one of the 64 bits set, because each cell mean sits exactly at the global mean and the comparison is greater-than-or-equal. Solid white and solid black are the same hash.

The worst bug here was a freeze that was not happening#

Reading the player's statistics is not a safe call. Every line of it dereferences native handles the engine frees when the user zaps, leaves the page, or a swapchain rebuild retires the player, and the sampler runs on its own task. Losing that race does not raise something a catch can absorb: libvlc faults with an access violation and the process disappears with nothing in the log. So the sampler runs behind the same gate the player pool uses.

Inside that gate, a caching decision that looks correct in isolation produced the worst bug this subsystem has shipped. The sampler held on to the media wrapper instead of fetching it every tick, because fetching it once per second seemed wasteful. A LibVLCSharp media wrapper takes its own native reference, so the cached wrapper stayed valid after the engine swapped the media out underneath it on a pool thread. It kept answering. It never threw. It returned the dead media's counters, which do not move, which is a total byte stall by every rule in the machine.

The symptom was a healthy channel reported as a dead network. The cost was a source switch away from a working stream, observed live during a field acceptance run on 2026-07-24 at 23:24. The comment that now sits over the fix says it plainly:

// Acquire and dispose the wrapper on every tick. A cached wrapper holds its own native
// retain, so after a pool-thread media swap it would keep returning the DEAD media's
// frozen counters without ever throwing, which reads as a net stall on a healthy stream
// (observed live: [...] 2026-07-24 23:24). One retain/release per second is negligible.

A detector reading stale input does not fail quietly. It produces the same verdict, at the same confidence, about a channel that is playing perfectly, and then it acts on it. Every other failure in a watchdog costs you a stall you did not catch. This one costs you a stall you invented, and the viewer sees the recovery.

Here is what twelve seconds of a freeze actually emits#

Below is a scripted session run through the shipped rule engine on the existing test harness, twelve one-second ticks. It was executed rather than traced by hand, and the driver was deleted afterward. Read the numbers as fixtures. Every snapshot request was answered 300 ms later, as a real host would, since the snapshot budget is one second.

The session opens, reports Playing, and reports one video output. Then:

Tick 1 produces nothing, because one sample is not a delta; it becomes the baseline. Tick 2 is the first evaluated pair, presentation progress is proven, and the machine leaves Starting. It also restates a sample rate it was already using, which is redundant and harmless. The snapshot request on that line is the opening sweep, and the host answered it with real content, 6 percent black pixels at a standard deviation of 41, so the sweep closed.

Ticks 3 to 6 emit nothing at all. That is the intended steady state: a healthy stream produces no effects, so an empty log is the success case.

Tick 7 is where the picture stops. The video stall tag is stamped with 7 s, the machine moves to Suspect, the sampler doubles to 2 Hz, and the probe is armed and asks for a frame. Nothing is declared, because a tag at age zero is not a verdict. Ticks 8 and 10 are the window running, and tick 9 is the probe re-sampling on its own cadence, two seconds after the last request rather than the configured 1500 ms, because it fires on the next message past the due time. Bytes and audio never falter through any of it: this session looks perfect to every rule except one.

Tick 11 is the verdict. The video stall tag reaches four seconds, the demuxer is not under suspicion, audio advanced during the stall and is still advancing, so it is a video freeze and not a total one, and the anomaly age reported to the executor is 4000 ms. Run the identical script with audio stopping at tick 7 as well and the same line declares a total freeze. Run it again with the snapshot timing out instead of returning content and the verdict is unchanged, but its reason string becomes video stalled while audio keeps playing (snapshot timeout observed): the wedged output corroborated, it did not decide.

Tick 12 emits nothing because the machine has handed off. The Playback suite around this code is 108 tests, all passing as of this writing.

A verdict buys five rungs, and one of them is not built#

The verdict is an input to a ladder, not an answer. Five rungs exist: a pause and play kick, which only total freezes get; a full re-apply of the same URL; a switch to the next candidate; a rebuild of the player itself; and give up. A fault within fifteen seconds of tuning enters at the switch rather than the re-apply, because a source that never proved itself does not deserve a retry of the URL that just failed. Mature sessions keep the re-apply first, so a hiccup ten minutes in does not cost the viewer a source change. There is exactly one restart attempt before escalation, because field data showed the second one never helps.

The rebuild rung is honest fiction at the moment. It has no host primitive, it is disabled in shipped settings, and the one condition that routes to it, a stop that timed out and left the native session wedged, currently ends the foreground ladder and lands on give up instead. That is conservative and correct, it is also a gap, and the code says so where it happens.

Between attempts the backoff is exponential with the first attempt free: base times two to the attempt index minus one, capped, then jittered by up to twenty percent either way. In practice, fire immediately, then one second, three, seven, and so on to the cap. Jitter multiplies after the cap rather than before it, so the true ceiling is thirty-six seconds, not the thirty the cap suggests. Small thing, but it is the kind of small thing that makes a graph look wrong at 3 a.m.

The pixels stay on the machine that captured them#

The probe writes two rotating PNG files into the app's own local data folder and overwrites them in place. Nothing uploads them and nothing sends them anywhere; the frame exists long enough to be reduced to two numbers and a hash, and then it is overwritten by the next one. Fault verdicts go to the local log on the same machine, and a channel there is identified by the first sixteen hex characters of a SHA-256, never by a URL.

That last part is why I cannot tell you how common the bright-frozen-frame case is. The telemetry event shapes are written and typed, they name a batched uploader in their doc comment, and that uploader does not exist. Three record types are declared and never constructed anywhere in the code. Every judgment in this article about which failure modes matter is drawn from one field acceptance run, a fault-injection rig and a test suite, and not from a fleet. That is a real limit on the confidence of everything above.

If you would rather use the detector than read about it, it ships on by default in the My TV Player Windows app, and you can create a free account to try it. When it does its job you will not see it: the picture will stutter for four seconds and come back, and nothing will have appeared on screen to tell you a verdict was ever reached.