Truncated gzip: the guide file that downloaded successfully and was still corrupt

var truncated = Gzip(Payload)[..^8];
await using var input = new MemoryStream(truncated);
await using var inflated = new MemoryStream();
await using (var gz = new GZipStream(input, CompressionMode.Decompress, leaveOpen: true))
{
    await gz.CopyToAsync(inflated);
}

Assert.Equal(Encoding.UTF8.GetByteCount(Payload), inflated.Length);

Eight bytes off the end of a gzip file, and the decompressor hands back the whole payload and raises nothing. That assertion sits in our test suite because the premise had to be pinned before anything could be built on it. No exception-based approach could have caught this, which is why the guard reads the file's own trailer instead of waiting to be told.

The eight bytes that went missing are the trailer, the only part of the file that could have disagreed with what came out.

Corrupt those eight bytes instead of removing them and the same decompressor throws at once. I flipped a single bit at each of the eight trailer positions, and every one raised InvalidDataException with nothing delivered. The reader checks the trailer scrupulously when it can see one. Truncation is dangerous because it takes the trailer away instead of damaging it, and a check that is never reached is the same as no check.

The ingest described below runs over guide addresses we curate, on files their publishers serve to anyone. A guide address you supply is downloaded and read on your own device, and we hold none of that data. The two decompressors are the same file ported between the two codebases, with the default ceilings forked eight to one, so one article covers both.

A member that ends early can still end cleanly#

A gzip file is a series of members, and each member is a ten-byte header with optional extras after it, a deflate stream, and an eight-byte trailer carrying a CRC32 followed by ISIZE, the uncompressed size modulo 2^32. Inside the deflate stream every block carries one bit saying whether it is the last block of the data set. The inflater stops when it has read that bit and that block has ended. Nothing in that sequence obliges it to read the trailer, and if the trailer is absent there is nothing to signal that one was expected. The stream is over because the stream said it was over.

The obvious next thought is that a deeper cut, landing inside the deflate data, must at least run the reader out of input somewhere it notices. It does not, and that surprised me. I removed 1 through 50 bytes from the end of a single-member file and the decompressor returned partial output at every depth without a word: 90 bytes back at a one-byte cut, 67 at a twenty-byte cut, 48 at a forty-byte cut. There is no depth at which the runtime starts complaining.

So what makes the member-boundary cut the dangerous one is the shape of the output, not the volume of the silence. Cut deeper and the XML reader is usually handed an element that never closes and rejects the file, which is exactly what our own comment on the household-wide guide path says. Cut at the boundary and the output is byte-for-byte a prefix of the real document, well-formed all the way to its last complete element, and the two fields that would have contradicted it are the two that were still in flight when the connection went away. A guide that arrives that way looks like a source with thin listings, which is a different complaint from a guide whose programs are all shifted by an hour and wants a different repair.

Cuts we can reason about come from HTTP, which frames a body two ways. A body short of a declared Content-Length is a broken response and the client says so out loud: against a socket that promised 100 bytes and sent 40, .NET raised HttpRequestException wrapping HttpIOException, "The response ended prematurely." Send the identical 40 bytes with no length header and close the connection, and the same client returns them with no error, because a close-delimited body is complete by definition unless the connection itself reported a fault. A proxy that gives up mid-body produces exactly that: a stream that ends, on purpose as far as any layer below can tell.

The check hunts for a number it already knows#

Here is the whole of it. It repays reading a line at a time, because almost every line is a concession to something real.

const int MinGzipLength = 18;
const int TailWindow = 512;

if (!input.CanSeek || input.Length < MinGzipLength || written is <= 0 or > uint.MaxValue)
{
    return;
}

var take = (int)Math.Min(TailWindow, input.Length);
var tail = new byte[take];
input.Position = input.Length - take;
await input.ReadExactlyAsync(tail, cancellationToken).ConfigureAwait(false);

var expected = (uint)written;
for (var i = 0; i + 8 <= tail.Length; i++)
{
    if (BinaryPrimitives.ReadUInt32LittleEndian(tail.AsSpan(i + 4, 4)) == expected)
    {
        return;
    }
}

var trailing = BinaryPrimitives.ReadUInt32LittleEndian(tail.AsSpan(take - 4, 4));
if (trailing > written)
{
    throw new InvalidDataException(
        "Gzip stream is truncated: the trailer reports more bytes than were decompressed.");
}

The 18 is a header plus a trailer with nothing between them, so it is a floor for "too short to hold a trailer at all", not a validity test. Nothing real gets that close: the smallest member this encoder produces for a single byte of input measures 21. The seek requirement is the first genuine limit: a caller streaming straight through cannot be checked, and the check stands down instead of rejecting those callers. All three of our ingest paths decompress from a file or a buffer, so all three are checked.

The > uint.MaxValue bail is the second limit, and it is structural. ISIZE is the uncompressed size modulo 2^32, so once a run has inflated four gibibytes the field cannot be compared with anything. That is the width of the field talking: the check switches itself off per run, quietly, for exactly the largest guides, and any absolute ceiling set above that point buys headroom by spending the truncation check.

Then the scan. It reads the last 512 bytes, walks every offset in that window, and reads the second of each pair of four-byte words as a candidate ISIZE. A match anywhere means a member of exactly the size we inflated ends somewhere in that tail, which is what tolerates a padded file. It has a cost: 505 candidate offsets against a four-byte value give a false match roughly one time in eight and a half million if the tail bytes are unrelated to us. That is arithmetic, not measurement, and the windows overlap, so they are not independent draws.

If no offset matches, the last four bytes are read as a number and compared. A trailer reporting fewer bytes than we wrote is consistent with the last of several concatenated members, so it passes. A trailer reporting more cannot come from a complete file, so it fails. That asymmetry is the entire verdict, and it is where the honesty has to be. When the trailer is gone, those four bytes are compressed data being read as a length, and the check works only because an arbitrary 32-bit value is usually larger than the number of bytes we inflated. At 40 MB inflated that slips about one time in a hundred and seven. At two gigabytes inflated it is 0.47, and because the number in that fraction is what we actually inflated and not the guide's true size, the odds are worst for the cut that matters most: the one that arrives almost complete.

So the guard gets weaker as guides get larger, which is the wrong way round, and the largest guides are the ones whose downloads take long enough to be cut.

Two of the three jobs are already done by the runtime#

The loop that iterates concatenated members carries what I long believed was the padding logic in its exception handler.

catch (Exception ex) when (ex is InvalidDataException or EndOfStreamException)
{
    // Trailing padding after a member that ended cleanly is tolerated;
    // some upstreams append junk after the gzip trailer. Anything else
    // is a truncated download, and swallowing it would write a partial
    // guide, hash it as the content, and then skip every later fetch as
    // "unchanged" until the upstream bytes happen to move.
    if (completedMembers == 0 || guard.Written > writtenBefore)
    {
        throw;
    }

    break;
}

I instrumented a faithful replica of that loop over twelve input shapes, as both a memory stream and a file stream, and the catch clause fired zero times in all 24 runs. GZipStream concatenates members itself, so two members arrive as one 180-byte read and the loop counts a single member. It tolerates trailing junk itself, so a padded file never raises either. On every shape our test suite pins, the loop runs exactly one iteration and the trailer check makes all three calls: accept the padded file, reject the truncated one, reject the file whose second member was cut. The handler is a leftover, and its comment is the only place the padding rule is written down.

That matters because the real rule is narrower than the comment. The padding tolerance comes from the 512-byte scan, so it lasts exactly as long as the true trailer stays inside the window. I swept padding lengths from 0 to 520 bytes: 504 bytes of padding are accepted and 505 are rejected as truncated. Past that the check falls through to the last four bytes of padding and reads them as a length. Zero-filled padding survives the boundary only by accident, since four zero bytes read as a length of nothing and the accept-smaller rule waves it through. Padding of 0xFF at the same length reads as 4,294,967,295 and gets rejected.

I have never sampled for the behavior that comment describes. Upstreams appending junk after the trailer is an assertion in our source and a test name, not an observation.

Two ceilings live next to the trailer check#

The same copy loop enforces two limits that have nothing to do with truncation: an absolute cap on bytes written, and a ratio cap of decompressed against compressed, which catches a small file that inflates into something that is not a guide. Both are tested inside the loop, on every 80 KB block, so a bomb fails partway through instead of after the whole thing is written.

I am not publishing our current numbers, and our own documents are the reason. The service specification, dated 19 July 2026, still records a pair of fetch caps that the shipped configuration has since passed. What moved them was one real national guide that turned out to run about 200 MB compressed and expand to roughly two gigabytes: the compressed cap of the day sat exactly on that line, and the decompressed cap rejected the file outright. A number in an article ages the same way a number in a specification does. So: cap the absolute size, cap the ratio, test both while writing, and keep the absolute cap below the point where ISIZE stops being comparable, or the headroom costs you the truncation check.

Two details carry more than the values. The ratio cap applies only when the caller knows the compressed size, so a caller that passes zero gets the absolute cap alone, and our truncation tests do exactly that on purpose. And the household-wide guide path on the device never calls the shared decompressor at all: it wraps a raw gzip stream in a counting stream that enforces a byte cap and no ratio cap, then calls the trailer check directly. That one is a gap rather than a decision, and I could not find it written down anywhere.

The gate that turns a bad body into a permanent one#

Now the half nobody writes about.

Guide ingest is expensive and most fetches change nothing, so the fetch is conditional: an If-None-Match and an If-Modified-Since from what the last run stored, and a 304 ends the run. Validators are not enough on their own. One public index we resolve serves an unquoted ETag, which does not round-trip, so conditional requests against it never short-circuit. The authoritative gate is a SHA-256 of the decompressed bytes, compared against the hash stored on the source row.

A content hash is the better gate in every way that matters. It is computed locally, and no upstream can fool it by lying about its validators or forgetting them. It also has no escape hatch, and that is the whole problem. The hash is taken at the fetch, over the decompressed temp file, before a parser has looked at a single element, so the gate cannot tell a body that merged cleanly from one that blew up two steps later.

Three steps and the failure is permanent:

1. A body arrives that should not be accepted, cut short or merely unusable.
2. Its hash is computed and stored as the source's content hash.
3. Every later fetch of the same bytes hashes to the same value and returns "skipped, unchanged" without the body reaching a parser again.

The recovery condition is that the upstream bytes happen to move, the phrase the decompressor's own comment uses. For a national guide regenerated nightly, that is a day. For a file nobody regenerates, it is never, and the retry machinery that exists for transient failures cannot help, because from the second run onward there is no failure to retry.

The last turn of the screw: this state reports healthy. A skipped-unchanged run means the upstream is reachable and the guide is current, so the health job counts it as a pass and carries the previous channel count forward as corroboration. A source frozen on the hash of a body that never merged looks, on every dashboard we have, like a source that is fine.

We had the write in the wrong place#

Our own version of this had nothing to do with truncation. It was ordering.

Validators used to be written immediately after the fetch, before the parse and before the merge. A run that fetched a good body and then failed later, on a database hiccup, on an out-of-memory kill, on a maintenance job that at the time dropped every per-run staging table including one an in-flight ingest was still using, recorded the run as failed and stored the new hash anyway. The next cycle fetched the identical bytes, matched, and skipped. The source went stale and reported healthy, and no amount of retrying touched it.

The staging-table collision is fixed on its own account now: maintenance skips any table whose run is still marked running and younger than the orphan threshold. The ordering fix is one moved call, and the comment where the write now lands says why:

// Validators persist only after the merge has succeeded. Writing
// them right after the fetch made any parse or merge failure
// permanent: the run was marked failed, but the new hash was already
// stored, so every later fetch of the same bytes short-circuited to
// skipped_unchanged and the source never re-merged until upstream
// changed. With the write here, a failed run leaves the old
// validators in place and the next cycle retries the full ingest.

The 304 path keeps its own write, and that one is safe, because a 304 proves the content matches what is already merged and refreshing the validators cannot mask anything. The rule is to write validators only once the body has finished being useful to you.

The regression test is the shape I would want on any gate like this. It stands up a real upstream serving 22 characters of broken XML, asserts the run failed with the stored content hash still null, then runs it again over identical bytes and asserts it failed a second time instead of skipping. A mirror test proves the skip optimization still works after a success, so the fix cannot be "stop caching" by accident.

We found the original by reading our own code during a platform audit in August 2026, not from a report, and that is the uncomfortable part. The audit produced 110 findings and rated this one High. I cannot tell you how many times it fired in production, because the state it produces is indistinguishable from health, and the guide it serves is old instead of absent.

The device has the same shape and no hash#

The server half is fixed. The device half is not, and I would rather write that down than let this page imply symmetry.

Our Windows app's per-source guide import stores its ETag and Last-Modified one line after the fetch returns, before decompression, before the parse, before anything reaches the local store. No code path clears them afterwards. The failure sink deletes the half-written guide generation and leaves the fetch metadata alone; disposal touches neither. So an upstream that honors conditional requests can serve a body we fail to parse, receive our refreshed validators, and answer the next request with a 304 that merges nothing.

Two things soften it, and neither closes it. The device keeps no content hash at all, only the two validators, so the trap needs an upstream that actually honors conditional requests, and the ones with broken validators are accidentally immune. And a failed import leaves the previous guide readable, because the generation pointer is swapped only on completion, so the visible symptom is a guide that stops advancing instead of a guide that vanishes. That is a better failure than the server's, and it is still the same bug.

The household-wide path, the one that pulls a whole household's guides into a single store, gets this right in an interesting way. It cannot inflate to a buffer and then check, because the aggregate is too large to hold, so it parses straight out of the gzip stream and validates the trailer afterwards. Before validating it drains whatever the XML reader left behind, because the count handed to the check has to be the whole member rather than however far the reader happened to get. Parse first, drain, then check. That ordering is a nuisance to read and it is the only one that works.

What this buys you#

Our Windows app runs this decompressor on your machine, against the guide address you gave it, and no stream passes through us at any point. A guide that fails the trailer check leaves the one you already have on screen, which is the behavior you want from a background refresh you were not watching. You can create a free account and point it at a source you already have.

If you take one habit from this page, make it distrusting the word "unchanged". A cache gate that keys on the content of a body is the correct gate and the one I would build again, and it is also a gate with no way to say "I accepted that body and then everything after it went wrong." Whatever you compute at the door, store it at the point where you know the work succeeded, and give your health check something to look at that is not the absence of an error.