deflate · API

deflate.gzip

gzip (RFC 1952) framing over the raw DEFLATE in raw.mojo.

gunzip decodes a whole .gz file: every member of a concatenated stream, each one's header (all five flags: FTEXT, FHCRC, FEXTRA, FNAME, FCOMMENT) parsed and skipped, and each one's CRC-32 and ISIZE checked against what it decoded to. gunzip_at decodes one member and reports where it ended, as inflate_at does for raw streams. gzip writes a single member.

The CRC-32 is the zlib/gzip one (reflected polynomial 0xEDB88320), slice-by-8 as in hashes.mojo's crc32. It is repeated here rather than imported so the tin keeps no sibling dependency: a consumer still needs nothing but -I deflate.mojo/src.

Structs

GzipReader

struct GzipReader

A gzip file decoded a window at a time: every member of it, or just the first.

The streaming form of gunzip, over Inflater: run decodes until the window (output()) holds want bytes, the input runs short or the file ends; compact drops what the caller has used. The input may still be arriving, and input already consumed may be discarded, exactly as for Inflater. Headers, CRC-32s and ISIZEs are checked as gunzip checks them, with the same messages; the CRC is folded in as output is produced, so it covers bytes long since compacted away.

var gz = GzipReader()
var have = 0  # how much of `data` has arrived
while True:
    var st = gz.run(Span(data)[:have], have == len(data), len(gz) + (1 << 20))
    parse(gz.output())
    _ = gz.compact(len(gz))
    if st == InflateStatus.FINISHED:
        break
    if st == InflateStatus.NEED_INPUT:
        have = ...  # wait for more

Implements Movable, Sized

Fields
inflater: Inflater: Holds the window and the DEFLATE state of the member being decoded.
state: Int
multi_member: Bool
member: Int
member_at: Int: Absolute input offset of the current member's header (for messages).
crc32: _Crc32
crc: UInt32
crc_at: Int: output()[crc_at:] is not in crc yet.
member_out: Int
padding_at: Int

__init__

def __init__(out self, capacity: Int = Int(1048576), multi_member: Bool = True)

capacity is the window's starting size. With multi_member false, decoding stops after the first member, and whatever follows it is left alone (input_used() says where it ended).

output

def output(self) -> Span[UInt8, origin_of(self.inflater.out.buf)]

Everything decoded and not yet compacted away.

__len__

def __len__(self) -> Int

compact

def compact(mut self, keep_from: Int) -> Int

Inflater.compact: drop output()[0:keep_from] as far as back-references allow; return how many bytes went.

total_out

def total_out(self) -> Int

members

def members(self) -> Int

Members decoded and verified so far.

finished

def finished(self) -> Bool

finish

def finish(deinit self) -> List[UInt8]

The window as a list, consuming the reader.

input_used

def input_used(self) -> Int

How many bytes of data the reader is done with; once finished, the offset just past the last trailer it read (or padding).

discard_input

def discard_input(mut self) -> Int

Inflater.discard_input: forget the input before input_used() and return how many bytes that is.

total_in

def total_in(self) -> Int

run

def run(mut self, data: Span[UInt8], complete: Bool, want: Int = Int.MAX) -> InflateStatus

Decode from data until the window holds want bytes: OUTPUT, NEED_INPUT or FINISHED, as Inflater.run. Raises what gunzip raises, once complete says no more input is coming (and on corruption before that).

Functions

crc32_backend

def crc32_backend() -> String

armv8-crc or slice-by-8: which CRC-32 this build uses.

gunzip

def gunzip(data: Span[UInt8]) -> List[UInt8]

Decompress a gzip file: all of its members, concatenated.

Each member's CRC-32 and length are verified. Zero bytes after the last member are padding (as gzip(1) and Python's gzip accept); anything else there raises, as does an empty input.

gunzip_at

def gunzip_at(data: Span[UInt8], mut end: Int) -> List[UInt8]

Decompress the one gzip member at the start of data, verifying its CRC-32 and length, and set end to the offset just past its trailer.

For walking a multi-member file one member at a time, so only one member's output is held at once:

var pos = 0
while pos < len(data):
    var end = 0
    var part = gunzip_at(Span(data)[pos:], end)
    pos += end

gzip

def gzip(data: Span[UInt8], level: Int = Int(6)) -> List[UInt8]

Compress data into a single gzip member.

level 0 writes stored (uncompressed) DEFLATE blocks; 1 to 9 all use deflate's one fixed-Huffman strategy, so they produce the same bytes. The header carries no name, comment or timestamp (MTIME 0, OS 255).