Skip to main content

Module q42_lex

Module q42_lex 

Source
Expand description

Q42LEX lexicon format. The byte-view reader (q42_lex::Q42LexMmap) and serializers are transport-neutral and compile on wasm32; only the filesystem/mmap handles (q42_lex::Q42LexFile, q42_lex::Q42Lexicon path loaders) stay native. Read .q42.lex reverse-lexicon sidecars (Q42LEX format from qualia-cli ingest).

Structs§

Q42LexFile
Memory-mapped .q42.lex file handle (native targets).
Q42LexMmap
Zero-allocation view over a memory-mapped .q42.lex slice (sorted hash index).
Q42Lexicon
In-memory hash → UTF-8 string map from a .q42.lex file (cold-path loader).

Enums§

LexError
LexiconEntry
Lexicon entry payload for serialization
LexiconKey
Largest UTF-8 prefix of s that fits in max_bytes, never splitting a codepoint. Zero-allocation lexicon key for in-memory lookups

Constants§

DEFAULT_LEX_PAGE_ENTRIES
Fixed upper bound used by the canonical writer. Pages are independently addressable, so an HTTP/IPFS reader only needs this many dictionary records (plus their strings) for one hash lookup.
INDEX_ENTRY_SIZE
LEX_HEADER_SIZE
LEX_MAGIC
LEX_TAG_NAMESPACED
v4 namespaced entry: [u16 ns_id][u16 local_len][utf-8 local]. Only legal inside a LEX_VERSION_V4 page with a namespace table.
LEX_TAG_STRING
Type tags for lexicon entries (1-byte prefix in payload)
LEX_VERSION_PAGED
LEX_VERSION_V4
Namespaced paged format: per-page namespace dictionaries plus 0x04 (ns_id, local) entries. The version word aligns with the Q42 volume generation that first ships it; there is no LEX v3 (ADR 0015).
PAGED_DIRECTORY_ENTRY_SIZE
PAGED_DIRECTORY_HEADER_SIZE
PAGED_PAGE_HEADER_SIZE

Functions§

serialize_paged_string_lexicon
Serialize the v2 paged Q42LEX representation.
serialize_string_lexicon
Serialize a hash → string map into the canonical Q42LEX byte layout that Q42LexMmap reads back (magic, sorted 16-byte index, tagged string blob). This is the write side of the lexicon: the RDF ingest calls it in lossless (“Complete”) mode so every subject/predicate URI and every literal is recoverable via Q42LexMmap::lookup_hash, instead of being hashed away.