Layer 1 · Plain-Language Overview

Data Formats & Unified .q42 v4 Volumes

QualiaDB turns human-readable knowledge graphs, rules, spreadsheets, and structured streams into self-contained, verifiable binary volumes. Whether you start from Turtle, N-Triples, N-Quads, Streaming N3, JSON-LD, CBOR-LD, or CSV, the ingestion pipeline compiles your data into a single portable .q42 v4 file that works identically on desktop, server, edge devices, and browser HTTP Range views.

Every statement is stored as a fixed 48-byte record (NQuin) protected by a canonical 5-field parity check, paired with an embedded namespaced dictionary (Q42LEX v4) and optional prose catalogue (PCAT v1). Physical simulations, fields, and neural weights use purpose-built 64-byte aligned companions (.10d, M64, EMF64, and P64) so graph indices remain compact and deterministic. Those companions travel together in a QBDL-profile Hyper Media Container under governed QDNF streaming.

The Attested RDF → Q42 v4 Pipeline

The streaming importer compiles source graphs directly into a unified .q42 v4 volume with embedded Q42LEX v4 namespace pages, BIDX / FIDX / PIDX lookup tables, and LZ4 SuperBlocks. It emits structured progress, supports resumable job directories, and proves encoded-set equivalence on completion.

qualia-cli import ./data/example.ttl ./data/example.q42 --job-dir ./jobs/example --progress json
qualia-cli ingest-job status ./jobs/example
qualia-cli q42 verify ./data/example.q42 --level full
qualia-cli verify-graph --encoder=import --input ./data/example.ttl --dataset ./data/example.q42
Layer 2 · .q42 v4 Unified Volume Architecture

.q42 v4 Unified Volume Architecture

A .q42 v4 volume (Q42_VERSION_V4 = 4) packs the 256-byte volume header, block and predicate indices (BIDX, FIDX, PIDX), LZ4-compressed SuperBlocks of 48-byte NQuin records, the paged Q42LEX v4 dictionary, and optional PCAT v1 prose sections into a single file without sidecar .c.q42 or .q42.lex artifacts.

48-Byte NQuin Layout & Canonical 5-Field XOR Parity

parity = subject ^ predicate ^ object ^ context ^ metadata

Every semantic statement in QualiaDB is lowered into a cache-line-friendly 48-byte NQuin consisting of six little-endian u64 words. The sixth word stores the canonical 5-field XOR parity over all five semantic and operational words (subject ^ predicate ^ object ^ context ^ metadata), ensuring that bit-flips in routing lanes, Lamport logical clocks, or modality payloads are detected alongside core quad terms.

Bytes Field Bit Layout Semantic & Runtime Role
0..8 subject [63] MSB flag · [0..62] FNV-1a / Webizen ID Entity, concept, or actor anchor (TAG_WEBIZEN = 0x8 in top nibble for person-controlled actors).
8..16 predicate [63] Defeater/MSB · [8..62] IRI hash · [0..7] u8 opcode Property IRI hash shifted left 8 bits, paired with modality opcodes (0x10+ for Deontic, Epistemic, Paraconsistent, LTL).
16..24 object [63] did:q42 ptr · [60..62] type tag · [0..59] payload 60-bit lexicon token (0b000), inline xsd:integer (0b001), xsd:decimal × 10⁶ (0b010), or xsd:boolean (0b011).
24..32 context [56..63] Sensitivity class · [0..55] World/Graph hash Named graph, agreement, or epistemic world DID hash plus sensitivity tier (PUBLIC=0, RESTRICTED=1, CLASSIFIED=2).
32..40 metadata [61..62] RoutingLane · [32..60] Lamport clock · [0..31] payload Routing lane (00=Passthrough, 01=Commons, 10=Bilateral, 11=Spatial), 29-bit Lamport clock, and expiry/confidence weight.
40..48 parity [0..63] 5-field XOR checksum Canonical 5-field XOR fold: subject ^ predicate ^ object ^ context ^ metadata.

Namespaced Paged Q42LEX v4 (ADR 0015)

0x04 Entry

Multi-ontology volumes share recurring IRI prefixes (such as http://www.w3.org/ns/shacl# or https://schema.org/). Q42LEX v4 (q42_lex_ns.rs) splits IRIs into a per-page namespace table and compact (ns_id, local) pairs:

  • Per-page self-sufficiency: Each lexicon page embeds its own namespace dictionary so a browser client can decode any 60-bit hash via a single HTTP Range request without fetching a global prefix table.
  • Zero-alloc parts lookup: Q42LexMmap::lookup_parts(hash) returns (namespace_prefix, local_name) slices directly from the mapped page.
  • Bounded term invariants: Verbatim and reconstructed terms enforce a strict 65,535-byte (u16) maximum in Q42LEX; longer prose routes to PCAT.

PCAT v1 Payload Catalogue (ADR 0014)

0x0080

Set via FLAG_PAYLOAD_CATALOGUE = 0x0080, the PCAT v1 section stores oversized UTF-8 prose, treaty articles, and long-form annotations out-of-line:

  • Unpolluted BIDX & SuperBlocks: Keeps the hot 48-byte NQuin stream and block index dense for sub-millisecond graph traversals.
  • Dual integrity verification: Every PCAT entry is verified by whole-section CRC-32C and per-record SHA-256 content digests.
  • Strict text-only gate: Binary media MIME types (image/*, audio/*, video/*, model/*) are rejected at ingest in favor of QBDL-profile Hyper Media Containers and .10d companions. The ZIP document-archive .hmc profile is legacy and is not a streaming target.

Publication Policy Flags (Sanctuary vs Commons)

Header Flags

Governance is enforced at the byte-header level so personal and clinical graphs cannot be accidentally syndicated to public mirrors:

  • FLAG_SANCTUARY = 0x0040: Hard publication lock for NaturalAgent personal, clinical, fiduciary, or guardianship data. Refuse-on-export gates in qualia-cli, HTTP range servers, and WebTorrent seeders abort syndication when this bit is set.
  • FLAG_PERMISSIVE_COMMONS = 0x0020: Marks open reference ontologies and public-good volumes cleared for replication across peer caches and CDN range mounts.

ExactnessProfile & 256-Bit 4-Prime CRT (ADR 0003)

M > 2.028×10⁷⁷

Scientific, financial, and deontic workloads declare their numerical contract via ExactnessProfile (ADR 0003), pairing IEEE-754 fast lanes with a deterministic 256-bit 4-Prime Chinese Remainder Theorem (CRT) cascade:

  • Composite modulus $M = p_1 p_2 p_3 p_4 > 2.028 \times 10^{77}$: Four 64-bit NTT-friendly primes reconstruct exact 256-bit integer and rational invariants without heap-allocated bigints in hot loops.
  • Zero-drift verification: Guarantees bit-identical evaluation across CPU architectures and GPU/WASM targets when verifying conserved quantities or audit proofs.
Layer 3 · Developer Formats & Streaming Compilers

Streaming Wire Formats & Rule Languages

CBOR-LD 5-Element Tuple Stream

QualiaDB's native wire format (cbor_compiler.rs) encodes each statement as a compact 5-element CBOR array [subject, predicate, object, context, metadata_u64]. Unlike 4-element quad streams, the 5th element preserves the full 64-bit metadata word (routing lane, 29-bit Lamport clock, and modality payload), and the stream parser advances an exact byte-length cursor so concatenated frames decode with zero heap allocation:

// 5-element CBOR-LD tuple: [s, p, o, c, metadata_u64]
// Major type 4 (array of length 5 = 0x85) followed by five u64/text items.
let (quin, bytes_consumed) = parse_cbor_ld_tuple_with_cursor(frame_slice)?;
assert_eq!(
    quin.parity,
    quin.subject ^ quin.predicate ^ quin.object ^ quin.context ^ quin.metadata
);
qualia-cli import data.cborld data.q42 --progress json

Streaming N3 Modal & Defeasible Rules

The zero-allocation n3_parser.rs engine natively lowers four rule arrows plus Answer Set Programming (#asp {}) and diffusion (qualia:diffuse {}) blocks directly into Webizen VM bytecode and Deontic NQuin records:

N3 Arrow RuleType Execution Semantics
=> Strict Classical modus ponens forward chaining.
~> Defeasible Holds presumptively with optional weight (0.8) unless overridden.
^> Defeater Sets DEFEATER_BIT (bit 63 of predicate) to block a matching defeasible norm.
-o Linear Linear logic transition: consumes the premise token upon firing.
@prefix ex: <http://example.org/> .
(0.85) { ?x a ex:EmergencyVehicle } ~> { ?x ex:mayPassRedLight true } .
{ ?x ex:hasRevokedPermit true } ^> { ?x ex:mayPassRedLight false } .
{ ?token a ex:SingleUseGrant } -o { ?token a ex:ConsumedGrant } .

Mapped CSV Ingestion

Tabular files map through SHACL/Turtle shapes directly into a unified .q42 v4 volume with inline XSD integer, decimal, and boolean tags in the object field:

name,age,city
Alice Smith,30,New York
Bob Jones,35,Los Angeles
qualia-cli ingest csv people.csv --map people-shape.ttl
# writes unified people.q42 v4 volume

JSON-LD & Contextual JSON

JSON-LD documents expand @context prefixes into Q42LEX v4 namespace pages and preserve named graphs (@graph) in the context field:

{
  "@context": { "ex": "http://example.org/" },
  "@graph": [
    { "@id": "ex:alice", "@type": "ex:Person", "ex:name": "Alice Smith", "ex:age": 30 }
  ]
}
qualia-cli import data.jsonld data.q42 --progress text

Standard RDF Serializations (Turtle, N-Triples, N-Quads, RDF/XML)

All W3C RDF serializations pass through the same q_hash and Q42LEX v4 encoder so cross-format imports can be verified for semantic equivalence via verify-graph --encoder=import:

Turtle (.ttl) & N-Triples (.nt)

@prefix ex: <http://example.org/> .
ex:alice a ex:Person ;
    ex:name "Alice"@en ;
    ex:age 30 .

N-Quads (.nq) & Resumable Jobs

# Local or remote resumable import into .q42 v4
qualia-cli import data.nq data.q42
qualia-cli import catalog.q42 --url https://example.org/catalog.ttl --job-dir ./jobs/catalog
qualia-cli ingest-job continue ./jobs/catalog
Layer 4 · 10D Spatial, Physical & Model Binary Containers

10D, Physical Matter, Field & Weight Containers

Volumetric geometry, physical strata, electromagnetic fields, acoustic takes, and neural network weights are stored in dedicated 64-byte-aligned binary containers (.10d, M64, EMF64, QAC, and P64) that travel alongside .q42 volumes inside a QBDL-profile Hyper Media Container. An HMC may hold many entries of the same family. Selection is over entry identities, optional time and region, and authenticated Q42 HTTP Range windows. The ZIP document-archive .hmc profile is a distinct legacy document and is not a streaming target; filename extension is never the discriminator. See QBDL mixed HMC streaming.

Container10D (.10d Living Container v1)

container_10d.rs

Encodes the 10-dimensional spatiotemporal and modal manifold substrate with a normative header, axis-role taxonomy, metric-completeness descriptor, and typed chunk stream:

  • PROC1 — Procedural CSG / SDF construction operators & parameters
  • EONT1 — Entity-ontology bindings linking geometry to NQuin hashes
  • MRP1 — Morph target & deformation field descriptors
  • SKL1 — Skeletal hierarchy & kinematic articulation rigs

M64 (64-Byte Matter Chunks)

m64_matter.rs

Canonical 64-byte aligned container for physical strata, point clouds, material constitutive tensors, and holographic volumetric entities.

Each 64-byte record aligns cleanly with a single CPU cache line and GPU storage-buffer stride, enabling zero-copy upload to WebGPU / Vulkan compute shaders without heap deserialization.

Q42EMF64 (v4 8-Mode EMF & v3 Display)

emf_container.rs

Unified 128-byte envelope & directory framing for electromagnetic/wave-optical fields (.emf) across 8 typed modes (0x0100..=0x0107: field, ray, Yee FDTD, spectrum, Mueller, I/Q, dose, biofield) plus explicit v3 64-byte display bricks.

Paired with Etalon Yee FDTD solvers, Hermitian Cloude coherency validation, Kramers–Kronig causal dispersion, and BLAKE3-256 / Bao integrity gates.

.10d SKL2 v2 (64B Joint Descriptor)

FaunaRig · 24-Joint

64-byte cache-line-aligned skeletal joint descriptor for 24-joint bilateral FaunaRig hierarchies, encoding Quaternion Swing-Twist reach cones, 2:1 scapulo-humeral rhythm coupling, and biarticular tendon attachments.

.m64 Mode 0x05 (M64BioTissueRecord)

64B Voxel Record

64-byte biological tissue voxel record (Modes 0x00..0x05) storing cortical/trabecular vBMD, BV/TV, fabric tensor M_fabric, muscle HU/IMAT myosteatosis, and fracture callus stage 0..4.

Q42QAC64 (QAC v1 Acoustic & Spatial Audio)

audio/qac/

Self-describing v1 containers (.qac) spanning 8 modes (0x0100..=0x0107): PCM, STFT, SOFA HRIR, ACN/SN3D Ambisonics, ADM objects, modal bodies, pressure-velocity probes, and BS.1770-5 perceptual loudness/MFCCs, rejecting legacy UA4Q v1–v3 fail-closed.

Unified with schemas 200..=210, rational clock discipline, and etalon-signal SIMD/DSP bridges.

DICOM & Acq (PS3.10 Preservation & Tomography)

medical/ & sensing/

Exact Part 10 preservation with RLE, JPEG-LS, JPEG 2000, JPEG XL, and Deflate codecs alongside NIfTI-1/2 bridges, iterative tomographic reconstruction (FBP/ART/SART/OSEM/MLEM), and Ed25519 calibration chains.

Includes D01..D06 acquisition custody, range admission, and original-byte round-trip verification.

P64 (64-Byte Aligned Weight Container)

p64_weight.rs

Identified by the distinct p64\0 magic header, P64 packages quantized GGUF/Safetensor model weights into 64-byte headers, tensor directory entries, and manifold records.

Enforces both metadata and per-tensor CRC-32C checksums with byte-exact disk round-trip verification and direct mmap residency.

Reproducibility & Encoded-Set Verification

Every ingested .q42 v4 volume can be inspected for structural integrity, Q42LEX v4 coverage, PCAT digests, and full encoded-set equivalence against the original RDF source within explicit memory and scratch-disk bounds:

# Inspect physical Q42 v4 sections (Header, BIDX, FIDX, PIDX, Q42LEX v4, PCAT v1)
qualia-cli q42 inspect data.q42
qualia-cli q42 verify data.q42 --level full

# Prove that the encoded graph agrees with the source input within explicit bounds
qualia-cli verify-graph --encoder=import --input data.ttl --dataset data.q42 --memory-mib 32 --temp-gib 24

Publish benchmark and equivalence receipts alongside the command, input digest, hardware profile, and raw JSON receipt so every claim remains independently auditable.