Qualia-DB Architecture

The 3-Core Triad, Webizen VM, Rights Ontology, and the Principal-Agent Ecosystem. Branch: 0.0.29 | Last updated: 2026-08-02

Qualia-DB abandons traditional cloud-centric, string-heavy JVM architectures in favour of a specialised 3-Core Triad built with ruthless mechanical sympathy (512 MB RAM floor). Raw multi-modal data (audio, camera feeds) would immediately breach this floor, so the ecosystem forces an Orchestration Sieve: the Primary Agent must coordinate deterministic tools (OpenCV, Audio DSP) to strip noise, extract contours, and build optimised files before handing them to the local LLM or the database.

The 3-Core Triad

1. Zero-Allocation Ingestion

CBOR-LD gatekeeping and WASM OPFS bridging bypass heap-saturation attacks, writing natively to disk. The qualia-cli ingest pipeline uses Rio multi-thread streaming, sorting Quins by subject before writing LZ4-compressed SuperBlocks, so the resulting .q42 file supports O(1) block-range lookups via a companion .q42.bidx index.

Supported ingest formats: CogAI Cognitive AI Chunks (.chk text — W3C CG ACT-R chunks-and-rules), CBOR-LD, N-Triples, N-Quads, Turtle, TriG, N3, JSON-LD, RDF/XML.

Ingest is honest and lossless by intent: IngestMode::Complete interns every literal (with language / datatype tags preserved), while IngestMode::StripLiterals is an explicit, reversibility-aware reduction — never silently discarded and never described as “compression” unless it round-trips. CBOR-LD is the native serialisation of the graph.

⚠ Capability envelope migration: CogAI Chunks remain .chk text files. QCHK capability envelopes are migrating to .qchk; legacy .chk QCHK files are compatibility-only. Use the QCHK magic bytes to detect old profile files during migration.

2. GPU Sieve (Geometric Pruning)

Graph nodes are mapped into Minkowski space within continuous 128 KB memory-mapped QualiaSuperBlocks. The GPU calculates bounding-hull collisions to retrieve data at sub-microsecond speeds without loading unrelated blocks. The WGSL compute shader (shaders/fused_tensor_contraction.wgsl) runs 64 threads/workgroup across DirectML / Vulkan / Metal / WebGPU via wgpu 30 (migrated from 29; naga 30, PollType API, and the wgpu-30 RequestAdapterOptions surface).

2.5. 10D Volumetric Tensor System (Zero-Heap Geometric Processing)

The 10D tensor system [q, v, w, x, y, z, t, α, μ, σ] provides absolute mechanical sympathy across heterogeneous hardware (edge phones to A2000 GPUs to scarce QPUs). It maps neuro-symbolic human-centric logic into raw geometric physics simulations executable via SIMD, GPU texture units, or asynchronous Ground-State Resolvers.

Coordinate System:

2.6. Qualia WASM portal & U3 AcousticPlane

Browser and edge embeds ship as one module (docs/pkg/qualia/qualia_bg.wasm): engine + wgpu viewport + U3 symbolic audio. JavaScript is glue only (QualiaPortal::tick). Audio is spectral-first: STFT sidecars + 64-bit Sonic Tokens + parametric DSP — never LLM PCM. Operator manual: qualia-wasm-portal.md. ADR: adr/0007-u3-acoustic-plane-symbolic-audio.md.

Hardware-Tier Dispatching:

Zero-Heap Guarantees:

Ground-State Resolver (GSR):

3. The Webizen VM (Logic Unification + Advanced Compilation)

Data filtering is not enough — human-centric databases must execute logic. Nested N3 implication rules, SHACL shapes, and defeasible logic are compiled by the WebizenCompiler (and a dedicated shacl_compiler) into compact L1-cache bytecodes. The VM supports:

The Query Engine (SPARQL 1.1 over quins)

Alongside the Webizen VM’s logic resolution, Qualia-DB runs a from-scratch, zero-allocation SPARQL 1.1 engine directly over the packed NQuin store (crates/qualia-core-db/src/sparql_library/). Parser → planner → physical executor is one in-process path shared by the daemon HTTP/WebSocket services, the MCP server, the CLI, and the desktop shell.

The engine carries 304 in-crate tests. The QISP immersive profile (immersive/) layers versioned webizen.org IRIs, Tensor10D geo/spectral predicates, and a typed function registry on top of this surface. See query-engine/ for the extension and DID-integration manuals.

Lazy SuperBlocks, LZ4 Compression & Massive Datasets

Core data lives in 40,960-byte SuperBlocks (exactly 10 disk sectors) with high-density LZ4 compression. The engine lazily scans only 16-byte headers and seeks over irrelevant blocks in O(1) time, decompressing on demand. “Missing” local blocks can be streamed from peers through the sync layer; the currently implemented daemon path is libp2p request-response over TCP + Noise + Yamux, while broader WebRTC transport language elsewhere in the repo remains a future or adjacent profile. This lets 50 GB+ semantic ledgers run comfortably inside the 512 MB floor.

Persistence is behind a cross-platform driver abstraction — open_storage(data_dir) in storage_driver.rs selects the backend for the host: ZNS NVMe zone-append, APFS clonefile, WinNVMe, or a portable Mmap fallback (and a real OPFS-backed wasm_storage counterpart on the browser target). Every write goes through a tamper-evident, Ed25519-signed Write-Ahead Log (wal.rs) that also records Webizen VM conduct violations.

Real-world example: WordNet (523 MB RDF) → 74.6 MB .q42 · 5.56 M quins · 6.5 ms first-query latency via demand-paging with no full load.

Fractal Sharding & Swarm AI Compute

While Qualia-DB rigorously enforces the 512 MB floor, it is capable of extreme horizontal scale on high-end hardware. Rather than bloating a single instance, it uses Fractal Sharding: on a rig with 64 GB RAM and 12 GB+ GPU, the daemon detects surplus hardware and dynamically spins up dozens of parallel, mathematically isolated 512 MB worker cells.

qualia-cli daemon --workers 100 --compute-swarm

This Swarm Orchestration enables massive parallel execution, deep neural-network offloading, and background Sleep-Cycle AI Compute without compromising core mechanical sympathy.

The LLM Inference Stack (native, no Ollama)

For the byte-to-runtime path, P64 compatibility aliases, native/WASM differences, governance boundary, and current implementation gaps, see the Q42/P64 Inference Pipeline. The table below is a compact component summary.

Qualia-DB runs LLM inference entirely in-process. There is no Ollama, no Python runtime, no HTTP server for models.

Step Component Detail
1 gguf_sharder.rs · GgufTokenizer Reads tokenizer metadata from GGUF or the embedded P64 Q42T section. Greedy longest-match encode(); SentencePiece ▁-aware decode().
2 p64_weight.rs · P64TensorIndex Validates P64 v3, exposes role-tagged tensor descriptors, and builds the synthetic GgufTensorIndex compatibility view used by the engine.
3 gguf_bridge · QTensorEngine Adopts an explicitly mounted P64 mmap or loads GGUF, reserves GEMM/KV arenas, and runs the production transformer-forward path.
4 shaders/fused_tensor_contraction.wgsl WGSL compute shader, 64 threads/workgroup, 4096 FMA ops per thread; backend via DirectML 1.15 / Vulkan / Metal / WebGPU on wgpu 30.
5 llm_agent.rs · LocalLlmAgent infer_local_model() runs the Phase 8 autoregressive decode loop: tokenise prompt → per-step GPU dispatch → SPSC logit stream → sentinel rollback check → argmax sample → EOS detection → detokenise.
6 orchestrator.rs · TaskOrchestrator orchestrate_inference() gates every call: validate_intent → infer → validate_output. Manages ModelLifecycle state machine and ThermalGovernor.

Platform GPU Priority

Platform Primary path Fallback
Windows x64 DirectML 1.15 (D3D12, hardware-vendor kernels) wgpu / D3D12
macOS Apple Silicon Accelerate cblas_sgemm (AMX coprocessor) wgpu / Metal
Linux (NVIDIA/AMD) wgpu / Vulkan (system ICD) —
WASM Browser WebGPU with P64 or GGUF Model-backed P64 browser release run remains tracked in the pipeline manual

Phase 8 Bifurcated Compute

Token generation uses two wait-free SPSC ring buffers (rtrb) keeping the governance intercept off the critical allocation path:

LLM Engine thread  ──LogitSummary──►  LogitStream  ──►  Webizen Sentinel (calling thread)
                   ◄──DenyRollback──  ControlStream ◄──  (checks anomaly byte; injects rollback)

Per decode step:

  1. LLM thread embeds the current token and calls dispatch_fused_transformer_block().
  2. Argmax + anomaly flag are packed into a fixed-size LogitSummary (no heap) and pushed to LogitStream.
  3. Sentinel reads the summary. If anomaly_byte == 0x99 (anachronism signature), it pushes DenyRollback to ControlStream.
  4. On the next step, the LLM thread pops ControlStream. If a rollback is pending, it substitutes a safe neighbour token instead of the argmax.
  5. Loop ends at EOS, the release decode budget (256), the 30-second cooperative deadline, or the absolute MAX_OUTPUT_TOKENS ceiling (2048).

Note — embedding lookup fully implemented. The decode loop uses real token embeddings via GgufTensorIndex::dequantize_token_embedding_into() which parses the GGUF tensor-info section and dequantizes per-token embeddings into caller-supplied buffers. The GPU compute, SPSC ring, governance pipeline, and tokeniser are all fully functional.

AgentBackend Variants

Local   // Explicitly mounted P64 or GGUF → wgpu → in-process. No outbound traffic.
Remote  // API call → Nym mixnet → ILP metered. Requires signed VC from Principal.
Hybrid  // Local-first. Falls back to Remote only with explicit Principal consent.

The MCP Fiduciary Mediation Layer

orchestrate_inference() in orchestrator.rs always runs three gates:

  1. validate_intent(intent) — pre-flight. Checks N3Logic Rights Ontology rules. If Deny, writes a conduct violation Quin to the WAL (signed with Ed25519) and aborts. The model is never invoked.
  2. agent.infer(prompt, graph_context) — the actual GPU inference.
  3. validate_output(output) — post-flight. Output must have ≥ 1 provenance NQuin citation. Ungrounded output is rejected.

The MCP server (mcp_server.rs) exposes the graph engine via McpIntentFrame (purpose_hash, deontic_constraints, profile_id, sanctuary_override). The state machine progresses: HandshakePhase → AllocationFirewallActive → SanctuaryGated.

The Rights Ontology & Semantic Adjudicator

Qualia-DB natively encodes a Rights Ontology directly into the Webizen VM (with SHACL compilation, defeasible rules, and modality bridges).

Intentional Computing (Anti-Usury Architecture)

Qualia-DB is a framework for Intentional Computing — computing that strictly honours the intent, agency, and Duty of Care of the natural person (the Principal).

DID:GIT & Staged Axiomatic Evolution

Data projects in this ecosystem possess Temporal Self-Governance.

The ILP Economic Shift Engine

Qualia-DB explicitly rejects the infinite rent-seeking paradigm of the legacy web.

Human-Facing Packaging

Qualia-DB ships with three tightly-bound human-facing interfaces:

  1. Webizen Studio (crates/webizen-studio/) + Webizen Desktop (crates/webizen-desktop/) — The Webizen environment (Windows, macOS, Linux). The UI is built in Rust with Dioxus 0.8.0-alpha.0 (compiled to web assets and hosted inside the webizen-desktop Tauri 2 shell), bypassing legacy Node.js/React overhead. It provides a flexible pane registry housing a Neuro-Symbolic Chat, an LLM Engine harness, a Vital Monitor, and an Ontology Builder designed to let people define personal ontological axioms via natural language, plus a large registry of domain “qapps”. The desktop shell ships signed, self-updating release artifacts (minisign updater; desktop / CLI / WASM release CI). Webizen Studio replaces the older Node.js/React prototypes and the Flutter desktop application.

  2. Qualia CLI (crates/qualia-cli) — The primary toolchain for data ingestion, benchmarking, daemon management, capability profile compilation, and resource catalog operations.

  3. WASM Bridge (crates/qualia-core-db, WASM target) — Builds to docs/playground/ for the browser demo (GitHub Pages), with feature-gated profiles (wasm-logic, wasm-scientific, wasm-llm, wasm-ontology; all wgpu/GPU code is gated behind gpu-runtime). Uses OPFS for block caching and SharedArrayBuffer for zero-copy IPC. The SPARQL query engine and the compute-engine primitives run in WASM; the autoregressive LLM decode loop still uses the mock ring-buffer path on WASM — real GPU decode requires native OS APIs.

W3C Solid Interoperability Bridge (Allocation Firewall)

Qualia-DB operates natively on .q42 CBOR-LD binary graphs with strict zero-allocation limits. The qualia-solid-bridge crate exposes a warp/tokio server translating incoming HTTP REST / JSON-LD / Turtle into minimal 64-bit Quin hashes via ldp_translator.rs. The multi-threaded tokio runtime is sandboxed to the network boundary — no string allocations bleed into the 512 MB floor.

Architectural Decision Records

Detailed rationale for specific design choices is in adr/.