Skip to main content

Crate qualia_audio

Crate qualia_audio 

Source
Expand description

Qualia Audio — native auditory intelligence.

Architecture: docs/plans/native-auditory-language-and-music-intelligence.md
Swarm: docs/plans/native-auditory-swarm-delivery.md

§Rules

  • Hot path caller-buffered; no Python.
  • Model outputs are epistemic proposals.
  • Frames authoritative; PCM never in NQuins.
  • Oral languages need no orthography.

Re-exports§

pub use aed_weights::AedWeightBundle;
pub use aed_weights::WeightedAedModel;
pub use capture::CapturePurpose;
pub use capture::CaptureSession;
pub use convert::mono_f32_to_i16_le;
pub use convert::to_mono_f32;
pub use cross_modal::events_overlapping_window;
pub use cross_modal::frames_to_media_ms;
pub use cross_modal::propose_temporal_correlations;
pub use cross_modal::AvCorrelationProposal;
pub use cross_modal::SharedMediaClock;
pub use cross_modal::TimeIntervalMs;
pub use events::ReferenceEventModel;
pub use events::CLASS_NOISE;
pub use events::CLASS_SILENCE;
pub use events::CLASS_SPEECH_LIKE;
pub use events::CLASS_TONAL;
pub use features::forward_cqt_mono;
pub use features::frame_energy;
pub use features::frame_zcr;
pub use features::log_mel_from_mono;
pub use features::magnitude_stft_chunk;
pub use features::StreamingStft;
pub use generation::separate_two_stems_reference;
pub use generation::synthesize_reference_tone;
pub use generation::StemReceipt;
pub use generation::SynthReceipt;
pub use generation::VoiceConsent;
pub use hash::media_digest;
pub use hash::q_hash;
pub use hash::MediaDigest;
pub use language::AccessClass;
pub use language::AnnotationTier;
pub use language::LanguageResourceBundle;
pub use media_store::guard_duration_bytes;
pub use media_store::synth_silence;
pub use media_store::synth_tone_f32;
pub use media_store::AudioMediaRecord;
pub use media_store::AudioMediaStore;
pub use media_store::RetentionClass;
pub use music::chroma12_from_mel;
pub use music::detect_onsets;
pub use music::estimate_f0_hz;
pub use music::estimate_tempo_from_onsets;
pub use music::propose_structure_segments;
pub use music::track_pitch;
pub use music::MusicAssumptions;
pub use music::OnsetEvent;
pub use music::PitchEstimate;
pub use music::StructureSegment;
pub use music::TempoEstimate;
pub use pipeline::run_ears_demo;
pub use pipeline::run_ears_on_wav_file;
pub use pipeline::run_ears_weighted;
pub use pipeline::section18_smoke;
pub use pipeline::sonify_demo_to_wav;
pub use pipeline::speech_phone_demo;
pub use pipeline::EarsDemoResult;
pub use production::apply_comp_sample;
pub use production::apply_eq_sample;
pub use production::DelayLine;
pub use production::ProcessPlan;
pub use production::StatefulMixer;
pub use production::TrackState;
pub use production::MAX_BLOCK;
pub use production::MAX_DELAY_SAMPLES;
pub use production::MAX_MIX_DELAY;
pub use production::MAX_TRACKS;
pub use resample::resample_linear_mono;
pub use semantic::compile_auditory_quins;
pub use semantic::human_correct_quin;
pub use semantic::human_reject_quin;
pub use semantic::AudioQuin;
pub use semantic::CTX_AUDIO;
pub use semantic::P_AUDITORY_OBSERVATION;
pub use semantic::P_PROPOSES_SOUND_CLASS;
pub use session_history::AutomationLane;
pub use session_history::AutomationPoint;
pub use session_history::OpKind;
pub use session_history::SessionHistory;
pub use session_history::SessionOp;
pub use session_history::MAX_AUTO_POINTS;
pub use session_history::MAX_HISTORY;
pub use sonify::class_to_hz;
pub use sonify::sonify_events_mono;
pub use speech::decode_for_language;
pub use speech::greedy_phone_decode;
pub use speech::SpeechEncoderWeights;
pub use speech::StreamingSpeechDecoder;
pub use speech::PHONES;
pub use wav::decode_wav;
pub use wav::encode_wav_i16_mono;
pub use wav::DecodedWav;
pub use types::*;

Modules§

aed_weights
P64-style acoustic event head (QVWT-like seed weights) — COMPLETE-WITH-GATE for real P64.
aqa
Audio Quality Assessment — intrusive (pure-Rust, no licence) + non-intrusive MOS (NeedsWeights). Never emit a fabricated MOS number: learned MOS fails closed until weights are supplied.
capability_registry
Capability registry — the honest, machine-readable status of every audio capability.
capture
Capture adapter — permission + session (platform device behind desktop).
convert
Sample format and channel conversion (caller-buffered after setup).
cross_modal
Swarm X — eyes + ears correlation (overlap ≠ causality).
events
A5 — reference VAD + energy event classifier (honest reference backend).
features
Streaming / block feature extraction (CPU reference path).
fx
Production FX — reverb, stereo. Real DSP the production block delegates to. Re-exports only (AU-PROD).
generation
Swarm G_audio — reference tone synthesis, consent revoke, simple stem separation.
hash
FNV-1a 64 — same family as Qualia q_hash (vision semantic).
io
I/O — metadata, equal-loudness loading, quality resampling. Re-exports only (AU-IO fills submodules).
language
Swarm L — language resource bundle scaffold (oral-first).
media_store
Content-addressed audio media store (mirror vision media store).
midi
Native MIDI engine — MIDI 1.0 + MIDI 2.0 (UMP), Standard MIDI File I/O, sequencer/transport, microtuning (Scala/MTS), and a zero-alloc synth. Symbolic authoring/production, under the Qualia ABI.
models
Learned audio heads — loaded via a common P64 loader that fails closed (abstains / NeedsWeights) when weights are absent. No Candle/Burn runtime, no Python. Real weights are gated on the principal (HA1 corpus, HA6 speech model). See ADR 007 + delivery plan Waves 7–8.
music
Swarm M — foundational music features (assumptions declared).
pipeline
End-to-end Ears MVP demo path (A5 product surface).
production
Swarm P — bounded production engine (zero-alloc process block + FX primitives).
qac_spatial
QAC Spatial Audio, Higher-Order Ambisonics (HOA) Bus, Wigner-D Rotation, Binaural HRIR/RIR Rendering, ITU-R BS.1770-5 Loudness & Modal Resonance Adapters (qac_spatial.rs).
resample
Linear resampler (cold/simple; declared latency 0 for linear interp).
semantic
Auditory observations → NQuin-compatible records (epistemic).
session_history
DAW session operation history (cold path — not in audio callback).
sonify
U3-style hear path: events → parametric mono PCM (navigate by time).
spatial
Zero-Heap 3D Spatial Audio & Doppler Engine (spatial.rs)
speech
Speech path scaffold — greedy “phone” decode over log-mel (not full ASR).
types
Stable auditory ABI (fixed-layout, caller-buffered where hot).
wav
Minimal WAV (PCM i16 / f32 LE) cold decoder — no external codec crate.

Structs§

SpatialEmitter
3D Spatial Sound Emitter.
SpatialListener
3D Spatial Listener (camera or avatar ears).
SpatialOcclusionBridge
Acoustic occlusion bridge parameters for SDF/BVH ray-marched scene audio graphs.
SpatialOcclusionFilterState
Zero-heap stereo DF2T biquad state for per-voice occlusion low-pass filtering.
SpatialVoiceParams
Computed spatial voice parameters for audio buffer rendering.

Enums§

DistanceModel
Distance attenuation model.

Constants§

MAX_SPATIAL_VOICES
Maximum spatial sound emitters processed in a single zero-heap mix pass.
SPEED_OF_SOUND_AIR
Speed of sound in dry air at 20°C in m/s.

Functions§

mix_spatial_voice_into
Zero-heap stereo mixer mixing mono PCM audio into a stereo output buffer with spatial gains.
mix_spatial_voice_occluded_into
Zero-heap occluded stereo mixer with DF2T biquad low-pass occlusion filtering and wet reverb send bus accumulation.