Module turtle_doc 

Turtle-document parser for QualiaDB ingest.

The other parsers in this directory (n3_star, turtle_star) are line-oriented: they split each physical line on whitespace and keep the first three tokens. That is fine for N-Triples-shaped input but shreds real Turtle — multi-line predicate-object lists (;), object lists (,), and multi-word quoted literals all break, and @prefix is never expanded (CURIEs get hashed verbatim, so two documents’ doc: terms collide).

This parser handles the Turtle subset the values-credentials corpus actually uses:

• @prefix pfx: <iri> . / @base <iri> . directives, expanded into full IRIs before hashing (so a query with PREFIX/full <IRI> matches the stored hash, and each instrument’s doc: namespace is unique).
• statements spanning multiple lines, terminated by .
• ; (repeat subject) and , (repeat subject + predicate) lists
• a as a synonym for rdf:type
• quoted literals — including spaces, escapes, """…""", and trailing @lang / ^^datatype tags (the tag is dropped; the lexical value is hashed)

Terms are hashed with generate_60bit_token over the expanded IRI (or the literal’s lexical form), matching the SPARQL query path.

Functions

parse_turtle_doc_into

Parse a Turtle document into sink, returning the number of triples emitted.

parse_turtle_doc_stream

Streaming entry point used by the CLI ingest pipeline (writes via ExternalSorter).