Skip to main content

Module trivia

Module trivia 

Source
Expand description

CST trivia — comments, whitespace, and exact token text (T36).

The AST drops comments and whitespace. The CST preserves them so that poet translate and poet format can round-trip source text without destroying commentary.

§Design

  • Trivia represents a comment or whitespace gap.
  • TriviaSink collects trivia as the lexer advances.
  • CstNode wraps an AST node with its leading and trailing trivia.

The trivia sink is optional — the lexer can run in “AST mode” (trivia discarded, current behaviour) or “CST mode” (trivia collected). This avoids changing existing behaviour.

Reference: docs/vibescript-full-impl-PLAN.md §8.8 T36.

Structs§

CstNode
A CST node — an AST node with its leading and trailing trivia.
Trivia
A single trivia element — a comment or whitespace gap.
TriviaSink
A sink that collects trivia as the lexer advances. Use TriviaSink::new to create a collecting sink, or TriviaSink::disabled to create a discarding sink.

Enums§

TriviaKind
The kind of trivia.

Functions§

extract_comments
Extract trivia from source text by scanning for comments.