Where a range of a parse's token stream is in its input, as an
Ichor.Node.source_span/0: {line, column, length}.
A parser records each capture's place as the token range [from, to)
it matched (Ichor.Capture.at/3) -- two integers, nothing to compute
while matching, and most of them never asked for. Ichor.Actions
turns a range into a span only when it builds the capture, through
this.
Built once per parse. new/2 reads the input once for where each
line starts, so a span is a few lookups whatever its size. A span is
measured from the input's line starts rather than by adding up token
texts, because a stream need not hold every token in between: the
shift-reduce engines drop the skip tokens.
Lines and columns are 1-based, and columns and lengths count
codepoints, as Grammar.Native.Runtime.Tokenizer does.
Summary
Types
input is the text parsed, kept to show a placed error's line;
stream is the token stream the parse ran over; line_starts the
codepoint offset each line of the input begins at; end_at the
{line, column} just past the input's last character.
Functions
The locator for a parse of input over stream (a tuple of
Grammar.VM.Tokens, as every engine matches against).
The span of the tokens [from, to): where the first one starts, and
how many codepoints there are from there to the end of the last one.
Types
@type t() :: %Grammar.Source.Locator{ end_at: {pos_integer(), pos_integer()}, input: String.t(), line_starts: tuple(), stream: tuple() }
input is the text parsed, kept to show a placed error's line;
stream is the token stream the parse ran over; line_starts the
codepoint offset each line of the input begins at; end_at the
{line, column} just past the input's last character.
Functions
The locator for a parse of input over stream (a tuple of
Grammar.VM.Tokens, as every engine matches against).
@spec span(t(), non_neg_integer(), non_neg_integer()) :: Ichor.Node.source_span()
The span of the tokens [from, to): where the first one starts, and
how many codepoints there are from there to the end of the last one.
An empty range (from == to) is a place, not a stretch: where the
token at from starts -- or the end of the input, past the last
token -- with length 0.