# `Ichor.Syntax`
[🔗](https://github.com/joetjen/ichor_runtime/blob/v0.6.0/lib/ichor/syntax.ex#L1)

The lossless tree: every byte of the input, as tokens under the rules
that matched them.

The ordinary parse result keeps what a grammar captures and steps over
the rest -- trivia, punctuation, the order of differently named
captures. A tool that edits a document and writes it back (a
formatter, a config updater keeping its comments) needs all of it.
A parser generated with `lossless: true` returns this tree beside its
ordinary value (`design/lossless-mode.md` in `ichor`).

The tree holds four promises, whatever the grammar:

  1. **Lossless.** `text(root)` is the input, byte for byte.
  2. **Ordered.** The leaves, left to right, are every token of the
     input: named tokens, literals, punctuation and trivia.
  3. **Trivia sits between, never at an edge.** No node but the root
     begins or ends with a trivia token. A trivia token belongs to the
     deepest node with something significant on both sides of it;
     the input's leading and trailing trivia belong to the root.
  4. **Every rule that matched is a node**, the ones the ordinary
     tree passes through included. A rule that matched nothing is an
     empty node.

Which comment belongs to which value is not decided here: that is a
policy of the format, and the tree hands a consumer what it needs to
decide it -- each trivia token's siblings, line and column.

# `record`

```elixir
@type record() :: {atom(), non_neg_integer(), non_neg_integer(), [record()]}
```

A rule's match as a parser records it: the rule, the token range
`[from, to)` of the trimmed stream it matched, and the records of the
rules it called, in order (`Ichor.Syntax.Record`).

# `text`

```elixir
@spec text(Ichor.Syntax.Node.t() | Ichor.Syntax.Token.t()) :: String.t()
```

The input a node or token covers: for the root, the whole input.

    iex> Ichor.Syntax.text(%Ichor.Syntax.Token{name: :N, text: "42"})
    "42"

# `tokens`

```elixir
@spec tokens(Ichor.Syntax.Node.t()) :: [Ichor.Syntax.Token.t()]
```

Every token under a node, in source order.

    iex> Ichor.Syntax.tokens(%Ichor.Syntax.Node{rule: :empty, children: []})
    []

# `tree`

```elixir
@spec tree(record(), [Grammar.VM.Token.t()], atom() | nil) :: Ichor.Syntax.Node.t()
```

The lossless tree of a parse: `record` is the root rule's match, its
token ranges counted in the stream with leading and trailing trivia
trimmed (`Grammar.Source.trimmed/2`); `tokens` is the whole stream,
untrimmed, exactly as the tokenizer returned it; `skip` is the
grammar's `@skip` token, or nil.

## Examples

    iex> tokens = [
    ...>   %Grammar.VM.Token{name: :N, text: "1", line: 1, column: 1},
    ...>   %Grammar.VM.Token{name: :WS, text: " ", line: 1, column: 2},
    ...>   %Grammar.VM.Token{name: :N, text: "2", line: 1, column: 3}
    ...> ]
    iex> tree = Ichor.Syntax.tree({:pair, 0, 3, [{:n, 2, 3, []}]}, tokens, :WS)
    iex> Enum.map(tree.children, & &1.__struct__)
    [Ichor.Syntax.Token, Ichor.Syntax.Token, Ichor.Syntax.Node]
    iex> Ichor.Syntax.text(tree)
    "1 2"

---

*Consult [api-reference.md](api-reference.md) for complete listing*