dom-eater

DOM Eater

A fast, error-tolerant AST parser for markup in HTML templates, JSX/TSX, JavaScript/TypeScript, Razor and Pug.

DOM Eater is built for editor tooling that highlights and completes CSS written in class attributes. Its AST is general enough for other tooling that needs the same information, but its scope and API follow what that tooling needs:

• It finds elements, attributes, text, and the script regions and string literals inside them, with exact source positions.
• It does not parse JavaScript or C#. It scans them the way the TypeScript parser and the Razor compiler read them, to decide what {, }, <, /, @ and quotes mean, and skips the rest. If you need a JavaScript AST, use a JavaScript parser.
• It never gives up on incomplete code, so the element being typed at the end of the text is still found.

Installation

npm i dom-eater

API

import { createAstSyntaxError, parseHtml, parseJsx, parsePug, parseRazor, parseRazorComponent, parseScript } from "dom-eater"
import type { AstNode, AstSyntaxError, Element } from "dom-eater"

Each parser returns { ast, errors } and never throws: syntax errors are reported in errors while parsing goes on. A text that is not a string gives no nodes and the error "The input is not a string.".

• ast: for parseHtml, parsePug, parseRazor and parseRazorComponent, the top-level nodes, typed Element["children"]; for parseJsx and parseScript, the top-level elements found in the script, typed Element[].
• errors: AstSyntaxError objects, which are instanceof Error, with name: "AstSyntaxError", message and the start and end of the problem as own properties and no stack. JSON.stringify, structuredClone and postMessage keep all four, so errors can be sent from a worker or a language server; the copies structuredClone and postMessage make are plain objects.
• include_text: pass true to add text, the source slice, to every node.

Positions are UTF-16 offsets into text: start is inclusive and end exclusive, so text.slice(node.start, node.end) is the node's source.

HTML templates in scripts

parseJsx and parseScript parse the content of these template literals as HTML:

• Tagged with html or svg, as in Lit: html`<p class="a">${b}</p>`
• Marked with a comment: /* html */ `<p>…</p>`
• The value of a template property, as in Angular and Vue components: @Component({ template: `<p>…</p>` })

Each ${…} becomes a Script node in the markup, starting at its $. It holds the strings and elements in its code, so templates nested in substitutions stay in place.

AST

Every node has type, start, end, and text when include_text is set.

The types are exported from dom-eater itself: import type { AstNode, Element } from "dom-eater". The type and subType fields narrow them: a "template" Script among an element's children has elements and one in a "backtick" String has none, so on any AstNode check "elements" in node.

Examples

parseHtml

import { parseHtml } from "dom-eater"

parseHtml(`<p class="note {tone}">Hi {name}</p>`)

{
  "ast": [
    {
      "attributes": [
        {
          "end": 22,
          "name": "class",
          "start": 3,
          "type": "Attribute",
          "value": {
            "end": 22,
            "scripts": [
              {
                "end": 21,
                "start": 15,
                "strings": [],
                "subType": "block",
                "type": "Script"
              }
            ],
            "start": 9,
            "subType": "double",
            "type": "String"
          }
        }
      ],
      "children": [
        {
          "end": 26,
          "start": 23,
          "type": "Text"
        },
        {
          "end": 32,
          "start": 26,
          "strings": [],
          "subType": "block",
          "type": "Script"
        }
      ],
      "end": 36,
      "name": "p",
      "start": 0,
      "subType": "open",
      "type": "Element"
    }
  ],
  "errors": []
}

parseRazor

import { parseRazor } from "dom-eater"

parseRazor(`<li class="@(done ? "done" : "")">@item.Name</li>`)

{
  "ast": [
    {
      "attributes": [
        {
          "end": 33,
          "name": "class",
          "start": 4,
          "type": "Attribute",
          "value": {
            "end": 33,
            "scripts": [
              {
                "elements": [],
                "end": 32,
                "start": 11,
                "strings": [
                  {
                    "end": 26,
                    "scripts": [],
                    "start": 20,
                    "subType": "double",
                    "type": "String"
                  },
                  {
                    "end": 31,
                    "scripts": [],
                    "start": 29,
                    "subType": "double",
                    "type": "String"
                  }
                ],
                "subType": "razor",
                "type": "Script"
              }
            ],
            "start": 10,
            "subType": "double",
            "type": "String"
          }
        }
      ],
      "children": [
        {
          "elements": [],
          "end": 44,
          "start": 34,
          "strings": [],
          "subType": "razor",
          "type": "Script"
        }
      ],
      "end": 49,
      "name": "li",
      "start": 0,
      "subType": "open",
      "type": "Element"
    }
  ],
  "errors": []
}

parseJsx, parseScript and parsePug produce the same shapes: {…} in JSX is a "jsx" Script with its elements, ${…} in an HTML template is a "template" Script, and Pug code and attribute expressions are "pug" Scripts.

Frameworks

How markup is parsed

parseHtml

• Close tags match the nearest open element with the same name, case-insensitively.
• End tags that HTML makes optional (p, li, td, option, …) are implied the way browsers imply them. Any other element that is never closed is reported, and the nodes after it stay its siblings.
• script, style, textarea, title, iframe, noembed, noframes and xmp hold raw text that ends at the first matching close tag, as in browsers. plaintext runs to the end of the input.
• Inside svg and math, only script and style hold raw text, as in Svelte, and <![CDATA[…]]> is Text. HTML integration points such as foreignObject switch back to HTML.
• {…} in text and attribute values is a Script. A leading Svelte tag such as {#if, {:else} or {@html is skipped.
• Angular block headers such as @if (a) { and } @else {, @let declarations, and the syntax of ICU expressions such as {count, plural, =0 {…}} stay Text, so the markup inside them is parsed.
• Attribute values have no backslash escapes.
• The content of <template lang="pug"> is parsed as Pug, after removing its common indentation as Vue does.

parsePug

It follows the Pug lexer and parser, and is checked against them on real templates.

• Tags, .class and #id literals, (…) attributes, &attributes(…), #{…} tags and +mixin(…) calls are elements and attributes. .class is an Attribute named class whose value is an unquoted String.
• Attribute values end where the Pug lexer ends them: at a comma or at whitespace after a complete JavaScript expression. A value that is one quoted string is a String; any other value is a Script with the string literals in it.
• -, = and != code, #{…} and !{…} interpolations, and the lines of if, else, unless, case, when, default, each, while and mixin are Script nodes. The nodes in their blocks stay in the enclosing element, as with Svelte blocks.
• Piped text, text after tags, . blocks, filters and lines starting with < are Text, with #[…] tags inside them. Unbuffered //- and buffered // comments are Comment nodes, including their blocks.
• Inconsistent indentation is reported, and the line becomes a sibling of the more deeply indented lines before it.

parseRazor and parseRazorComponent

They follow the Razor compiler of the .NET SDK rather than browsers, and are checked against it on real views, pages and components.

• Close tags match the nearest open element with the same name. No end tags are implied, and only script holds text, which may still contain @ code.
• @name.Member, @(…), @{…}, control flow such as @if, @foreach and @switch, and directives such as @model, @section and @code are Script nodes. The markup inside code blocks, @<p>…</p> templates and <text> elements is in their elements, and the C# string literals, including interpolated, verbatim and raw strings, in their strings.
• @@ and email addresses such as a@b.com stay Text.
• parseRazor reads @ where an attribute name goes as C# code, an Attribute with the name "". parseRazorComponent reads it as a directive attribute such as @onclick or @bind-Value.

Scripts

parseJsx, parseScript, and the script regions of parseHtml are scanned rather than parsed. To tell a tag from a comparison, a regular expression from a division, and the end of a statement, the scanner follows the rules of the TypeScript parser:

• Operands and keywords: return <p/> has a tag, while a < b and f<T>(x) do not. x = a / 2 divides, while if (a) /re/.test(b) has a regular expression.
• Automatic semicolon insertion, such as a line break after var a, break, a function overload or import … from "x".
• Where await and yield are keywords: async functions, generators, methods, arrow functions, class bodies, and the top level of modules.
• TypeScript: type parameters such as <T,>, type assertions with as and satisfies, type annotations, type aliases, interfaces, overloads and decorators.

The scanner is checked against the TypeScript parser on real code from node_modules, on the same code with probes inserted at every statement and operand, and on generated programs.

Limitations

• Syntax that depends on the file type is decided without it. At the top level of parseHtml scripts, await is a keyword, as in modules. a < b > /c/ follows JavaScript, where TypeScript reads an instantiation expression.
• parseJsx reads <T>value as a JSX tag, as the TypeScript parser does in .tsx files. Use parseScript for .ts files.
• Razor code inside <!-- … -->, <!…>, <?…?> and <![CDATA[…]]> is part of the Comment node, although the Razor compiler still reads it as code. In code the compiler rejects, #define is not applied to later #if directives, and a # directive right after markup on the same line can be read differently.
• In Pug attributes and #{…}, a / inside a regular expression's character class ends the regular expression, as in Pug: p(a=/[/]/.test(x)) fails in Pug with "Mismatched Bracket: ]", and parsePug reports the same error. Write /[\/]/ instead. Where that misreading still leaves the brackets balanced, Pug checks the value with a JavaScript parser while parsePug judges it from its last token, so the value can end at a different place.
• Very deep nesting, such as thousands of nested blocks, is reported as "The input is nested too deeply." instead of parsed. The TypeScript parser itself overflows at a smaller depth.