highlighter

Tokens

2026-06-20T21:52:06.626409609Z

When you render to something other than a web page — a PDF, a terminal, a GUI — take the tokens directly instead of producing HTML and parsing it back apart. tokens returns the same information highlight uses, structured rather than serialized.

Tokenizing

tokens(code) returns one list of tokens per source line, with adjacent same-scope tokens merged:

val Right(hl) = Highlighter.fromJson(grammar): @unchecked

for line <- hl.tokens("val x = 42\n// note") do
  for tok <- line do
    render(tok.text, hl.categoryOf(tok))

Each Token is:

case class Token(text: String, scopes: List[String])

The line grouping is preserved: tokens(code).length is the number of lines, and concatenating a line’s token texts reproduces that line exactly — so you control line breaks and indentation yourself.

Categories and colours

A token’s effective scope is its innermost (last) one. categoryOf collapses it to one of the nine rendering categories — the same mapping the HTML back ends use — so you can drive your own palette:

def colour(tok: Token): MyColour =
  hl.categoryOf(tok) match           // keyword | string | comment | number | type |
    case "keyword" => myKeyword      // function | variable | operator | punctuation | ""
    case "string"  => myString
    case "comment" => myComment
    case "number"  => myNumber
    case _         => myDefault

An empty category ("") means the token carries no colour of its own — plain source text. category(scope) is the same mapping applied to a raw scope string, if you have one in hand.

Pairing with a theme

The built-in themes are just category-to-hex maps, so you can reuse one as your palette instead of hand-rolling colours:

val theme = Theme.OneDark
def hex(tok: Token): String =
  hl.categoryOf(tok) match
    case "keyword" => theme.keyword
    case "string"  => theme.string
    case "number"  => theme.number
    case _         => theme.default

Themes

2026-06-20T21:52:06.626409609Z

A Theme maps each of the nine highlight categories to a colour. Themes are used by InlineMode to emit inline style colours, and you can read a theme yourself when rendering tokens.

Built-in themes

Three presets ship with the library:

InlineMode(Theme.OneDark)   // dark
InlineMode(Theme.OneLight)  // light
InlineMode(Theme.Monokai)   // dark

The same Scala snippet under each — and every one of these blocks was produced by highlighter itself, in inline mode:

OneDark

// quicksort
def sort(xs: List[Int]): List[Int] = xs match
  case Nil => Nil
  case pivot :: rest =>
    val (less, more) = rest.partition(_ < pivot)
    sort(less) ++ (pivot :: sort(more))

val nums = List(5, 2, 8, 1, 9)
println(sort(nums))  // List(1, 2, 5, 8, 9)

OneLight

// quicksort
def sort(xs: List[Int]): List[Int] = xs match
  case Nil => Nil
  case pivot :: rest =>
    val (less, more) = rest.partition(_ < pivot)
    sort(less) ++ (pivot :: sort(more))

val nums = List(5, 2, 8, 1, 9)
println(sort(nums))  // List(1, 2, 5, 8, 9)

Monokai

// quicksort
def sort(xs: List[Int]): List[Int] = xs match
  case Nil => Nil
  case pivot :: rest =>
    val (less, more) = rest.partition(_ < pivot)
    sort(less) ++ (pivot :: sort(more))

val nums = List(5, 2, 8, 1, 9)
println(sort(nums))  // List(1, 2, 5, 8, 9)

Custom themes

Theme is a plain case class with a default colour per category, so override only what you want:

val mine = Theme(
  keyword     = "#ff0000",
  string      = "#00aa00",
  comment     = "#888888",
  number      = "#d19a66",
  `type`      = "#e5c07b",
  function    = "#61afef",
  variable    = "#e06c75",
  operator    = "#56b6c2",
  punctuation = "#abb2bf",
  default     = "#abb2bf",   // anything with no category of its own
)

val Right(hl) = Highlighter.fromJson(grammar, InlineMode(mine)): @unchecked

keyword covers control words and storage modifiers; default is used for text that carries a scope highlighter doesn’t map to a category. (type is a Scala keyword, so it is written `type` in source.)

HTML Rendering

2026-06-20T21:52:06.626409609Z

highlight(code) returns an HTML string with one <span> per highlighted run. How those spans are styled depends on the render mode you pass when building the highlighter. Text is HTML-escaped, and adjacent runs with the same scope are merged into one span.

Class mode

ClassMode(prefix) emits <span class="prefix-category">, where the category is one of nine names. You supply the colours in your own stylesheet:

val Right(hl) = Highlighter.fromJson(grammar, ClassMode("hl-")): @unchecked
hl.highlight("val x = 42")
// <span class="hl-keyword">val</span> x = <span class="hl-number">42</span>

Change the prefix to match your CSS:

ClassMode("hl-")    // <span class="hl-keyword">
ClassMode("code-")  // <span class="code-keyword">

The nine categories are: keyword, string, comment, number, type, function, variable, operator, punctuation.

Inline mode

InlineMode(theme) emits <span style="color:#hex"> straight from a theme, so the output is self-contained — no stylesheet required:

val Right(hl) = Highlighter.fromJson(grammar, InlineMode(Theme.OneDark)): @unchecked
hl.highlight("val x = 42")
// <span style="color:#c678dd">val</span> x = <span style="color:#d19a66">42</span>

This very page is built that way: the plain fenced ```scala blocks here are highlighted at build time by the same library, through the site generator’s codeHighlighter hook:

// quicksort
def sort(xs: List[Int]): List[Int] = xs match
  case Nil => Nil
  case pivot :: rest =>
    val (less, more) = rest.partition(_ < pivot)
    sort(less) ++ (pivot :: sort(more))

val nums = List(5, 2, 8, 1, 9)
println(sort(nums))  // List(1, 2, 5, 8, 9)

See Themes for the built-in schemes and how to define your own.

Lines

highlight joins lines with \n; each source line is highlighted independently and a multi-line begin/end span carries its scope across the break. If you are rendering to something other than HTML — and want the line structure handed to you — use the token API instead.

Quick Start

2026-06-20T21:52:06.626409609Z

Highlighting takes three steps: load a grammar, pick how you want the output, and pass it some code.

A first grammar

A TextMate grammar is JSON describing the patterns of a language. A tiny one is enough to start — keywords, numbers, strings, and line comments:

import io.github.edadma.highlighter.*

val grammar = """{
  "scopeName": "source.example",
  "patterns": [
    { "match": "\\b(val|def|if|else)\\b", "name": "keyword.control" },
    { "match": "\\b\\d+\\b", "name": "constant.numeric" },
    { "begin": "\"", "end": "\"", "name": "string.quoted.double" },
    { "match": "//.*$", "name": "comment.line" }
  ]
}"""

Build a highlighter

Highlighter.fromJson parses the grammar and returns Either[String, Highlighter] — a Left carries a parse error message, so this never throws on a bad grammar:

val Right(hl) = Highlighter.fromJson(grammar, ClassMode("hl-")): @unchecked

Highlight some code

println(hl.highlight("val x = 42"))
// <span class="hl-keyword">val</span> x = <span class="hl-number">42</span>

That is the whole loop. From here:

Loading Grammars — use a real VS Code .tmLanguage.json, and the grammar features highlighter supports.
HTML Rendering — CSS-class output versus inline theme colours.
Themes — the built-in colour schemes and writing your own.
Tokens — skip the HTML and render the tokens yourself.

Installation

2026-06-20T21:52:06.626409609Z

highlighter is cross-published for the JVM, Scala.js, and Scala Native. Add it with the %%% operator so sbt selects the right platform artifact:

libraryDependencies += "io.github.edadma" %%% "highlighter" % "0.0.10"

It is published to Maven Central, so no extra resolver is needed.

Requirements

highlighter is a pure-Scala library — its only dependencies are zio-json for reading grammar files and the pure-Scala oniguruma regex engine. There is no native dependency, so the same code links and runs on the JVM, in the browser through Scala.js, and as a Scala Native binary.

Importing

Everything lives in one package:

import io.github.edadma.highlighter.*

That brings in Highlighter, the render modes (ClassMode, InlineMode), Theme, and the Token type. The next page highlights a snippet end to end.

Loading Grammars

2026-06-20T21:52:06.626409609Z

highlighter reads TextMate grammars — the .tmLanguage.json files that VS Code and TextMate use to describe a language. Any VS Code-compatible grammar works unchanged.

Loading a VS Code grammar

Read the grammar file as a string and hand it to fromJson:

val grammarJson = scala.io.Source.fromFile("JavaScript.tmLanguage.json").mkString
val Right(hl)   = Highlighter.fromJson(grammarJson, ClassMode("code-")): @unchecked

println(hl.highlight("const x = 42;"))

On Scala.js or Scala Native, load the grammar however that platform reads text — a bundled resource, an HTTP fetch, a file — and pass the resulting string in the same way.

Supported grammar features

The grammar engine is a regex state machine, not a flat list of patterns. It supports the features real-world grammars rely on:

match patterns with a name and numbered captures.
begin/end patterns with beginCaptures, endCaptures, and nested patterns, so a construct can span lines (a block comment, a multi-line string).
contentName to scope the region between a begin and end separately from the delimiters.
A repository of named rule sets, referenced with include.
include references: #ruleName for a repository rule and $self for the whole grammar — recursive and cyclic includes are handled with cycle detection.
POSIX character classes ([:alpha:], [:digit:], …), translated per platform.

When a pattern can’t compile

A grammar occasionally uses an Oniguruma regex feature that doesn’t translate. Rather than fail the whole grammar, highlighter skips the offending pattern and records a diagnostic:

val Right(hl) = Highlighter.fromJson(grammarJson): @unchecked
hl.loadWarnings.foreach(println)   // empty when every pattern compiled cleanly

The buffer is captured when the highlighter is built, so reading it is free. Print it once at load time to see whether any patterns were dropped.