Building a VS Code Extension for a New Language

A programming language without IDE support is like a car without a dashboard — technically functional, but nobody wants to drive it. When I started building Lateralus, I knew that syntax highlighting, snippets, and editor intelligence would be just as important as the compiler itself. Here's how we built the VS Code extension from scratch.

Why VS Code First

The numbers made this an easy decision. VS Code has over 70% market share among developers, it's free, and its extension API is well-documented. For a new language trying to gain adoption, you want to be where the developers already are.

The extension ecosystem also has a clear progression path: start with syntax highlighting (TextMate grammar), add snippets and bracket matching (declarative configuration), then graduate to a Language Server Protocol (LSP) implementation for full intelligence. We followed this path exactly.

TextMate Grammars: The Foundation

VS Code uses TextMate grammars for syntax highlighting — a system of regex patterns organized into scope rules. Each pattern assigns a scope name to matched text, and the color theme maps scopes to colors.

Here's a snippet from our lateralus.tmLanguage.json:

{
  "scopeName": "source.lateralus",
  "patterns": [
    { "include": "#comments" },
    { "include": "#keywords" },
    { "include": "#operators" },
    { "include": "#strings" },
    { "include": "#types" }
  ],
  "repository": {
    "keywords": {
      "match": "\\b(let|match|if|then|else|type|spawn|await|fn|extern|when)\\b",
      "name": "keyword.control.lateralus"
    },
    "operators": {
      "match": "\\|>|=>|\\->|\\|",
      "name": "keyword.operator.lateralus"
    }
  }
}

The Pipeline Operator Challenge

The hardest part was getting the |> operator right. TextMate grammars are regex-based, and | is a regex metacharacter. We needed to match |> as a single pipeline operator while also matching | alone as a pattern match separator. The solution was ordering: match the longer token first.

"operators": {
  "patterns": [
    {
      "match": "\\|>",
      "name": "keyword.operator.pipeline.lateralus"
    },
    {
      "match": "=>",
      "name": "keyword.operator.arrow.lateralus"
    },
    {
      "match": "\\|",
      "name": "keyword.operator.match-arm.lateralus"
    }
  ]
}

We also gave |> its own scope (keyword.operator.pipeline) so that color themes can style it distinctly. In the official Lateralus theme, the pipeline operator renders in bright orange — making data flow visually pop.

180+ Snippets

Snippets are the unsung heroes of developer productivity. We shipped the extension with over 180 snippets covering every common pattern. The design philosophy: every snippet should save at least 10 keystrokes and produce idiomatic code.

// Snippet: "pipe" → expands to pipeline template
// Trigger: pipe + Tab

${1:value}
    |> ${2:transform}
    |> ${3:transform}
    |> ${0:result}

// Snippet: "matchr" → match with Result pattern
// Trigger: matchr + Tab

match ${1:expr}
    | Ok(${2:value}) => ${3:body}
    | Err(${4:error}) => ${0:handler}

Bracket Matching and Auto-Indentation

VS Code's language-configuration.json handles bracket matching, auto-closing pairs, and indentation rules. Getting this right is crucial for the editing experience:

{
  "brackets": [
    ["{", "}"],
    ["[", "]"],
    ["(", ")"]
  ],
  "autoClosingPairs": [
    { "open": "{", "close": "}" },
    { "open": "[", "close": "]" },
    { "open": "(", "close": ")" },
    { "open": "\"", "close": "\"" },
    { "open": "/*", "close": "*/" }
  ],
  "indentationRules": {
    "increaseIndentPattern": "(=>|\\{|\\|>)\\s*$",
    "decreaseIndentPattern": "^\\s*(\\}|\\]|\\))"
  }
}

The key insight: pipeline operators should trigger auto-indent. When you type |> at the end of a line and press Enter, VS Code should indent the next line to align with the pipeline chain. That one indentation rule took us three iterations to get right, but it makes writing multi-line pipelines feel effortless.

Testing the Extension

We test the extension at two levels. First, snapshot tests for the TextMate grammar — we feed sample Lateralus code through the tokenizer and compare the output tokens against saved snapshots. If a grammar change accidentally breaks highlighting for existing code, the snapshot fails.

Second, integration tests using VS Code's test runner. These launch a real VS Code instance with the extension installed and verify that snippets expand correctly, brackets match, and syntax highlighting covers edge cases.

// Integration test example
test("pipeline operator is highlighted as operator", () => {
    let tokens = tokenize("x |> f")
    assert_eq(tokens[1].scope, "keyword.operator.pipeline.lateralus")
})

test("match arm | is not confused with pipeline", () => {
    let tokens = tokenize("| Ok(x) => x")
    assert_eq(tokens[0].scope, "keyword.operator.match-arm.lateralus")
})

Publishing and Adoption

We published the extension to the VS Code Marketplace under the ID bad-antics.lateralus-lang. The publishing process itself is straightforward — vsce package to build, vsce publish to ship. What surprised me was how much the marketplace listing matters for discoverability. A good icon, detailed README with screenshots, and clear feature list make a real difference in install rates.

Lesson learned: Your extension's marketplace page is your language's first impression for many developers. Invest in it like you'd invest in a landing page.

What's Next: LSP Server

TextMate grammars and snippets get you 80% of the way, but the remaining 20% — go-to-definition, find references, inline errors, hover documentation, auto-completion — requires a Language Server Protocol implementation. We're building the Lateralus LSP server now, written in Lateralus itself (naturally). It reuses the compiler's parser and type checker to provide real-time diagnostics as you type.

What's next for the extension

The current extension is a solid foundation, but there's more planned for v2.0:

• Full LSP server — replacing regex-based highlighting with real type information from the compiler. This enables go-to-definition, find-all-references, and type-on-hover across the entire project.
• Integrated REPL — run Lateralus code directly in a VS Code panel with inline results, similar to Jupyter notebooks but for Lateralus pipelines.
• Security scanner integration — a panel showing real-time output from Lateralus security pipelines, with clickable findings that link to code locations.

What's next for the extension

The current extension is a solid foundation, but there's more planned for v2.0:

• Full LSP server — replacing regex-based highlighting with real type information from the compiler. This enables go-to-definition, find-all-references, and type-on-hover across the entire project.
• Integrated REPL — run Lateralus code directly in a VS Code panel with inline results, similar to Jupyter notebooks but for Lateralus pipelines.
• Security scanner integration — a panel showing real-time output from Lateralus security pipelines, with clickable findings that link to code locations.