February 2024 · 8 min read

Designing the Standard Library

A pipeline language needs a pipeline-friendly standard library. Every function in Lateralus's stdlib takes its primary data as the first argument.

The first-argument convention

In most languages, map(fn, list) takes the function first. In Lateralus, map(list, fn) takes the data first. This means pipelines read naturally:

// Natural left-to-right flow
data |> map(transform) |> filter(predicate) |> sort()

// Equivalent to:
sort(filter(map(data, transform), predicate))

Module overview

std::io — print, read, file I/O
std::collections — Vec, HashMap, Set, Deque
std::string — split, join, trim, replace, regex
std::math — abs, sqrt, pow, trig, constants
std::fs — read_file, write_file, walk_dir
std::net — HTTP client, TCP/UDP sockets
std::json — parse, stringify, query
std::time — now, sleep, duration, formatting
std::crypto — SHA-256, AES, RSA, random

80+ pipeline-ready functions

Here's a sample:

// String processing
"hello world" |> split(" ") |> map(capitalize) |> join("-")
// "Hello-World"

// File processing
read_dir("src/")
    |> filter(|f| f |> ends_with(".ltl"))
    |> map(read_to_string)
    |> flat_map(|s| s |> lines())
    |> filter(|l| l |> contains("TODO"))
    |> enumerate()
    |> each(|(i, l)| println("{i}: {l}"))

Design principles

Data first — primary argument is always the first parameter
Immutable by default — functions return new values, never mutate input
No nulls — functions return Option or Result
Consistent naming — to_X for conversions, is_X for predicates

Module overview

The standard library is organized into focused modules. Here's the complete map:

std::io — file I/O, stdin/stdout, buffered readers/writers
std::net — TCP/UDP sockets, HTTP client, DNS resolution
std::collections — HashMap, BTreeMap, VecDeque, PriorityQueue, BitSet
std::text — string manipulation, regex, Unicode normalization, templating
std::math — numeric operations, random number generation, big integers
std::time — clocks, durations, date/time parsing, timezones
std::json — JSON parsing/serialization with streaming support
std::csv — CSV parsing with type inference
std::crypto — hashing (SHA-256, BLAKE3), HMAC, AES, RSA, Ed25519
std::process — subprocess spawning, signal handling, environment variables
std::async — async/await runtime, channels, timers
std::test — test runner, assertions, property-based testing, benchmarks

Pipeline-first API design

Every standard library function takes its primary data as the first argument. This is the fundamental rule that makes pipelines work:

// Good — data flows naturally through the pipeline
let result = users
    |> filter(|u| u.active)
    |> sort_by(|u| u.name)
    |> map(|u| u.email)
    |> join(", ")

// This works because:
// filter(collection, predicate) — collection is first
// sort_by(collection, key_fn)  — collection is first
// map(collection, transform)   — collection is first
// join(collection, separator)  — collection is first

When we reviewed other standard libraries (Rust, Go, Python), many had inconsistent argument ordering. A function might take a config object first, or a destination buffer. We spent weeks auditing our API surface to ensure the "data first" rule has zero exceptions.

Zero-copy I/O

The I/O module is designed around zero-copy streaming. Large files are never loaded entirely into memory:

// Process a 10 GB log file with constant memory usage
read_lines("/var/log/huge.log")       // Returns a lazy iterator
    |> filter(|line| line |> contains("ERROR"))
    |> map(|line| parse_log_entry(line))
    |> group_by(|entry| entry.module)
    |> map_values(|entries| entries.len())
    |> to_json()
    |> write_file("error-summary.json")
// Peak memory: ~64 KB (one read buffer + one write buffer)

The compiler recognizes pipeline chains on iterators and fuses them into a single pass. The filter |> map |> group_by chain above compiles to a single loop that reads one line at a time.

The crypto module

A language designed for security tools needs serious cryptography. The std::crypto module wraps vetted C implementations (libsodium for modern constructions, OpenSSL for legacy compatibility):

use std::crypto::{hash, aes, rsa}

// Hashing
let digest = "hello world" |> hash::sha256()
let fast_hash = large_data |> hash::blake3()

// Symmetric encryption
let key = aes::generate_key(256)
let encrypted = plaintext |> aes::encrypt_gcm(key)
let decrypted = encrypted |> aes::decrypt_gcm(key)

// Asymmetric encryption
let (pub_key, priv_key) = rsa::generate_keypair(4096)
let signature = message |> rsa::sign(priv_key)
let valid = message |> rsa::verify(signature, pub_key)

We deliberately chose not to implement crypto in pure Lateralus. Cryptographic implementations need constant-time guarantees, side-channel resistance, and years of security audits. Wrapping proven C libraries via the C99 FFI is both safer and faster.