Example: validate on the server
The browser is not the last word on a submission. In this example the page delivers the envelope that datum-submit carries to an endpoint of its own server, and the endpoint judges it again with the same sandbox program the browser runs, over the server's own copy of the form and the rules. The verdict comes back as codes and fields, never text, so the page words a 422 with the same messages it uses for its own errors, through set. Nothing is written again in the server's language, and nothing the page did is trusted.
The endpoint is given in four stacks: Python with FastAPI, Java with Spring Boot, C# with ASP.NET Core and Node.js with Express.
The page
There is no server behind these docs. The result under the code is answered by a stand-in in the browser that runs the same program, from the same built files, over the same form and rules. The endpoint files further down are what a real server runs.
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<title>Customer intake</title>
<link rel="stylesheet" href="/datum/datum-form.css">
<script type="module" src="/datum/datum-form.js"></script>
</head>
<body>
<!-- No sandbox on the page: field constraints validate in the bundle, the rules run on the server. -->
<datum-form
id="intake"
src="/forms/customer-intake.json"
></datum-form>
<script type="module">
const form = document.getElementById("intake");
form.dataSources = {
countries: [
{ value: "CA", label: "Canada" },
{ value: "US", label: "United States" },
{ value: "MX", label: "Mexico" }
]
};
// The server answers with codes, as the browser's sandbox does; these word them per language.
form.messages = {
required: {
default: "This field is required.",
locales: { "fr-CA": "Ce champ est requis." }
},
"custom.fullNameIncomplete": {
default: "Enter your first and last name.",
locales: { "fr-CA": "Entrez votre prénom et votre nom." }
},
"custom.phoneNotOffered": {
default: "Phone contact is not offered in that country yet.",
locales: {
"fr-CA": "Le contact par téléphone n'est pas encore offert dans ce pays."
}
},
"wasm.execution_error": {
default: "Validation could not run. Try again.",
locales: { "fr-CA": "La validation n'a pas pu s'exécuter. Réessayez." }
}
};
form.addEventListener("datum-submit", ({ detail }) => {
if (!detail.accepted) return; // the form shows its own validation errors
detail.waitUntil(deliver(detail));
});
// The server runs the same program over its own copy of the form and answers 422 with the
// same error contract, or accepts. What comes back goes into the form through setErrors.
async function deliver(detail) {
try {
const response = await fetch("/api/submissions", {
method: "POST",
headers: { "content-type": "application/json" },
body: JSON.stringify({ document: detail.submission, submissionKey: detail.submissionKey })
});
if (response.ok) {
form.setNotice({ kind: "success", title: "Received" });
return;
}
// { errors: [{ field, code }] }: codes and fields, never text. The messages above word them.
const { errors } = await response.json();
form.setErrors(errors.map((e) => ({ componentId: e.field ?? undefined, code: e.code })));
} catch {
form.setErrors([{ code: "delivery.failed", message: "The form could not be sent. Try again." }]);
}
}
</script>
</body>
</html>Nothing yet.
Type a first name alone, such as "Ada", and submit. The form lets it through, the server answers 422 with custom. on full, and the page's message appears on the field. Choose Mexico and Phone for the second rule, custom.. Then turn on Run the same rules in the browser too: the page loads the same rules file into its own sandbox with wasm="auto", and the same answers are refused before they leave the page.
The log shows each delivery and the run log the endpoint forwards to its own logger: the program's steps and what the rules wrote with context..
The rules both sides read
The rules are one file, rules/, a plain module whose validators return codes. The server reads its copy from disk; with the browser option on, the page fetches the same file. It is on the rules/ tab of the example above.
The endpoint
Each file is complete. Its first lines say what to install and how to fetch the engine, a WASM build of a JavaScript engine (qjs-wasi.), from its release page. Beside it the server keeps the sandbox program datum-sandbox-program. from the bundle folder (the Java and Node.js files also copy datum-pdf-program. at startup, which this endpoint does not use), its copy of the form at forms/ and the rules at rules/.
Python (FastAPI)
# main.py — the endpoint the page delivers to.
# pip install fastapi uvicorn wasmtime
# curl -LO https://github.com/quickjs-ng/quickjs/releases/download/v0.16.2/qjs-wasi.wasm
# uvicorn main:app
#
# The submission is re-validated in the very program the browser's sandbox runs
# (datum-sandbox-program.js, written beside the bundles), in a QuickJS of this process's own —
# QuickJS-ng's WebAssembly build, run by wasmtime: the same kernel over the same schema, the
# same rules, so the same verdict — codes and fields, never text — and the same sanitized
# values. Nothing is re-implemented here; nothing the page did is trusted either.
import json
import logging
import math
import tempfile
import threading
import time
from pathlib import Path
from fastapi import FastAPI
from fastapi.responses import JSONResponse
from wasmtime import Config, Engine, ExitTrap, Linker, Module, Store, Trap, WasiConfig
FORM = json.loads(Path("forms/customer-intake.json").read_text())
# The journey the answers were given through, from this server's own copy of the form: the
# envelope names it (submission.journey.id) and nothing more. Handed to the sandbox, it has the
# kernel work the path out again from the values, as the browser did: an answer on a page the
# path skipped is judged hidden, and a form field one of the journey's own fields fills or works
# out counts as asked on that page. None named, or one the form does not have: null, and the
# answers are judged as the whole form, which asks for every required field.
def journey_of(submission: dict):
named = (submission.get("journey") or {}).get("id")
return next((j for j in FORM.get("journey", {}).get("journeys", []) if j["id"] == named), None)
# The engine: qjs-wasi.wasm, QuickJS-ng's own WebAssembly build from its release page, compiled
# once and shared. Every request runs it afresh in a store of its own, so requests need no lock.
config = Config()
config.epoch_interruption = True # the time budget: a run past its deadline traps
engine = Engine(config)
QJS = Module.from_file(engine, "qjs-wasi.wasm")
linker = Linker(engine)
linker.define_wasi()
# The clock every budget is kept by, one for all the runs at once: a tick every 50 ms; a run's
# deadline is its budget in ticks, plus one for the tick already under way when it started. The
# epoch it advances is the engine's, shared by every store on it, so no run may bump it for
# itself: a timer of the run's own, fired when its budget was spent, would advance the epoch under
# every other run in flight too, and cut short one that had set its deadline a tick earlier — a
# wrong verdict for a submission that was never judged. One clock, and no run can touch another.
TICK = 0.05
def clock() -> None:
while True:
time.sleep(TICK)
engine.increment_epoch()
threading.Thread(target=clock, daemon=True).start()
class SandboxError(Exception):
"""A run that ended without an outcome: a limit cut it off, or a program or rule module would not load."""
def qjs(argv: list, request: str, seconds: float, memory: int) -> str:
# One run: this directory mounted read-only as /app (the programs), the request on stdin, the
# answer on stdout — all the run can reach; wasmtime hands it nothing else. The limits are
# wasmtime's, at the browser's profile: the budget in seconds, the memory in bytes. Runaway
# recursion exhausts the instance's own stack and traps, which ends the run too.
out, err = [], []
with tempfile.NamedTemporaryFile("w", suffix=".json") as stdin:
stdin.write(request)
stdin.flush()
store = Store(engine)
store.set_limits(memory_size=memory)
store.set_epoch_deadline(math.ceil(seconds / TICK) + 1)
wasi = WasiConfig()
wasi.argv = ["qjs", "--std", *argv] # --std: the std module, which reads stdin
wasi.preopen_dir(".", "/app", fs_mutable=False)
wasi.stdin_file = stdin.name
wasi.stdout_custom = out.append
wasi.stderr_custom = err.append
store.set_wasi(wasi)
try:
linker.instantiate(store, QJS).exports(store)["_start"](store)
except ExitTrap as exit: # a throw: qjs printed it to stderr and exited 1
if exit.code != 0:
raise SandboxError(b"".join(err).decode().strip()) from exit
except Trap as trap: # "interrupt": the budget; "stack_overflow": the recursion
raise SandboxError(f"{trap.trap_code}") from trap
return b"".join(out).decode().rstrip("\n")
# What one run evaluates: the program, the page's rules registered into it, then the call — the
# context from stdin, the outcome to stdout. The program itself disables Date and Math.random.
RULES = Path("rules/intake-rules.js").read_text()
VALIDATE = ["-I", "/app/datum-sandbox-program.js", "-e", "__datum_register(" + RULES + "); print(__datum_run(std.in.readAsString()));"]
# What the rules read beside the values. The page injected a copy of its own; the server has the say.
PHONE_COUNTRIES = ["CA", "US"]
# What the Country select shows, resolved here: the program never fetches a data source. An answer to a
# choice whose source is missing from the optionSnapshot below is refused (option.sourceUnavailable).
COUNTRIES = [{"value": "CA", "label": "Canada"}, {"value": "US", "label": "United States"}, {"value": "MX", "label": "Mexico"}]
# The run log comes back with the outcome — the sandbox has no console — and goes to this
# process's logger: what the kernel did, step by step, and what the rules said, at LOG_LEVEL.
LOG_LEVEL = "debug" # "off", "error", "warn" (the kernel's default), "info" or "debug"
LEVELS = {"debug": logging.DEBUG, "info": logging.INFO, "warn": logging.WARNING, "error": logging.ERROR}
log = logging.getLogger("datum.sandbox")
logging.basicConfig(level=logging.DEBUG) # so the forwarded entries reach the console
app = FastAPI()
@app.post("/api/submissions")
def submit(body: dict):
key = body.get("submissionKey") # one per attempt: the same key again is the same submission
if not isinstance(key, str) or not key: # without one a retry could not be told from a new submission
return JSONResponse(status_code=400, content={"errors": [{"code": "bad_document"}]})
submission = body["document"]["submission"] # the delivery is { document, submissionKey }
context = {
"formId": FORM["id"],
"form": FORM["root"], # the schema: field constraints and sanitization come from it
"values": submission["values"],
"locale": submission.get("locale", "en"),
"externalData": {"phoneCountries": PHONE_COUNTRIES},
"optionSnapshot": {"countries": COUNTRIES}, # the lists a choice's data source holds, by source id
"journey": journey_of(submission), # the path, worked out again from the values (see journey_of)
"logLevel": LOG_LEVEL,
}
try:
outcome = json.loads(qjs(VALIDATE, json.dumps(context), 0.25, 32 * 1024 * 1024))
except SandboxError: # a limit cut the run off, or a rule module would not load: fail closed, as the browser does
log.exception("sandbox failed for %s", FORM["id"])
return JSONResponse(status_code=503, content={"errors": [{"code": "wasm.execution_error"}]})
for entry in outcome.get("logs", []):
log.log(LEVELS[entry["level"]], "%s %s", entry["message"], json.dumps(entry["data"]) if "data" in entry else "")
if outcome["errors"]:
# 422 with the error contract, as the browser would have seen it; the page's messages word it.
errors = [{"field": e.get("componentId"), "code": e["code"]} for e in outcome["errors"]]
return JSONResponse(status_code=422, content={"errors": errors})
store(key, outcome["values"]) # the sanitized values: what the form defines, nothing else
return {"ok": True}
def store(key: str, values: dict) -> None:
... # wherever submissions go: one record per key: the same key with the same values is that record again; the same key with other values is refused (409 submission.keyConflict)Java (Spring Boot)
// Application.java — the endpoint the page delivers to.
// pom.xml beside it: Spring Boot, and Endive with its Maven plugin, which compiles the engine in.
// curl -Lo src/main/resources/qjs-wasi.wasm https://github.com/quickjs-ng/quickjs/releases/download/v0.16.2/qjs-wasi.wasm
// mvn spring-boot:run
//
// The submission is re-validated in the very program the browser's sandbox runs
// (datum-sandbox-program.js, written beside the bundles), in a QuickJS of this process's own —
// QuickJS-ng's WebAssembly build, run by Endive: the same kernel over the same schema, the
// same rules, so the same verdict — codes and fields, never text — and the same sanitized
// values. Nothing is re-implemented here; nothing the page did is trusted either.
package datum;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.UncheckedIOException;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import com.google.common.jimfs.Configuration;
import com.google.common.jimfs.Jimfs;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.slf4j.event.Level;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
import run.endive.runtime.ImportValues;
import run.endive.wasi.WasiExitException;
import run.endive.wasi.WasiOptions;
import run.endive.wasi.WasiPreview1;
import run.endive.wasm.types.MemoryLimits;
import tools.jackson.databind.JsonNode;
import tools.jackson.databind.json.JsonMapper;
@SpringBootApplication
@RestController
public class Application {
static final JsonMapper JSON = JsonMapper.shared();
static final JsonNode FORM = JSON.readTree(text("forms/customer-intake.json"));
// The journey the answers were given through, from this server's own copy of the form: the
// envelope names it (submission.journey.id) and nothing more. Handed to the sandbox, it has the
// kernel work the path out again from the values, as the browser did: an answer on a page the
// path skipped is judged hidden, and a form field one of the journey's own fields fills or works
// out counts as asked on that page. None named, or one the form does not have: null, and the
// answers are judged as the whole form, which asks for every required field.
static JsonNode journeyOf(JsonNode submission) {
for (JsonNode journey : FORM.path("journey").path("journeys")) {
if (journey.path("id").equals(submission.path("journey").path("id"))) {
return journey;
}
}
return null; // set as JSON null: no journey
}
// The engine: qjs-wasi.wasm, QuickJS-ng's own WebAssembly build from its release page, compiled once,
// at build time, by Endive's plugin (see pom.xml) into the class Qjs — native code where the platform
// has it, JVM bytecode where not. Every request runs it afresh in an instance of its own, so requests
// need no lock. The module's own initial memory, which a limit must keep.
static final int QJS_PAGES = Qjs.load().memorySection().get().getMemory(0).limits().initialPages();
// What a run can reach: one in-memory directory holding the two programs, and nothing else — not this
// directory, not the disk. Endive mounts any java.nio path, and Jimfs makes one.
static final Path APP = programs();
static Path programs() {
try {
var app = Files.createDirectories(Jimfs.newFileSystem(Configuration.unix()).getPath("/app"));
for (var name : List.of("datum-sandbox-program.js", "datum-pdf-program.js")) {
Files.copy(Path.of(name), app.resolve(name));
}
return app;
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
static {
// A JVM's first runs are slow until its JIT has compiled the runner's own setup, several times
// over what they settle to: pay for that at start, not on the first requests.
for (int i = 0; i < 3; i++) {
try {
qjs(List.of("-e", "0"), "", 30, 32 * 1024 * 1024);
} catch (SandboxException ignored) {
// the budget is wide; a failure here shows on the first request instead
}
}
}
/** A run that ended without an outcome: a limit cut it off, or a program or rule module would not load. */
static class SandboxException extends Exception {
SandboxException(String message) {
super(message);
}
}
static String qjs(List<String> argv, String request, double seconds, int memory) throws SandboxException {
// One run: the programs mounted as /app, the request on stdin, the answer on stdout — all the run can
// reach; Endive hands it nothing else. The limits are Endive's, at the browser's profile: the budget in
// seconds — past it the run is interrupted, native or bytecode — and the memory in bytes. The budget
// is this run's alone, a Future of its own cancelled at the timeout, so a run past it interrupts no
// other: there is no engine-wide clock here, as there is under wasmtime (see main.py). Runaway
// recursion exhausts the instance's own stack and traps, which ends the run too.
var out = new ByteArrayOutputStream();
var err = new ByteArrayOutputStream();
var args = new ArrayList<>(List.of("qjs", "--std")); // --std: the std module, which reads stdin
args.addAll(argv);
var options = WasiOptions.builder()
.withArguments(args)
.withDirectory("/app", APP)
.withStdin(new ByteArrayInputStream(request.getBytes(StandardCharsets.UTF_8)))
.withStdout(out)
.withStderr(err)
.build();
var worker = Executors.newSingleThreadExecutor();
try (var wasi = WasiPreview1.builder().withOptions(options).build()) {
var imports = ImportValues.builder().addFunction(wasi.toHostFunctions()).build();
var run = worker.submit(() -> {
// Building the instance runs _start: qjs's main, through to its exit.
try (var instance = Qjs.builder()
.withImportValues(imports)
.withMemoryLimits(new MemoryLimits(QJS_PAGES, memory / 65536))
.build()) {
return null;
}
});
try {
run.get((long) (seconds * 1000), TimeUnit.MILLISECONDS);
} catch (TimeoutException budget) {
run.cancel(true); // the interrupt that ends the run
throw new SandboxException("interrupt");
}
} catch (ExecutionException failed) {
var cause = failed.getCause();
if (cause instanceof WasiExitException exit) { // a throw: qjs printed it to stderr and exited 1
if (exit.exitCode() != 0) {
throw new SandboxException(err.toString(StandardCharsets.UTF_8).strip());
}
} else { // a trap: "call stack exhausted", "out of bounds memory access"
throw new SandboxException(cause.getMessage());
}
} catch (InterruptedException interrupted) {
Thread.currentThread().interrupt();
throw new SandboxException("interrupted");
} finally {
worker.shutdownNow();
}
return out.toString(StandardCharsets.UTF_8).stripTrailing();
}
static String text(String file) {
try {
return Files.readString(Path.of(file));
} catch (IOException e) {
throw new UncheckedIOException(e);
}
}
// What one run evaluates: the program, the page's rules registered into it, then the call — the
// context from stdin, the outcome to stdout. The program itself disables Date and Math.random.
static final String RULES = text("rules/intake-rules.js");
static final List<String> VALIDATE = List.of("-I", "/app/datum-sandbox-program.js", "-e", "__datum_register(" + RULES + "); print(__datum_run(std.in.readAsString()));");
// What the rules read beside the values. The page injected a copy of its own; the server has the say.
static final List<String> PHONE_COUNTRIES = List.of("CA", "US");
// What the Country select shows, resolved here: the program never fetches a data source. An answer to a
// choice whose source is missing from the optionSnapshot below is refused (option.sourceUnavailable).
static final List<Map<String, String>> COUNTRIES = List.of(
Map.of("value", "CA", "label", "Canada"), Map.of("value", "US", "label", "United States"), Map.of("value", "MX", "label", "Mexico"));
// The run log comes back with the outcome — the sandbox has no console — and goes to this
// process's logger: what the kernel did, step by step, and what the rules said, at LOG_LEVEL.
static final String LOG_LEVEL = "debug"; // "off", "error", "warn" (the kernel's default), "info" or "debug"
static final Map<String, Level> LEVELS = Map.of("debug", Level.DEBUG, "info", Level.INFO, "warn", Level.WARN, "error", Level.ERROR);
static final Logger log = LoggerFactory.getLogger("datum.sandbox");
public static void main(String[] args) {
var app = new SpringApplication(Application.class);
app.setDefaultProperties(Map.of("logging.level.datum", "debug")); // so the forwarded entries reach the console
app.run(args);
}
record FieldError(String field, String code) {}
@PostMapping("/api/submissions")
public ResponseEntity<?> submit(@RequestBody JsonNode body) {
var key = body.path("submissionKey").asString(null); // one per attempt: the same key again is the same submission
if (key == null || key.isBlank()) { // without it a retry could not be told from a new submission
return ResponseEntity.status(400).body(Map.of("errors", List.of(new FieldError(null, "bad_document"))));
}
var submission = body.path("document").path("submission");
var context = JSON.createObjectNode()
.put("formId", FORM.path("id").asString())
.set("form", FORM.path("root")) // the schema: field constraints and sanitization come from it
.set("values", submission.path("values"))
.put("locale", submission.path("locale").asString("en"))
.set("externalData", JSON.valueToTree(Map.of("phoneCountries", PHONE_COUNTRIES)))
.set("optionSnapshot", JSON.valueToTree(Map.of("countries", COUNTRIES))) // the lists a choice's data source holds, by source id
.set("journey", journeyOf(submission)) // the path, worked out again from the values (see journeyOf)
.put("logLevel", LOG_LEVEL);
JsonNode outcome;
try {
outcome = JSON.readTree(qjs(VALIDATE, JSON.writeValueAsString(context), 5, 32 * 1024 * 1024));
} catch (SandboxException failure) { // a limit cut the run off, or a rule module would not load: fail closed, as the browser does
log.error("sandbox failed for {}", FORM.path("id").asString(), failure);
return ResponseEntity.status(503).body(Map.of("errors", List.of(new FieldError(null, "wasm.execution_error"))));
}
for (JsonNode entry : outcome.path("logs")) {
log.atLevel(LEVELS.get(entry.path("level").asString())).log("{} {}", entry.path("message").asString(), entry.path("data"));
}
if (!outcome.path("errors").isEmpty()) {
// 422 with the error contract, as the browser would have seen it; the page's messages word it.
var errors = new ArrayList<FieldError>();
for (JsonNode e : outcome.path("errors")) {
errors.add(new FieldError(e.path("componentId").asString(null), e.path("code").asString()));
}
return ResponseEntity.status(422).body(Map.of("errors", errors));
}
store(key, outcome.path("values")); // the sanitized values: what the form defines, nothing else
return ResponseEntity.ok(Map.of("ok", true));
}
static void store(String key, JsonNode values) {
// wherever submissions go: one record per key: the same key with the same values is that record again; the same key with other values is refused (409 submission.keyConflict)
}
}<!-- pom.xml — beside Application.java. Spring Boot for the endpoint; Endive, the Bytecode Alliance's
WebAssembly runtime for the JVM, for the engine: its Maven plugin compiles qjs-wasi.wasm at build
time — to native code through Redline (Cranelift, the code generator behind wasmtime) for the six
platforms it knows, and to JVM bytecode as the fallback for any other — into the class datum.Qjs,
packaged in the jar. No native library to install, no JNI: the jar runs wherever a JDK does. Redline
is experimental in Endive 1.1.0; the bytecode runs the same program the same way, more slowly. -->
<project xmlns="http://maven.apache.org/POM/4.0.0">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>4.1.1</version>
</parent>
<groupId>example</groupId>
<artifactId>intake</artifactId>
<version>1.0.0</version>
<properties>
<!-- The native runner uses the Panama FFM API, so Java 25; on 17 to 24, redline-runner-jffi-experimental instead. -->
<java.version>25</java.version>
<endive.version>1.1.0</endive.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webmvc</artifactId>
</dependency>
<!-- Endive: the WASI layer (stdin, stdout, the mount) and the runner for the native code. -->
<dependency>
<groupId>run.endive</groupId>
<artifactId>wasi</artifactId>
<version>${endive.version}</version>
</dependency>
<dependency>
<groupId>run.endive</groupId>
<artifactId>redline-runner-experimental</artifactId>
<version>${endive.version}</version>
</dependency>
<!-- An in-memory file system: the one directory a run is given. -->
<dependency>
<groupId>com.google.jimfs</groupId>
<artifactId>jimfs</artifactId>
<version>1.3.2</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<!-- The native runner is a restricted API: without this the JDK warns at the first run. -->
<jvmArguments>--enable-native-access=ALL-UNNAMED</jvmArguments>
</configuration>
</plugin>
<plugin>
<groupId>run.endive</groupId>
<artifactId>endive-compiler-maven-plugin</artifactId>
<version>${endive.version}</version>
<executions>
<execution>
<goals>
<goal>compile</goal>
</goals>
<configuration>
<name>datum.Qjs</name>
<wasmFile>src/main/resources/qjs-wasi.wasm</wasmFile>
<redlineExperimental>true</redlineExperimental>
<!-- QuickJS's interpreter loop, JS_CallInternal, is too big for one JVM method: the bytecode
fallback interprets that one function. The native code has no such limit. -->
<interpreterFallback>WARN</interpreterFallback>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>C# (ASP.NET Core)
// Program.cs — the endpoint the page delivers to.
// intake.csproj beside it: ASP.NET Core, and Wasmtime, the .NET embedding of wasmtime.
// curl -LO https://github.com/quickjs-ng/quickjs/releases/download/v0.16.2/qjs-wasi.wasm
// dotnet run
//
// The submission is re-validated in the very program the browser's sandbox runs
// (datum-sandbox-program.js, written beside the bundles), in a QuickJS of this process's own —
// QuickJS-ng's WebAssembly build, run by wasmtime: the same kernel over the same schema, the
// same rules, so the same verdict — codes and fields, never text — and the same sanitized
// values. Nothing is re-implemented here; nothing the page did is trusted either.
using System.Text.Json;
using System.Text.Json.Nodes;
using Wasmtime;
var form = JsonNode.Parse(File.ReadAllText("forms/customer-intake.json"))!;
// The journey the answers were given through, from this server's own copy of the form: the
// envelope names it (submission.journey.id) and nothing more. Handed to the sandbox, it has the
// kernel work the path out again from the values, as the browser did: an answer on a page the
// path skipped is judged hidden, and a form field one of the journey's own fields fills or works
// out counts as asked on that page. None named, or one the form does not have: null, and the
// answers are judged as the whole form, which asks for every required field.
JsonNode? JourneyOf(JsonNode submission) =>
form["journey"]?["journeys"]?.AsArray().FirstOrDefault(journey => (string?)journey!["id"] == (string?)submission["journey"]?["id"]);
// The engine: qjs-wasi.wasm, QuickJS-ng's own WebAssembly build from its release page, compiled
// once, now, and shared (the Sandbox class below). Every request runs it afresh in a store of its
// own, so requests need no lock.
var sandbox = new Sandbox();
// What one run evaluates: the program, the page's rules registered into it, then the call — the
// context from stdin, the outcome to stdout. The program itself disables Date and Math.random.
var rules = File.ReadAllText("rules/intake-rules.js");
string[] validate = ["-I", "/app/datum-sandbox-program.js", "-e", "__datum_register(" + rules + "); print(__datum_run(std.in.readAsString()));"];
// What the rules read beside the values. The page injected a copy of its own; the server has the say.
string[] phoneCountries = ["CA", "US"];
// What the Country select shows, resolved here: the program never fetches a data source. An answer to a
// choice whose source is missing from the optionSnapshot below is refused (option.sourceUnavailable).
object[] countries = [new { value = "CA", label = "Canada" }, new { value = "US", label = "United States" }, new { value = "MX", label = "Mexico" }];
// The run log comes back with the outcome — the sandbox has no console — and goes to this
// process's logger: what the kernel did, step by step, and what the rules said, at logLevel.
var logLevel = "debug"; // "off", "error", "warn" (the kernel's default), "info" or "debug"
var levels = new Dictionary<string, LogLevel> { ["debug"] = LogLevel.Debug, ["info"] = LogLevel.Information, ["warn"] = LogLevel.Warning, ["error"] = LogLevel.Error };
var builder = WebApplication.CreateBuilder(args);
builder.Logging.AddFilter("datum", LogLevel.Debug); // so the forwarded entries reach the console
var app = builder.Build();
var log = app.Services.GetRequiredService<ILoggerFactory>().CreateLogger("datum.sandbox");
app.MapPost("/api/submissions", (JsonNode body) =>
{
var key = body["submissionKey"] is JsonValue keyValue && keyValue.TryGetValue<string>(out var text) ? text : null; // one per attempt: the same key again is the same submission
if (string.IsNullOrWhiteSpace(key)) // without it a retry could not be told from a new submission
{
return Results.Json(new { errors = new[] { new { code = "bad_document" } } }, statusCode: 400);
}
var submission = body["document"]!["submission"]!;
var context = JsonSerializer.Serialize(new
{
formId = form["id"],
form = form["root"], // the schema: field constraints and sanitization come from it
values = submission["values"],
locale = (string?)submission["locale"] ?? "en",
externalData = new { phoneCountries },
optionSnapshot = new { countries }, // the lists a choice's data source holds, by source id
journey = JourneyOf(submission), // the path, worked out again from the values (see JourneyOf)
logLevel
});
JsonNode outcome;
try
{
outcome = JsonNode.Parse(sandbox.Run(validate, context, 0.25, 32 * 1024 * 1024))!;
}
catch (SandboxException failure) // a limit cut the run off, or a program or rule module would not load: fail closed, as the browser does
{
log.LogError(failure, "sandbox failed for {Form}", (string?)form["id"]);
return Results.Json(new { errors = new[] { new { code = "wasm.execution_error" } } }, statusCode: 503);
}
foreach (var entry in outcome["logs"]?.AsArray() ?? [])
{
log.Log(levels[(string)entry!["level"]!], "{Message} {Data}", (string?)entry["message"], entry["data"]?.ToJsonString() ?? "");
}
if (outcome["errors"]!.AsArray().Count > 0)
{
// 422 with the error contract, as the browser would have seen it; the page's messages word it.
var errors = outcome["errors"]!.AsArray().Select(e => new { field = (string?)e!["componentId"], code = (string?)e["code"] });
return Results.Json(new { errors }, statusCode: 422);
}
Store(key, outcome["values"]!); // the sanitized values: what the form defines, nothing else
return Results.Ok(new { ok = true });
});
app.Run();
static void Store(string key, JsonNode values)
{
// wherever submissions go: one record per key: the same key with the same values is that record again; the same key with other values is refused (409 submission.keyConflict)
}
/// <summary>A run that ended without an outcome: a limit cut it off, or a program or rule module would not load.</summary>
class SandboxException(string message) : Exception(message);
/// <summary>The engine, compiled once as the process starts, and one run of it per request.</summary>
class Sandbox
{
readonly Engine engine = new(new Config().WithEpochInterruption(true)); // the time budget: a run past its deadline traps
readonly Module qjs;
readonly Linker linker;
// The clock every budget is kept by, one for all the runs at once: a tick every 50 ms; a run's
// deadline is its budget in ticks, plus one for the tick already under way when it started. The
// epoch it advances is the engine's, shared by every store on it, so no run may bump it for
// itself: a timer of the run's own, fired when its budget was spent, would advance the epoch under
// every other run in flight too, and cut short one that had set its deadline a tick earlier — a
// wrong verdict for a submission that was never judged. One clock, and no run can touch another.
static readonly TimeSpan Tick = TimeSpan.FromSeconds(0.05);
readonly Timer clock;
public Sandbox()
{
qjs = Module.FromFile(engine, "qjs-wasi.wasm");
linker = new Linker(engine);
linker.DefineWasi();
clock = new Timer(_ => engine.IncrementEpoch(), null, Tick, Tick);
}
public string Run(string[] argv, string request, double seconds, long memory)
{
// One run: this directory mounted read-only as /app (the programs), the request on stdin, the
// answer on stdout — all the run can reach; wasmtime hands it nothing else. The limits are
// wasmtime's, at the browser's profile: the budget in seconds, the memory in bytes. Runaway
// recursion exhausts the instance's own stack and traps, which ends the run too. WASI's
// standard streams are files here: the request goes in as one, the answer comes back as one.
var stdin = Path.GetTempFileName();
var stdout = Path.GetTempFileName();
var stderr = Path.GetTempFileName();
try
{
File.WriteAllText(stdin, request);
string[] args = ["qjs", "--std", .. argv]; // --std: the std module, which reads stdin
using var store = new Store(engine);
store.SetLimits(memorySize: memory);
store.SetEpochDeadline((ulong)Math.Ceiling(seconds / Tick.TotalSeconds) + 1);
store.SetWasiConfiguration(new WasiConfiguration()
.WithArgs(args)
.WithPreopenedDirectory(".", "/app", WasiDirectoryPermissions.Read, WasiFilePermissions.Read)
.WithStandardInput(stdin)
.WithStandardOutput(stdout)
.WithStandardError(stderr));
try
{
linker.Instantiate(store, qjs).GetFunction("_start")!.Invoke();
}
catch (WasmtimeException exit) when (exit.ExitCode is int code) // a throw: qjs printed it to stderr and exited 1
{
if (code != 0)
{
throw new SandboxException(File.ReadAllText(stderr).Trim());
}
}
catch (TrapException trap) // Interrupt: the budget; StackOverflow: the recursion
{
throw new SandboxException(trap.Type.ToString());
}
return File.ReadAllText(stdout).TrimEnd('\n');
}
finally
{
File.Delete(stdin);
File.Delete(stdout);
File.Delete(stderr);
}
}
}<!-- intake.csproj — beside Program.cs. ASP.NET Core for the endpoint; Wasmtime, the Bytecode Alliance's
.NET embedding of wasmtime, for the engine: the runtime main.py runs under, as a NuGet package that
carries its native library for Windows, macOS and Linux on x64 and arm64. Nothing else to install. -->
<Project Sdk="Microsoft.NET.Sdk.Web">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Wasmtime" Version="44.0.0" />
</ItemGroup>
</Project>Node.js (Express)
// main.mjs — the endpoint the page delivers to.
// qjs.mjs beside it: the engine, one process per run — Node 22 or later, nothing to install for it.
// npm install express
// curl -LO https://github.com/quickjs-ng/quickjs/releases/download/v0.16.2/qjs-wasi.wasm
// node main.mjs
//
// The submission is re-validated in the very program the browser's sandbox runs
// (datum-sandbox-program.js, written beside the bundles), in a QuickJS of this process's own —
// QuickJS-ng's WebAssembly build, run by node:wasi: the same kernel over the same schema, the
// same rules, so the same verdict — codes and fields, never text — and the same sanitized
// values. Nothing is re-implemented here; nothing the page did is trusted either.
import { spawn } from "node:child_process";
import { copyFileSync, mkdtempSync, readFileSync, rmSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { fileURLToPath } from "node:url";
import express from "express";
const FORM = JSON.parse(readFileSync("forms/customer-intake.json", "utf8"));
// The journey the answers were given through, from this server's own copy of the form: the
// envelope names it (submission.journey.id) and nothing more. Handed to the sandbox, it has the
// kernel work the path out again from the values, as the browser did: an answer on a page the
// path skipped is judged hidden, and a form field one of the journey's own fields fills or works
// out counts as asked on that page. None named, or one the form does not have: null, and the
// answers are judged as the whole form, which asks for every required field.
const journeyOf = (submission) => FORM.journey?.journeys?.find((journey) => journey.id === submission.journey?.id) ?? null;
// The engine: qjs-wasi.wasm, QuickJS-ng's own WebAssembly build from its release page, on V8 with
// node:wasi (qjs.mjs) — nothing to install. Every request runs it in a process of its own, so requests
// need no lock and share nothing: V8 has no store to hang a limit on, as wasmtime has, so the
// operating system draws the line instead, and the flag and the timeout below are that process's.
const RUNNER = fileURLToPath(new URL("qjs.mjs", import.meta.url));
// What a run can reach: one directory holding the two programs, and nothing else — not this
// directory, not the disk. Node's WASI mounts a directory read-write and does not promise the
// sandbox wasmtime's does, so a run is started in a directory of its own with nothing else in it.
const APP = programs();
function programs() {
const app = mkdtempSync(join(tmpdir(), "datum-"));
for (const name of ["datum-sandbox-program.js", "datum-pdf-program.js"]) {
copyFileSync(name, join(app, name));
}
// Gone with the process: at its exit, and on the signals that would otherwise end it without one.
process.on("exit", () => rmSync(app, { recursive: true, force: true }));
for (const signal of ["SIGINT", "SIGTERM"]) process.once(signal, () => process.exit());
return app;
}
/** A run that ended without an outcome: a limit cut it off, or a program or rule module would not load. */
class SandboxError extends Error {}
function qjs(argv, request, seconds, memory) {
// One run: a process started in the programs' directory, which it mounts as /app, the request on
// its stdin, the answer on its stdout — all the run can reach; the runner hands it nothing else.
// The limits are the process's, at the browser's profile: the memory in bytes, as the pages V8
// lets the module's memory grow to, and the budget in seconds, a timeout that ends the process —
// this run's alone, so a run past it interrupts no other: there is no engine-wide clock here, as
// there is under wasmtime (see main.py). The budget counts the process's own start and the
// module's compilation, a fifth of a second: it is a bound on a runaway rule, not a latency to
// meet. Runaway recursion ends the run too: QuickJS's own stack check throws, and V8 traps the
// instance's stack if it does not.
return new Promise((resolve, reject) => {
const flags = ["--wasm-max-mem-pages=" + memory / 65536, "--disable-warning=ExperimentalWarning"];
const run = spawn(process.execPath, [...flags, RUNNER, ...argv], { cwd: APP, timeout: seconds * 1000 });
const out = [];
const err = [];
run.stdout.on("data", (chunk) => out.push(chunk));
run.stderr.on("data", (chunk) => err.push(chunk));
run.stdin.on("error", () => {}); // a run that ended before reading its request: what ended it is on stderr
run.stdin.end(request);
run.on("error", reject);
run.on("close", (code, signal) => {
if (signal !== null) { // the budget: the timeout ended the process
reject(new SandboxError("interrupt"));
} else if (code !== 0) { // a throw: qjs printed it to stderr and exited 1
reject(new SandboxError(Buffer.concat(err).toString().trim()));
} else {
resolve(Buffer.concat(out).toString().trimEnd());
}
});
});
}
// What one run evaluates: the program, the page's rules registered into it, then the call — the
// context from stdin, the outcome to stdout. The program itself disables Date and Math.random.
const RULES = readFileSync("rules/intake-rules.js", "utf8");
const VALIDATE = ["-I", "/app/datum-sandbox-program.js", "-e", "__datum_register(" + RULES + "); print(__datum_run(std.in.readAsString()));"];
// What the rules read beside the values. The page injected a copy of its own; the server has the say.
const PHONE_COUNTRIES = ["CA", "US"];
// What the Country select shows, resolved here: the program never fetches a data source. An answer to a
// choice whose source is missing from the optionSnapshot below is refused (option.sourceUnavailable).
const COUNTRIES = [{ value: "CA", label: "Canada" }, { value: "US", label: "United States" }, { value: "MX", label: "Mexico" }];
// The run log comes back with the outcome — the sandbox has no console — and goes to this
// process's console, each entry at its own level: what the kernel did, step by step, and what
// the rules said, at LOG_LEVEL.
const LOG_LEVEL = "debug"; // "off", "error", "warn" (the kernel's default), "info" or "debug"
const app = express();
app.use(express.json());
app.post("/api/submissions", async (req, res) => {
const { submissionKey } = req.body; // the delivery is { document, submissionKey }: one key per attempt, the same key again is the same submission
if (typeof submissionKey !== "string" || !submissionKey) { // without one a retry could not be told from a new submission
return res.status(400).json({ errors: [{ code: "bad_document" }] });
}
const { document: { submission } } = req.body;
const context = {
formId: FORM.id,
form: FORM.root, // the schema: field constraints and sanitization come from it
values: submission.values,
locale: submission.locale ?? "en",
externalData: { phoneCountries: PHONE_COUNTRIES },
optionSnapshot: { countries: COUNTRIES }, // the lists a choice's data source holds, by source id
journey: journeyOf(submission), // the path, worked out again from the values (see journeyOf)
logLevel: LOG_LEVEL
};
let outcome;
try {
outcome = JSON.parse(await qjs(VALIDATE, JSON.stringify(context), 5, 32 * 1024 * 1024));
} catch (failure) { // a limit cut the run off, or a program or rule module would not load: fail closed, as the browser does
if (!(failure instanceof SandboxError)) throw failure; // anything else is this server's own: a 500
console.error("sandbox failed for " + FORM.id, failure);
return res.status(503).json({ errors: [{ code: "wasm.execution_error" }] });
}
for (const entry of outcome.logs ?? []) {
console[entry.level](entry.message, entry.data ?? "");
}
if (outcome.errors.length > 0) {
// 422 with the error contract, as the browser would have seen it; the page's messages word it.
const errors = outcome.errors.map((e) => ({ field: e.componentId ?? null, code: e.code }));
return res.status(422).json({ errors });
}
store(submissionKey, outcome.values); // the sanitized values: what the form defines, nothing else
res.json({ ok: true });
});
app.listen(8000);
function store(key, values) {
// wherever submissions go: one record per key: the same key with the same values is that record again; the same key with other values is refused (409 submission.keyConflict)
}// qjs.mjs — one run of the engine, in a process of its own: qjs-wasi.wasm, QuickJS-ng's own
// WebAssembly build from its release page, on V8, with node:wasi for its system interface — nothing
// to install beside Node 22. The endpoint (main.mjs) starts one of these per request:
// node --wasm-max-mem-pages=<pages> --disable-warning=ExperimentalWarning qjs.mjs <qjs arguments>
// The request comes in on stdin and the answer goes out on stdout; the directory the process was
// started in is mounted as /app, all the run can reach. The flag is the memory the module may grow
// to, per process. The warning silenced is node:wasi's own, that it is still marked experimental,
// so that stderr is the run's alone.
import { readFileSync } from "node:fs";
import { WASI } from "node:wasi";
const wasi = new WASI({
version: "preview1",
args: ["qjs", "--std", ...process.argv.slice(2)], // --std: the std module, which reads stdin
preopens: { "/app": "." },
returnOnExit: true // qjs's exit code comes back from start() below, and becomes this process's
});
const qjs = await WebAssembly.compile(readFileSync(new URL("qjs-wasi.wasm", import.meta.url)));
const instance = await WebAssembly.instantiate(qjs, wasi.getImportObject());
process.exitCode = wasi.start(instance);What to notice
- One program, one verdict. The endpoint does not port the rules. It hands the program the form's root, the submitted values, the locale and what only the server knows, and reads back the outcome. The same input gives the same codes in the browser and on every host.
- The server's copies decide. The form, the rules, the list of countries behind the select and the countries phone contact is offered in are the server's own. The page sends its copy of the phone countries to its own sandbox, but the server has the say.
- Codes, not text. A 422 carries
{ errors: [{ field, code }] }. The page'smessagesword each code per language, so the server never sends wording. - One record per attempt. The page posts
{ document, submission. The endpoint refuses a delivery without a key with 400Key } bad_, and stores one record per key, so a retry of the same attempt is the same record.document - The sanitised values. On success the endpoint stores
valuesfrom the outcome, not what was posted: keys the form does not define are gone. - Fail closed. A rule that throws blocks the submission with
validator.. A run that ends without an outcome, because its time or memory limit stopped it or a rule module would not load, gives no verdict: the endpoint answers 503failed wasm.and stores nothing.execution_ error - The run log stays on the server. It comes back inside the outcome at the level the request asks for (
debughere), and the endpoint writes it to its own log. It never goes to the browser.
Simple on purpose
These endpoints are short so that the part that is Datum's stays in view: hand the program the form and the answers, read the outcome, answer with its codes. Each is one file you can run in a few minutes. They are not a pattern for a production server.
What they skip, so that you know it is there:
- They answer 422 for any entry in the outcome. A warning alone need not block a submission.
- They take the program's whole output as the answer, and return its error message when a PDF run fails. A production host frames the answer and keeps messages in its log.
- They keep nothing: no store, no record of a submission key already seen, no limit on runs in flight.
The Spring Boot and FastAPI hosts are packages that do all of that for you. For a host of your own, Running the programs on your own server has the full protocol and the conformance suite checks it.
Next
- The same delivery, answered with a PDF: Example: render a PDF on the server.
- What the sandbox checks and how a rule is written: Validation and the sandbox.
- The request, the outcome and the limits a host sets: Running the programs on your own server.
- Hosts that do all of this for you: Server hosts.