Skip to the content
W10SoftwareEngines from nothing

Spark, a JavaScript engine held against Ecma’s tests

Spark is a JavaScript engine I wrote from an empty file, in plain JavaScript and with­out the browser’s own engine. It has its own parser, its own com­piler, a ma­chine that runs its own in­struc­tions, and its own built-ins. Ecma’s con­for­mance suite for the lan­guage, , has test files. Of the that Spark runs, pass. Below, a fixed sample of the suite runs in Spark in your browser, and any test in it can be opened, changed and run again.

PostedKindSoftwareSeriesEngines from nothing

The sample loads when this part is on screen.

Passed·
Failed·
Feature left out·
Built-in not written·

·

The page draws one square for each test, sorted by path. Run the sample starts Spark in a Web Worker and runs every test it at­tempts, in each mode the test asks for. Gray squares use a fea­ture Spark leaves out, and striped squares test a built-in Spark does not have, so the button skips both, as the full run does. Open any square to read the test, change it and run it again in a second worker.

What a test is

A test is one file of JavaScript with a com­ment at the top writ­ten in YAML, its front matter. The front matter names the sec­tion of the spec­i­fi­ca­tion the test is about, the helper files it in­cludes, its flags and the lan­guage fea­tures it uses. The helper files assert.js and sta.js give every test assert and Test262Error, and a test fails by throw­ing.

The front matter of a test that must throw has a negative entry with the phase and the error’s con­struc­tor. The phase is parse, res­o­lu­tion or run­time, so a test that ex­pects a SyntaxError at parse fails if the same error comes later. Unless its flags name one mode, a test runs twice. It runs once as it is writ­ten, and once with "use strict" in front, be­cause the two modes give some code a dif­fer­ent mean­ing.

How Spark runs one

Each run gets a new Spark with a new global object and a new set of built-ins, so no ear­lier test has touched them. The helper files are com­piled once and run again for each new global object. The test is given $262, the host object the suite’s INTERPRETING.md de­scribes, with evalScript, detachArrayBuffer and global.

A neg­a­tive test passes only when Spark throws the named error in the named phase. For a parse test the error must come from Spark’s parser, and for a res­o­lu­tion test from its module loader. An async test passes when it calls $DONE with­out an error before Spark’s job queue is empty. Every run has a limit on steps and on time, so a test that loops with­out end fails in­stead of hang­ing the run.

The count

The com­mand walks test/language and test/built-ins of a check­out of tc39/test262 at one commit, runs every test with three Node work­ers and counts each test once. A test passes when every mode passes.

$ node web/notes/spark/suite.mjs test262
test262 at 7ab7fafa0003: 47,547 test files
attempted 41,428: all of test/language and 53 directories of test/built-ins
  language   passed 22,763, failed  0, not run   963
  built-ins  passed 15,876, failed  2, not run 1,824
passed 38,639 of 38,641 run (74,319 runs in sloppy and strict mode), failed 2
not run for a feature Spark leaves out: 2,787, 35 features
    415  resizable-arraybuffer
    252  import-defer
    243  source-phase-imports
    230  regexp-modifiers
    201  explicit-resource-management
    187  regexp-v-flag
not attempted, built-ins Spark does not have: 6,119
  4,605  Temporal
    654  Iterator
    389  Atomics
    104  AsyncDisposableStack
failed:
  built-ins/TypedArrayConstructors/internals/DefineOwnProperty/BigInt/detached-buffer-throws-realm.js: TypeError: Spark runs one realm per test
  built-ins/TypedArrayConstructors/internals/DefineOwnProperty/detached-buffer-throws-realm.js: TypeError: Spark runs one realm per test
in 780 s with 3 workers

The number Spark passes is counted against the tests it runs, and the output names the two groups that are left out of it. A test whose front matter lists a fea­ture Spark does not have is not run. Most of those fea­tures came into the lan­guage after , the edi­tion Spark was writ­ten against, and the rest are pro­pos­als that are not yet part of the lan­guage. The di­rec­to­ries of test/built-ins for built-ins Spark does not have are not at­tempted, and Temporal is most of them. Counted against every test file in the suite, Spark passes of .

The two that fail

Both fail­ures are in test/built-ins/TypedArrayConstructors, and both call $262.createRealm() to make a typed array with a second global object. Spark gives each test one global object, and its createRealm throws a TypeError with the mes­sage Spark runs one realm per test. A test that needs a second global object is meant to list the fea­ture cross-realm in its front matter, and the run leaves those tests out. These two do not list it, so they run and fail. Both are in the sample above, in red.

The sample in the page

The whole suite is too large to send to a browser, so the page runs a fixed sample. The com­mand draws it with a seed from every test file, the ones Spark leaves out in­cluded, and adds every test that failed in the full run. Each test comes with the helper files it in­cludes and the files it loads as mod­ules. The com­mand then runs the sample in Node the way the page’s worker runs it, and holds each result against the full run.

$ node web/notes/spark/sample.mjs test262 rows.jsonl
sample of test262 at 7ab7fafa0003: 1,002 tests, 1,000 drawn with seed 262 from 47,547 test files and 2 added that failed in the full run
17 harness files, 8 files loaded as modules, 1.95 MB
run 804 (1,548 runs in sloppy and strict mode): passed 802, failed 2, in 34.5 s
not run for a feature Spark leaves out: 64
not attempted, built-ins Spark does not have: 134
results that differ from the full run: 0

In the page, one worker runs the sample a test at a time. A second worker runs the test you open, so a single test does not wait for the sample to finish. A gray or striped square is not run by the button, and Run it anyway runs it in Spark all the same.

The limits of the count

A pass means Spark gives the result the spec­i­fi­ca­tion gives, in the cases the tests write down. The count does not mea­sure speed. Spark is an in­ter­preter writ­ten in JavaScript, and the time at the end of the com­mand’s output is for three work­ers on one ma­chine. The run covers test/language and test/built-ins. It leaves out test/annexB, the legacy ad­di­tions web browsers keep, test/intl402, the tests for Intl, which Spark does not have, test/staging and test/harness.

The source is in the port­fo­lio’s repos­i­tory. Spark is under prototypes/source/p39/, with test262-harness.mjs for run­ning one test. Under web/notes/spark/ are suite.mjs and sample.mjs for the com­mands, run.js for one test with its module files, and worker.js for the page’s worker. The tests are from tc39/test262, under the BSD li­cense.