LuaSH 2.0 implementation plan¶
Status. This plan is kept as a record. Its work packages WP0 to WP8 landed as pull requests #3 to #14 on 2026-11-08, and
handoff/, where this plan and DESIGN.md lived while the work was open, is gone: the two documents moved todocs/, and the reference code is in the git history underhandoff/reference/.
Companion to DESIGN.md; a reference like §6.3 points to a section
there. The work is cut into packages WP0 to WP8 that own disjoint files, so
WP1 to WP6 can run as parallel sessions. Each package is written as a brief
for one implementer who has read DESIGN.md and this section 1, and
nothing else from the discussion that produced them.
1. Rules for every package¶
1.1 Branches and ownership¶
- Start from
main. Work on a branch namedwp<N>-<slug>(for examplewp2-posix). Commit on it as you go; push nothing; merge nothing. WP8 merges the branches. - Edit only the files your package owns (listed in each brief). When you
need another package's module, code against its interface in
DESIGN.md; when you need it to run, usetests/backend_fake.lua(WP0) or pointLUA_PATHathandoff/reference/sh/as a stand-in, and say in your commit message what you could not verify. - Never edit
handoff/**,tests/gambiarra.lua, or the rootsh.lua(WP5 deletes it, nothing else touches it). - Every
.luafile undersh/is a module:local M = {}at the top,return Mat the bottom, no globals, noarg, no shebang.
1.2 Environment¶
Development machine: macOS, no luarocks, no timeout(1), no /dev/shm,
shellcheck installed. Interpreters:
REG=$HOME/Developer/cli-util-mods/opt/bin # lua-regolith
$REG/lua # Lua 5.5.1 with luaposix 36.3 and luv 1.52.1
$REG/lua-static # Lua 5.5.1, same modules preloaded into the binary
$REG/luac # `luac -p file.lua` is the parse check
lua # /opt/homebrew/bin/lua, Lua 5.5.1, luv only, no luaposix
luajit # /opt/homebrew/bin/luajit, LuaJIT 2.1, luv only
Run everything from the repository root with the package path set
explicitly (;; appends the interpreter's default path):
export LUA_PATH="./?/init.lua;./?.lua;./tests/?.lua;;"
./?/init.lua comes first so that require("sh") resolves to
sh/init.lua while the old root sh.lua still exists on your branch.
./tests/?.lua makes require("backend_fake") work.
To simulate a host without luaposix and luv on the regolith interpreter:
$REG/lua -e 'package.cpath = ""; package.preload.posix = nil; package.preload.luv = nil' tests/test.lua
Any require("posix") or require("luv") then fails, and the library
must fall back to the popen backend.
1.3 Tests¶
- Framework:
tests/gambiarra.lua, used as intests/test.lua:test("name", function() ok(cond, "message") end), a reporter that counts failures,os.exit(1)when any failed. - Every test file sets
package.pathfrom its own location with a copy of theinit()function at the top oftests/test.lua, extended byprepend_path(pth .. "/../?/init.lua"), so that it also runs aslua tests/test_x.luawithoutLUA_PATH. - A test file that cannot run on the current interpreter (backend missing,
feature missing) prints one line
SKIP: <reason>and exits with status 0. The matrix runner counts those separately. - Tests use only programs that exist on macOS and Linux:
sh,cat,echo,printf,seq,wc,grep,head,sort,true,false,ls,pwd,uname,kill. No network, nosleeplonger than one second. - Expected output of the current suite, for reference:
$REG/lua tests/test.luaprints one✔line per check and exits 0.
1.4 Code style: AGENTS.md applied to this repository¶
AGENTS.md at the repository root is the style guide. It was written for
several languages and repositories; this is what applies here.
Applies to every Lua file:
- 4 spaces, about 80 columns, one action per line, local alignment of a small group of related assignments or table fields.
- Wrapped calls: function name and
(together, arguments at +4 from the statement's indentation, several short arguments per line,)alone at the statement's indentation. A single table or call argument hugs the parentheses. - Wrapped function definitions use the 12/8 layout: parameters at +12,
)at +8, body at +4. It applies tofunction M.f(andlocal function f(alike. Most Lua signatures fit on one line; wrap only those that do not. - A short guard such as
if not ok then return nil, err endmay stay on one line; a line with two actions is split. - Apply the style to the code you write and change; do not reformat a file you only pass through.
local function run_with_files(
tmpdir, argv, input, opts,
on_stdout, on_stderr
)
local infile = tmppath(tmpdir, "in")
-- ...
end
local result = backend.run(
argv, input,
{ on_stdout = out_cb, on_stderr = err_cb, cwd = cwd }
)
setmetatable(M, {
__call = function(_, name, ...) return command(name, ...) end,
__index = index_module,
})
local ok, posix = pcall(require, "posix")
Applies in a limited way:
- Bash refinements: only to
tests/run_all.sh(a bash script:set -euo pipefail,localper line, arrays for optional arguments, shellcheck clean) and to the shell strings the popen backend andsys.luabuild forio.popenandos.execute. Those strings are POSIXsh: no[[ ]], no arrays, no&>, every argument single-quoted. The rule that a script starts withset -euis for scripts, not for a one-command string. - YAML: if WP6 adds
.github/workflows/test.yml, it uses 2-space indentation as the YAML rule says.
Does not apply: the Rust and Nushell sections, the Containerfile, RUN,
SHELL and /etc/profile.d rules, and the section "Scope in this
repository", which names vendor/**, slinky-* symlinks, generated/**
and SchedMD headers that belong to another repository. For luash read that
section as:
Never restyle
tests/gambiarra.luaoropt/bin/build/amalg.lua: they are vendored third-party code, kept verbatim so their diff against upstream stays empty.standalone/sh.luais generated bymake standalone; regenerate it, never edit it.handoff/reference/**is frozen reference material. Apply the style to new and changed code without reformatting unrelated sections; convert a whole file only in a dedicated formatting commit.
(That paragraph is proposed as the replacement text for the section; the
repository owner decides whether to put it into AGENTS.md.)
Lua conventions that AGENTS.md does not cover: local for everything;
public functions get a --- doc comment line (one sentence, then @param
and @return lines in Lua Language Server syntax, as sh.lua already
does for command); error messages start with the function or module
name ("sh.popen: ..."); strings use double quotes unless the string
contains one.
1.5 Compatibility checklist (§9)¶
Target: Lua 5.1, 5.2, 5.3, 5.4, 5.5 and LuaJIT 2.1. Before you commit, grep your files for each of these:
table.unpack->local unpack = table.unpack or unpackat the top.goto-> not allowed.os.execute(...)result -> success isr == true or r == 0.h:close()on a popen handle -> may return onlytrue; never rely on the status it returns without the feature test in §6.3.__gcon a table -> usenewproxy(true)whennewproxyexists.- Bit operators
|,&,~,//and integer division -> not allowed (syntax error on 5.1 to 5.2 and LuaJIT);__boras a metamethod key is fine. - Assignment to a
forloop variable -> not allowed (Lua 5.5). #!line orargin a module -> not allowed.require("posix")orrequire("luv")outsidepcall-> not allowed anywhere, includingsys.luaand the backends.string.format("%d", x)with a non-integer floatxraises on 5.3 and newer; format counters with%donly when they are integers.
Parse check on both syntaxes: $REG/luac -p file.lua and
luajit -bl file.lua > /dev/null.
1.6 Definition of done¶
A package is done when all of these hold and the commit message says so:
- Every owned test file passes on every interpreter of section 1.2, or
prints
SKIP:with a reason that section 1.3 allows. - Every owned Lua file passes both parse checks of section 1.5.
grep -n 'require("posix")\|require("luv")' <owned files>shows only lines that containpcall.- The acceptance commands in the brief produce the stated result.
- The style of section 1.4 is applied;
shellcheckis clean for shell code.
2. Dependency graph and schedule¶
wave 1 (parallel): WP0 contract test WP1 command.lua WP6 build tooling
| |
wave 2 (parallel): WP2 posix WP3 luv WP4 popen WP5 init/pipe/sys
\ | / |
wave 3: WP8 stage 1: merge WP0-WP6 into main ---+
|
wave 4: WP7 tests and docs
|
wave 5: WP8 stage 2: standalone, matrix, release
- WP0 is small and the coordinator does it first; WP2, WP3 and WP4 need its contract test from the start.
- WP5 needs the
sh/command.luainterface. It can start as soon as WP1'sM.parseandM.Commandexist on thewp1-commandbranch (copy that one file into your checkout while WP1 finishes; do not commit it). - WP6 needs only the module names in §2.
- WP7 starts from
mainafter stage 1, so it runs against the real modules.
Rough sizes: WP0 1 h, WP1 3 h, WP2 4 h, WP3 3 h, WP4 5 h, WP5 5 h, WP6 3 h, WP7 5 h, WP8 3 h.
3. Work packages¶
WP0: backend contract test and fake backend (coordinator)¶
Owns tests/backend_fake.lua, tests/test_backend.lua.
tests/backend_fake.lua is the smallest implementation of §4 and exists
so that WP5 and the contract test itself can be developed before the real
backends exist:
name = "fake";available()returnspcall(io.popen, "exit 0")truthiness;configure(opts)does nothing.capabilities():stdin = true,separate_stderr = false,streaming = false,signals = false,exitcodefrom the feature test of §6.3 (io.popen("exit 7"):close()reports 7 or not).run(argv, input, opts): checkargv(non-empty array of strings,error()otherwise); writeinputto a file fromos.tmpname()(opened withio.open(path, "wb")); build'<argv[1]>' '<argv[2]>' ... < '<file>' 2>&1, with'in an element written as'\'', prefixed bycd '<cwd>' &&whenopts.cwdis set;io.popen(cmd, "r"),read("*a"),close(); remove the file; callopts.on_stdout(out)once whenout ~= ""; return{ exitcode = code or 0, signal = 0, stdout = out, stderr = "", cause = "exited" }.
tests/test_backend.lua <name> [strategy] loads sh.backend.<name>, or
backend_fake when <name> is fake, calls configure({strategy =
strategy}) when a strategy is given, prints SKIP: and exits 0 when
available() is false, and otherwise checks, with caps =
B.capabilities():
B.name == name;caps.name == name; the five boolean fields exist.run({"echo", "hello"}, ""):stdout == "hello\n",stderr == "",cause == "exited",signal == 0, andexitcode == 0whencaps.exitcode.- When
caps.stdin:run({"cat"}, "abc\n")givesstdout == "abc\n";run({"cat"}, big)withbig = string.rep("0123456789abcdef\n", 20000)(340 KB) givesstdout == big(the pipe-buffer deadlock test; it must return, not hang). run({"sh", "-c", "echo out; echo err >&2"}, ""): whencaps.separate_stderr,stdout == "out\n"andstderr == "err\n"; otherwisestdoutcontains bothoutanderrandstderr == "".- When
caps.exitcode:run({"sh", "-c", "exit 3"}, "")hasexitcode == 3. run({"luash-no-such-program-xyz"}, ""):exitcode == 127whencaps.exitcode; withcaps.separate_stderr,stderr ~= ""andstdout == "", otherwise the shell's message is instdout.- Streaming callbacks: run `{"sh", "-c", "printf 'a\rb\r'; printf 'x\n'
&2"}
andwithon_stdoutandon_stderrappending to buffers; the concatenated buffers equal the returnedstdoutandstderrbyte for byte (for a merged backend the stderr buffer is empty). Whencaps.streaming, theseq` run delivered more than one stdout chunk. - When
caps.signals:run({"sh", "-c", "kill -9 $$"}, "")hascause == "killed",signal == 9andexitcode == 127; otherwisecause == "exited"andexitcode == 137whencaps.exitcode. - Argument fidelity:
run({"printf", "%s\n", "a b", "it's", "", "--", "$HOME", "*", "a;b", "x"}, "")returns"a b\nit's\n\n--\n$HOME\n*\na;b\nx\n". run({"pwd"}, "", {cwd = "/"})returns"/\n".pcall(B.run, {}, "")andpcall(B.run, {"echo", 1}, "")both return false.
Acceptance: lua tests/test_backend.lua fake passes on all four
interpreters; lua tests/test_backend.lua posix prints FAIL: cannot load
sh.backend.posix on every interpreter until WP2 lands. Once a backend
module exists, an interpreter without its C module prints SKIP:.
WP1: sh/command.lua¶
Owns sh/command.lua, tests/test_command.lua. Replaces
handoff/reference/sh/args.lua, whose flatten is the starting point.
Read §3.4, §3.5 (Result shape), §7.
Exports:
M.parse(...)applies §7 to its arguments and returns{ args = {...}, input = "", sources = {...}, fields = {...} }:argsthe positional strings and flags in order;inputthe concatenated__inputstrings;sourcesthe__pipelinevalues of Result arguments in order (skipping nil);fieldsthe pass-through__stdout,__stderr(concatenated like today'sflatten),__exitcodeand__signal(the last seen) for builtins. A Result is recognised by having a string field__stdout.M.quote(s): the rendering rule of §3.4 for one argument.M.Command(name, ...),M.Pipeline(...)with the methods of §3.4;__tostringon both;__boron both;__nameset;__indexto the method tables.Command:argv()andPipeline:commands()return fresh tables.Command:input(x)accepts a string or a Result (then it takes__inputand records the Result's__pipelineas a source).M.set_runner(fn);Command:run()callsfn(self)and returns its value;Pipeline:run()runs the stages in order, passing the previous stage's Result as input to the next one, and returns the last Result.M.render(pipeline): the string behind__tostring, with the grouped form{ a; b; } | cwhen a stage has more than one source.M.is_command(x),M.is_pipeline(x): metatable__namechecks.
tests/test_command.lua (no process is started; set_runner with a stub
that records calls):
parse("a", 1, {"-b", {"c"}})gives{"a", "1", "-b", "c"}.parse({format = "%a", v = true, q = false, n = 3, e = ""})gives{"--format=%a", "-n=3", "-v", "--e="}in exactly that (sorted) order;qis absent; a function value raises an error naming the key.parse({__input = "x"}, {__input = "y"})hasinput == "xy"; a fake Result{__stdout = "o", __stderr = "e", __exitcode = 2, __signal = 0, __input = "o", __pipeline = p}contributesinput == "o",sources == {p},fields.__exitcode == 2.tostring(Command("ls", "-l", {color = "auto"}):arg("a b", ""))isls -l --color=auto 'a b' '';quote("it's")is'it'\''s';quote("a/b.c_d=e")is bare.flag("l")gives-l,flag("color", "auto")gives--color=auto,flag("x", false)adds nothing, calls chain and return the command.tostring(Command("seq", 1, 10):pipe(Command("wc", "-l")))isseq 1 10 | wc -l;Pipeline(a, b):pipe(c):commands()has three entries and#of the original stays 2 aftercommands()is modified.- A stage with two sources renders as
{ seq 1 10; seq 20 25; } | wc -l. __bor: set_G.luash_test_a = caand_G.luash_test_b = cb, then(loadstring or load)("return luash_test_a | luash_test_b"). On Lua 5.3 and newer the chunk compiles and returns a Pipeline whose rendering isa | b; on 5.1, 5.2 and LuaJIT the compile fails (nil and a syntax error) and the test records a pass with a note.Command("x"):run()beforeset_runnerraises a message containingno runner; afterset_runner,run()returns what the stub returns and the stub received the Command;Pipeline:run()called the stub once per stage and passed the previous Result as the next stage's input.
Acceptance: lua tests/test_command.lua passes on all four interpreters
(__bor check passes on 5.5 and notes on LuaJIT).
WP2: sh/backend/posix.lua¶
Owns sh/backend/posix.lua. Read §4, §5, §6.1, handoff/reference/sh/
pipe_posix.lua and the popen3 function in the root sh.lua (the
fork/pipe/dup2 sequence is right; keep it).
luaposix 36 facts you need (all verified on the regolith interpreter):
local ok, posix = pcall(require, "posix")at the top;available()returnsokand, when false, the error text.posix.pipe()returnsr, w;posix.fork();posix.dup2(old, new);posix.close(fd);posix.chdir(dir)returns0ornil, err.posix.execp(path, argt):argtis the argument table without the program name (argt[0]may hold argv[0]; omit it). It returns only on failure, asnil, err.posix.write(fd, s)returns the number of bytes written (may be less than#son a non-blocking fd) ornil, msg, errno; compareerrnowithposix.EPIPEandposix.EAGAIN.posix.fcntl(w, posix.F_SETFL, posix.O_NONBLOCK)on the fresh write end (its flags are 0, so noF_GETFLround trip and no bit operators are needed).posix.poll(fds, timeout)withfds = { [fd] = { events = { IN = true } }, [w] = { events = { OUT = true } } }; after it returns,fds[fd].reventshas booleansIN,OUT,HUP,ERR,NVAL. Usetimeout = -1.posix.read(fd, 4096)returns""ornilat EOF.posix.wait(pid)returnspid, "exited" | "killed" | "stopped", code.posix.signal(posix.SIGPIPE, posix.SIG_IGN)returns the previous handler ("SIG_DFL","SIG_IGN"or a function); restore it with the same call.posix._exit(code)in the child.
Structure: B.run = spawn (fork, child setup, exec or _exit(127)
with a message on fd 2) + pump (the poll loop of §6.1, writing input
in slices of 65536 bytes, reading both outputs into chunk arrays, calling
opts.on_stdout / opts.on_stderr per chunk) + wait (map
exited/killed/stopped to exitcode, signal, cause). Return
stdout = table.concat(chunks). No print, no io.write anywhere in the
module.
Acceptance: $REG/lua tests/test_backend.lua posix and
$REG/lua-static tests/test_backend.lua posix pass every check including
the 340 KB cat and the kill -9; lua tests/test_backend.lua posix
prints SKIP:. Also run this by hand and confirm the process exits 0, not
141:
$REG/lua -e 'local B = require("sh.backend.posix"); local r = B.run({"head", "-c", "1"}, string.rep("x", 1000000), {}); print(r.exitcode, #r.stdout)'
WP3: sh/backend/luv.lua¶
Owns sh/backend/luv.lua. Read §4, §5, §6.2 and
handoff/reference/sh/pipe_luv.lua, which is close to the final shape.
luv facts: local ok, uv = pcall(require, "luv"); uv.new_pipe(false);
uv.spawn(path, { args = rest, stdio = { stdin, stdout, stderr }, cwd =
opts.cwd }, function(code, signal) ... end) returns handle, pid or
nil, err, name (args excludes the program name); pipe:read_start(
function(err, chunk) ... end) where chunk == nil means EOF;
pipe:write(data, function(err) ... end); pipe:shutdown(cb);
pipe:close(); handle:close(); uv.run("once") runs one loop
iteration, uv.run("nowait") drains callbacks without blocking.
Changes from the reference: return the cause = "failed" result on spawn
failure instead of error(); close every pipe and the process handle in
all paths; count three completion events (exit, stdout EOF, stderr EOF)
before leaving the loop; a write error (child closed stdin) closes the
stdin pipe and is not an error; signal > 0 in on_exit gives cause =
"killed"; empty input closes stdin right after spawn; honour opts.cwd.
Document in the module comment that run must not be called from inside
a libuv callback.
Acceptance: lua tests/test_backend.lua luv, luajit
tests/test_backend.lua luv, $REG/lua tests/test_backend.lua luv and
$REG/lua-static tests/test_backend.lua luv all pass every check.
WP4: sh/backend/popen.lua¶
Owns sh/backend/popen.lua, tests/test_popen.lua. Read §4, §5, §6.3,
§9 and handoff/reference/sh/pipe_popen.lua (keep its strategy names
and its shell_escape; replace its temp-file handling).
Structure:
render(argv, files, cwd, mode)builds the shell string;filesholds the absolute pathsinput,stderr,statusor is nil for the merged strategies;modeis"sh"or"cmd"(Windows, §6.3, one function, marked best effort). Expose it asB._renderfor the tests.tmpdir()creates the private directory lazily (§6.3:mkdir -m 700, random suffix, retry on failure,os.executesuccess testr == true or r == 0), remembers its path, and registers the__gcsentinel (newproxy(true)whennewproxyexists, a table otherwise).strategies = { shmem = ..., standard = ..., basic = ..., minimal = ... }, each withavailable()andrun(argv, input, opts);shmemandstandardshare one implementation parameterised by the temp root.B.configure({strategy = ..., tmpdir = ...});B.capabilities()reflects the resolved strategy;B.available_strategies();B.cleanup_all().- Exit status: file strategies read it from the status file; merged
strategies use
h:close()only when the load-time feature test says it reports a status (§6.3). - Reading:
h:read(4096)loop,on_stdoutper chunk; stderr file read after close and passed toon_stderronce. - Every temp file is removed in a
pcall-protected cleanup that runs whether the command succeeded or not.
tests/test_popen.lua:
- For each strategy in
B.available_strategies(), run the contract checks by loadingtests/test_backend.lualogic (factor its checks into a function it exports when called with no arguments, or simply spawnlua tests/test_backend.lua popen <strategy>throughio.popenand check its exit status). B._render({"a b", "it's"}, {input = "/t/in-1", stderr = "/t/err-1", status = "/t/st-1"}, "/w", "sh")equalscd '/w' && 'a b' 'it'\''s' < '/t/in-1' 2> '/t/err-1'; echo "$?" > '/t/st-1'and the merged forms end in2>&1with</dev/nullfor minimal; whenshellcheckis onPATH, write each rendered string to a file and runshellcheck -s shon it (skip with a note otherwise).- After a
standardrun the private directory exists,ls -ldshowsdrwx------, and it contains noin-*,err-*orst-*files;B.cleanup_all()removes it; the next run recreates it. - Two runs with the same input in a row do not share files (counter
advances); the status file gives
exitcode == 3forsh -c 'exit 3'on LuaJIT, whereclose()reports nothing. shmemis reported unavailable on macOS and the test says so.
Acceptance: tests/test_popen.lua and tests/test_backend.lua popen
<strategy> pass for standard, basic and minimal on all four
interpreters (minimal skips the stdin checks through its capability).
WP5: sh/init.lua, sh/pipe.lua, sh/sys.lua, sh/stack.lua¶
Owns those four files; deletes the root sh.lua and the
experiments/sh.lua symlink (git rm). Read §3 (all of it), §6.4, §6.5,
§7, the root sh.lua (the behaviour to keep: builtins, install(),
Result metatable, tostring/concat) and handoff/reference/sh/init.lua,
pipe.lua, stack.lua.
sh/stack.lua: the reference file, restyled.
sh/sys.lua (§6.5): platform(), chdir(dir), getcwd(),
emulated_cwd() (nil when a real chdir is available), each require
behind a cached pcall.
sh/pipe.lua (§6.4): the ORDER table, lazy pcall(require,
"sh.backend." .. name), resolve() honouring config.backend,
LUASH_BACKEND and LUASH_POPEN_STRATEGY, set_backend(name_or_table,
opts), backend_name(), available_backends(), capabilities(),
run(argv, input, pipeline): builds the callbacks from config
(sink or io.stdout/io.stderr with flush() after each write),
writes the trace line, passes sys.emulated_cwd() as opts.cwd, raises
on non-empty input without the stdin capability. It takes the config
table at set_config(tbl) from init.lua so that it never requires
sh.init.
sh/init.lua: M.version; M.config (§3.2); module metatable with
__call, __index (aliases __verbose/__raise_errors, then builtins
from M.FUNCTIONS for names in M.__index_ignore_function, then
command(name)) and __newindex (aliases, else rawset);
make_result(fields, pipeline) with the metatable of §3.5; command(name,
preargs...) returning the closure that calls command_mod.parse, builds
the Command (preargs first), appends it to the Pipeline of the sources
(a new Pipeline when there are none; a grouped stage when there are
several), calls pipe.run, applies §3.6, returns the Result; the runner
installed with command_mod.set_runner is the same function, so
Command:run() and sh.name(...) share one path; builtins cd,
pushd, popd, stdout, stderr, print with today's messages and
pass-through, pushd using sys.getcwd(); install() unchanged in
behaviour; exports listed in §3.3 and M.Command, M.Pipeline.
Acceptance, with LUA_PATH as in section 1.2 and sh/command.lua from
the wp1-command branch copied in:
# structure and behaviour with the fake backend, all four interpreters
lua -e 'local sh = require("sh"); sh.set_backend(require("backend_fake"))' tests/test.lua
passes every test except "Check Version" (WP7 updates the string to
2.0.0). Then, by hand on $REG/lua:
require("sh").backend_name()printsposixwhensh/backend/posix.luafromwp2-posixis present,popenunder the no-C-modules command of section 1.2 whensh/backend/popen.luafromwp4-popenis present, and a clear error naming the missing module when neither exists (never a nil index error).sh.__verbose = true; sh.seq(1, 3)streams1\n2\n3\nto stdout unchanged; withsh.config.sink = function(s, c) io.write("[", s, "]", c) endit prints[stdout]1\n2\n3\n.sh.config.trace = true; sh.seq(1, 3):wc("-l")prints+ seq 1 3then+ seq 1 3 | wc -lon stderr.sh.config.raise_errors = true; sh.ls("/missing")raises thelsmessage;sh.sh("-c", "exit 3")raisessh -c 'exit 3': exit status 3.tostring(sh.seq(1, 10):wc("-l").__pipeline)isseq 1 10 | wc -l.sh.cd("/"); print(sh.pwd())prints/on every backend; the same withpackage.cpath = ""exercises the emulation (only oncewp4-popenis present).
WP6: build tooling, rockspec, CI, test matrix¶
Owns opt/bin/build.lua, opt/bin/build/amalg.lua, Makefile,
rockspec/sh-scm-3.rockspec (replaces sh-scm-2.rockspec),
rockspec/sh-2.0-1.rockspec, tests/run_all.sh,
.github/workflows/test.yml, .travis.yml (delete), .luarc.json.
Read §2, §8, §9 and ../simple-templates/opt/bin/build.lua,
../simple-templates/Makefile.
opt/bin/build/amalg.lua: copy verbatim from../simple-templates/opt/bin/build/amalg.lua. Never restyle it.opt/bin/build.lua: a Lua script with a#!/usr/bin/env lualine and no dependencies beyond the standard library. It finds the repository root fromarg[0](debug.getinfo(1).sourceas a fallback), setspackage.path = root .. "/?.lua;" .. root .. "/?/init.lua", loadsopt/bin/build/amalg.luawithloadfilein an environment that inherits_G(as the simple-templates script does), and calls it asamalg("-o", root .. "/standalone/sh.lua", "-s", root .. "/sh/init.lua", "sh.command", "sh.pipe", "sh.sys", "sh.stack", "sh.backend.posix", "sh.backend.luv", "sh.backend.popen"). The module list is derived by scanningsh/(subdirectorybackendgivessh.backend.<file>) so a new module cannot be forgotten;init.luais excluded from the list because it is the main script. Runamalgfrom the repository root (-sis read relative to the current directory; use absolute paths to be safe).Makefile: targetsstandalone(default, buildsstandalone/sh.lua),test(runstests/run_all.sh),clean. No.exexecutable, noawkshebang step: the output is a library.- Rockspecs:
type = "builtin"withmodulesmappingshtosh/init.luaand eachsh.*module to its file;install.binkeepssh.autogen; dependencies"lua >= 5.1"only, with a comment that luaposix and luv are optional and detected at run time. Check the file loads:lua rockspec/sh-2.0-1.rockspec. tests/run_all.sh: bash,set -euo pipefail, shellcheck clean. For each interpreter found (the four of section 1.2; skip missing ones with a note), for eachLUASH_BACKENDinposix luv popen, runtests/test.lua; forpopenalso eachLUASH_POPEN_STRATEGYinstandard basic minimal; thentests/test_command.lua,tests/test_backend.lua <backend>andtests/test_popen.lua; thentests/test.luawithLUASH_STANDALONE=1againststandalone/sh.luaforposixandpopen; then the no-C-modules command of section 1.2 against both layouts. Count pass, fail and skip per line; print a summary table; exit 1 on any fail. Runs that printSKIP:count as skipped, not failed..github/workflows/test.yml(optional but recommended, 2-space YAML): matrix overlua 5.1, 5.2, 5.3, 5.4, luajitwithleafo/gh-actions-luaandleafo/gh-actions-luarocks, installingluaposixandluvthrough luarocks, runningtests/run_all.shwith the interpreter list overridden by an environment variable (LUASH_INTERPRETERS="lua"). Delete.travis.yml..luarc.json: add"sh/?.lua"-style workspace settings only if the Lua Language Server needs them; otherwise leave it.
Until sh/ exists, test build.lua in a scratch copy of the repository
where sh/ is populated with stubs: init.lua requiring the seven
modules and returning a table, each module returning {}. The built file
must luac -p clean, dofile to a table, and show every require(
"posix")/require("luv") line only inside pcall (there are none in
the stubs; the real check is at WP8).
Acceptance: make standalone in the scratch copy produces
standalone/sh.lua; lua -e 'package.path = "./standalone/?.lua"; print(
require("sh")) prints table: ...; tests/run_all.sh runs against the
current repository with the old tests/test.lua and reports the matrix
(most cells SKIP or FAIL at this stage; the runner itself must not
crash); shellcheck tests/run_all.sh is clean.
WP7: test suite and documentation (after stage 1)¶
Owns tests/test.lua, README.md, example.lua. Starts from main
after WP8 stage 1 so that every module is real.
tests/test.lua:
init()also prependspth .. "/../?/init.lua", and whenLUASH_STANDALONEis set prependspth .. "/../standalone/?.lua"last (so it wins) and skips../?/init.lua.- After
require("sh"): whenLUASH_BACKENDnames a backend that is not insh.available_backends(), printSKIP: backend <name> unavailableandos.exit(0). Printsh.backend_name()and the capabilities table once at the top. - "Check Version" compares with
... LuaSH 2.0.0. - Keep every existing test. Tests that need stdin or separate stderr
check
sh.capabilities()and record a skip note instead of failing underminimal/basic. - New tests:
sh.Command/sh.Pipelinerendering through the public module;__pipelineon a chained Result; table flags sorted andfalseomitted (sh.printf("%s\n", {b = false, a = true})prints-a); verbose byte exactness throughsh.config.sinkforseq 1 1500and forsh -c "printf 'a\rb\r'; printf 'x\n' >&2"(the sink receives exactly the stdout and stderr bytes, on the right stream whenseparate_stderr); trace line format; missing command gives__exitcode == 127and non-empty__stderrwithraise_errorsoff, and raises with it on; 340 KB throughcat;head -c 1on 1 MB input returns (SIGPIPE); error message formats of §3.6;sh.cd,sh.pushd,sh.popdround trip withsh.pwd();sh.__verbosealias reads backsh.config.verbose.
README.md: rewrite the installation section (luarocks from the
rockspec, or copy standalone/sh.lua; luaposix and luv optional), add
sections for backends and sh.config, sh.Command/sh.Pipeline, verbose
and trace, errors (the 127 convention replaces the traceback example),
cd emulation on the popen backend, and "Upgrading from 1.x" (what
changed: flags sorted, false omitted, sh.PIPES gone, sh.__verbose
and sh.__raise_errors still work, standalone file). Keep the existing
examples where they still hold. example.lua: add a Command/Pipeline
example.
Acceptance: tests/run_all.sh is green (every cell pass or skip) on the
development machine; README.md examples were each run once.
WP8: integration (coordinator)¶
Stage 1, after WP0 to WP6: merge wp0, wp1, wp2, wp3, wp4, wp5,
wp6 into main (disjoint files; resolve nothing by hand except
tests/ additions), run tests/test_backend.lua for every backend on
every interpreter, run tests/test.lua per backend with the old version
string expected to fail, build standalone/sh.lua with make standalone,
run the gating grep of §8 on it, run the no-C-modules command against the
standalone file. Fix integration mismatches in small commits on main
(or send them back to the package's session). Then start WP7.
Stage 2, after WP7: make standalone again, commit the generated file,
tests/run_all.sh green, smoke-test simple-modules: in a scratch copy
of ../simple-modules, replace opt/lib/sh.lua with standalone/sh.lua
and run opt/bin/module.lua on one example with sh.__verbose = true
in effect; curl-style progress output must render on one updating line.
Update rockspec version, tag, and remove handoff/ or move it to
docs/design/ as the owner prefers.
4. Risks and open points¶
- Windows paths in the popen backend and
sys.luaare untested; they are isolated in one function each so a Windows user can fix them. - Android ordering (luv before posix) is an assumption, one table line.
- The posix poll loop is new code on a path that today works for small inputs; the 340 KB and SIGPIPE checks in the contract test are the guard.
luajitwithoutLUAJIT_ENABLE_LUA52COMPATcannot report exit codes for popen merged strategies; the capability table says so and the tests branch on it.simple-modulesis the one known downstream user; its usage list in §1 ofDESIGN.mdis what must keep working.