puck/zilch
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: More samples.

Browse source 
Change-Id: Ibcab3b04ae1650cd213a5a0915b7c6056a6a6964
This commit is contained in:
puck 2025-11-24 18:37:10 +00:00
parent 8385c3ac9b
commit 3ee4d894f5
14 changed files with 552 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

1
docs/highlighter.js
View file

@ -67,6 +67,7 @@ const highlighter = createHighlighterCoreSync({
require("@shikijs/langs/shellsession").default,
require("@shikijs/langs/bash").default,
require("@shikijs/langs/nix").default,
require("@shikijs/langs/json").default,
],
engine: createJavaScriptRegexEngine(),
});

5
docs/modules/ROOT/nav.adoc
View file

@ -13,6 +13,11 @@
*** xref:ninja/usage.adoc[]
*** xref:ninja/man.adoc[Man page]
* Samples
** xref:samples/1-go.adoc[1: Go]
** xref:samples/2-rust.adoc[2: Rust]
** xref:samples/3-cpp.adoc[3: C++]
* Core concepts
** xref:core/derivations.adoc[]
** xref:core/zexp.adoc[]

79
docs/modules/ROOT/pages/samples/1-go.adoc Normal file
View file

@ -0,0 +1,79 @@
= 1: Go
:page-pagination: next
This is a minimal sample for Zilch's Go support, intended to show off the
ability to edit dependencies with ease, without losing the reproducibility that
Zilch provides.
To start, run `zilch-cli-go -m examples/go` and see the resulting binary
operate:
[,console]
----
$ zilch-cli-go -m examples/go
example.com/go /nix/store/wh1iaiplnkznh8x4rgqxs89qn90xaf3v-example.com_go
$ /nix/store/wh1iaiplnkznh8x4rgqxs89qn90xaf3v-example.com_go
1980-01-01T13:37:00.000Z INFO main/main.go:9 This is an example Go program. {"exampleString": "yes", "example": true}
----
---
If, say, you find a bug (e.g. you are of the opinion the line number should come
before the file name), this has to be patched in the logging module, which is
provided by zap. Clone the repo (`https://github.com/uber-go/zap`). In our case
we have it at `./zap`. To make sure the bug isn't patched in this current
version, let's first try building the example program again with it. This can be
done with the `--replace` command, or `-r` for short. This takes a directory
with a `go.mod`, and prioritizes it over the version of the Go module used in
the `go.mod` of the main module.
[,console]
----
$ zilch-cli-go -m examples/go -r zap
example.com/go /nix/store/75r8574ccq8vhsjyrr8jm9icdy5f19c1-example.com_go
$ /nix/store/75r8574ccq8vhsjyrr8jm9icdy5f19c1-example.com_go
1980-01-01T13:37:00.000Z INFO main/main.go:9 This is an example Go program. {"exampleString": "yes", "example": true}
----
Sadly, after updating, nothing changed. So we'll have to make changes. The
filename and line number are written in `zap/zapcore/entry.go`, around line 91;
inside `FullPath()`. After patching the function to swap them around, rerunning
the `zilch-cli-go` command shown previously will go through and recompile only
the changed files. So:
[,console]
----
$ zilch-cli-go -m examples/go -r zap
example.com/go /nix/store/vl6dd9d4ry7xhh31vpimjsn9khpqmxgb-example.com_go
$ /nix/store/vl6dd9d4ry7xhh31vpimjsn9khpqmxgb-example.com_go
1980-01-01T13:37:00.000Z INFO 9:main/main.go This is an example Go program. {"exampleString": "yes", "example": true}
----
As you can see, the bug is fixed, by testing with a project in a different repo!
---
Whenever you run `zilch-cli-go`, the script runs from scratch, not using any
previous information. It performs a series of steps to build a build graph:
First, it walks the `go.mod` of the main module, as well as all its
dependencies, using the `go.sum` as a guide to ensure this all stays content-
addressed, similar to the `go` tool. Once it has all the modules that are used
in the program, it looks for all packages inside each of the modules. Each
package is then analyzed for which other packages it imports, which is used to
build Zilch-native build information.
At this point, enough information has been gathered to build the resulting
build graph. The main package's derivation is materialized into the Nix store,
and with it, all its dependencies are pulled in. All its dependencies are built
with two outputs; both the native code and the API are separately stored. This
ensures that any changes that don't affect the interface that other packages
depend on do not cause a butterfly effect, and stay localized to the native
code.
At no point does Zilch run any of this code on the local system, either. All
builds and information gathering are done entirely inside the Nix daemon, which
provides for a stable and sandboxed environment.

67
docs/modules/ROOT/pages/samples/2-rust.adoc Normal file
View file

@ -0,0 +1,67 @@
= 2: Rust
:page-pagination:
This sample assumes you've tried the Go example, as the Rust implementation of
these concepts work identically. This example dives into the details of the
overrides, which are necessary to run Rust builds in a minimal sandbox.
Most Rust projects depend on external dependencies, written in e.g. C, and
cannot be compiled with purely `rustc` inside a sandbox. To allow for this,
Zilch's Rust support has been designed with this fact in mind.
If we try to compile the example in this directory as-is, it fails, as the
crate depends on `pyo3`, which depends on a working Python interpreter:
[,console]
----
$ zilch-cli-rust -m examples/rust
…
Error: (error Error 0 builder for '/nix/store/n5l4w8m3fj3m1s06p75r0pqj8s9l8cmf-build.rs-run.drv' failed with exit code 101;
last 17 log lines:
> error: no Python 3.x interpreter found
…
> Package: pyo3-build-config
> note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
----
To fix this, we'll need to write an override file. This file describes to Zilch
how to override the Rust environment. There's three override points, as described
in xref:rust/usage.adoc[the more direct usage file]. We'll only use `buildScript`
here, as `pyo3` is simple. Looking at the code, it seems that it expects a `python3`
on `PATH`. To do this, we just need to write a fragment for that. Zilch knows how
to append to these variables, so all we need is a file that contains the following:
.zilch-override.json
[,json]
----
{
"pyo3-build-config": {
"buildScript": {
"PATH": "${pkgs.python3}/bin"
}
}
}
----
Now, when the `build.rs` for `pyo3-build-config` is run, it will have `python3`
available to it. The output that this build script generates, which is stored
in the Nix store, also has references to the parts of `python3` that the build
script used, ensuring that it is properly passed through to the crates that, in
turn, depend on this build script; and the final binary has references to not
just libc, but also python3:
[,console]
----
$ zilch-cli-rust -m examples/rust -z override.json
zilch-pyo3-example zilch_pyo3_example bin /nix/store/vqpj68dgswf8v64sq45banq6vn0fshnz-rustc-bin-zilch_pyo3_example-link
$ nix-store -qR /nix/store/vqpj68dgswf8v64sq45banq6vn0fshnz-rustc-bin-zilch_pyo3_example-link | grep python
/nix/store/cfapjd2rvqrpry4grb0kljnp8bvnvfxz-python3-3.13.8
# hooray!
----
Zilch ships with a series of overrides, primarily around what I (the developer)
have come across. Some of these also use a helper that Zilch provides, which
sets up `pkg-config` based packages recursively, the same way Nixpkgs does.

135
docs/modules/ROOT/pages/samples/3-cpp.adoc Normal file
View file

@ -0,0 +1,135 @@
= 3: C++
:page-pagination: prev
Following up from Rust, Zilch has support for using the build graph from a Ninja
file. However, unlike Rust and Go, there's no one "blessed" way to build C/C++
code, so this isn't as plug-and-play as the other two languages.
To start, the support in Zilch rests upon Nixpkgs' existing derivations. If you
don't have an existing Nixpkgs derivation for the program, you'll need one. As
an example, we'll use `libinput` here.
[,scheme]
----
(environment: "pkgs.libinput"
depfile-path: "libinput-deps.scm")
----
In this sample, `pkgs.libinput` is a Nix expression run inside `nixpkgs`, similar
to how `nix-shell -p` behaves. When you run `zilch-cli-ninja -f libinput.scm build`,
Zilch splits up the derivation into three steps, split up by the `stdenv` build phases:
[,subs="verbatim,macros"]
----
unpackPhase
patchPhase
configurePhase
----8<---- cut derivation up here...
pass:[<del>buildPhase</del>] replace this with Zilch magic...
---->8---- ...and cut here!
checkPhase
installPhase
fixupPhase
installCheckPhase
distPhase
----
The original derivation is run up to the point where it would actually build
code, then the information is extracted from it, and Zilch does its magic: it
finds the Ninja file, and evaluates it just as Ninja itself would do; just with
each output being generated in its own little sandboxed derivation. Header files,
as well as source files (e.g. `.c`, and `.cpp`) are only visible to a build step
when it needs it, as each of these files causes spurious rebuilds if they aren't
actually necessary.
Once the build is complete, Zilch runs the last few phases, and stitches the
resulting derivation back together; making for a store path that is identical
to what `nixpkgs` normally generates!
The `depfile-path`, as passed to Zilch, is the one exception to the "stateless"
behavior: When compiling a source file, a side effect is figuring out all the
header files that get included. To save compute time (and because having them
around shouldn't materially affect builds), Zilch will build files with all
headers the first time, and use that to figure out which headers to include
next time. This information is stored in the `depfile`, which has a big map
of files to their dependencies. When a file fails to build (because it includes
a new header that wasn't previously visible), it will get rebuilt with all
headers again, to regenerate this database. This way, the `depfile` always
contains the minimal amount of headers necessary to build each file.
---
In some cases, projects may not compile inside the Nix sandbox as-is, due to
various reasons, e.g. symlinks, or missing dependencies. In this case, it's
possible to add targeted patches. As an example, the expression I use to build
Lix inside the sandbox is as follows:
[,scheme]
----
(environment: "(pkgs.lixPackageSets.lix_2_93.override { enableDocumentation = false; }).overrideAttrs (a: { doCheck = false; doInstallCheck = false; separateDebugInfo = false; __structuredAttrs = false; })"
depfile-path: "lix-deps.scm"
; The `config.h` file needs to be available to _all_ builds this file is
; inserted via -include on the command line, which doesn't make it into the
; dependency files, so dependency tracking thinks it's unused.
disallow-elide: (lambda (path) (string=? path "config.h"))
patch:
(begin
(import (zilch lang ninja) (srfi 152))
(lambda (target)
(if
; If our first implicit dependency is "python"...
(and
(not (null? (build-edge-implicit-dependencies-target target)))
(string-contains (car (build-edge-implicit-dependencies target)) "python"))
; Materialize (copy the actual files, rather than symlink them) the code-generation logic,
; python resolves imports relative to the target of the symlink, which is somewhere in the
; Nix store, as opposed to the source build tree we build up.
(string-append
"rm -rf src/lix/code-generation; "
"cp -rf -L --no-preserve=ownership \"$_ZILCH_ROOT\"/src/lix/code-generation src/lix/code-generation; "
"chmod ugo+rw -R src/lix/code-generation")
#f))))
----
This is a bit more of a complicated configuration, but shows the
configurability that's (sadly necessarily) possible for Zilch.
---
Editing the source of a project in a way that is compatible with Zilch is a bit
more complex than Go and Rust. To simplify this, `zilch-cli-ninja` has two
subcommands to handle this complexity: `zilch-cli-ninja source [DIR]` and
`zilch-cli-ninja diff`. The former will extract the source code of the project
just before build, and the latter will show the current diff between the
original derivation's source and the exposed one.
Once you've made changes to the project, you can incrementally rebuild it using
`--source` (`-s`), or by adding `override-source: "new/path"` to the
configuration file. The latter approach is mostly convenient for the next feature
we'll cover: nested patches!
---
The `--rewrite` feature on `zilch-cli-go` and `zilch-cli-rust` has an equivalent
in its Ninja support. However, as with the previous explanations, it's not _quite_
as simple. It needs to be configured inside the configuration file:
[,scheme]
----
(environment: "pkgs.labwc"
depfile-path: "labwc-deps.scm"
rewrite:
("libinput-1.29.1"
environment: "pkgs.libinput"
override-source: "./libinput-src"
depfile-path: "libinput-deps.scm"))
----
As you can see, the original libinput configuration is now nested _inside_ labwc's,
and using `-p libinput-1.29.1` lets you operate on it. But now, running just
`zilch-cli-ninja build` builds `labwc`, with the patched `libinput`! And it's
possible to arbitrarily nest these, of course; and any changes should
incrementally percolate down the full chain.

7
examples/go/go.mod Normal file
View file

@ -0,0 +1,7 @@
module example.com/go
go 1.25.2
require go.uber.org/zap v1.27.1
require go.uber.org/multierr v1.11.0 // indirect

16
examples/go/go.sum Normal file
View file

@ -0,0 +1,16 @@
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/testify v1.8.1 h1:w7B6lhMri9wdJUVmEZPGGhZzrYTPvgJArz7wNPgYKsk=
github.com/stretchr/testify v1.8.1/go.mod h1:w2LPCIKwWwSfY2zedu0+kehJoqGctiVI29o6fzry7u4=
go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
go.uber.org/multierr v1.10.0 h1:S0h4aNzvfcFsC3dRF1jLoaov7oRaKqRGC/pUEJ2yvPQ=
go.uber.org/multierr v1.10.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
go.uber.org/multierr v1.11.0/go.mod h1:20+QtiLqy0Nd6FdQB9TLXag12DsQkrbs3htMFfDN80Y=
go.uber.org/zap v1.27.1 h1:08RqriUEv8+ArZRYSTXy1LeBScaMpVSTBhCeaZYfMYc=
go.uber.org/zap v1.27.1/go.mod h1:GB2qFLM7cTU87MWRP2mPIjqfIDnGu+VIO4V/SdhGo2E=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=

10
examples/go/main.go Normal file
View file

@ -0,0 +1,10 @@
package main
import "go.uber.org/zap"
func main() {
logger, _ := zap.NewDevelopment()
defer logger.Sync()
logger.Info("This is an example Go program.", zap.String("exampleString", "yes"), zap.Bool("example", true))
}

7
examples/ninja/labwc-with-libinput.scm Normal file
View file

@ -0,0 +1,7 @@
(environment: "pkgs.labwc"
depfile-path: "labwc-deps.scm"
rewrite:
("libinput-1.29.1"
environment: "pkgs.libinput"
; override-source: "libinput-src"
depfile-path: "libinput-deps.scm"))

28
examples/ninja/lix-eval-jobs-lix-pegtl.scm Normal file
View file

@ -0,0 +1,28 @@
(environment: "pkgs.lixPackageSets.lix_2_93.nix-eval-jobs"
depfile-path: "lix-eval-jobs-dep.scm"
rewrite:
("lix-2.93.3"
environment: "(pkgs.lixPackageSets.lix_2_93.lix.override { enableDocumentation = false; }).overrideAttrs (a: { doCheck = false; doInstallCheck = false; separateDebugInfo = false; __structuredAttrs = false; })"
disallow-elide: (lambda (path) (string=? path "config.h"))
patch:
(begin
(import
(zilch lang ninja)
(srfi 152))
(lambda (target)
(cond
; Fix python codegen code
((and
(not (null? (build-edge-implicit-dependencies target)))
(string-contains (car (build-edge-implicit-dependencies target)) "python"))
(string-append
"rm -rf src/lix/code-generation; "
"cp -rf -L --no-preserve=ownership \"$_ZILCH_ROOT\"/src/lix/code-generation src/lix/code-generation; "
"chmod ugo+rw -R src/lix/code-generation"))
(else #f))))
depfile-path: "lix-dep.scm"
rewrite:
("pegtl-3.2.8"
environment: "pkgs.pegtl"
depfile-path: "pegtl-dep.scm")))
;override-source: "pegtl-src")))

1
examples/rust/.gitignore vendored Normal file
View file

@ -0,0 +1 @@
/target

172
examples/rust/Cargo.lock generated Normal file
View file

@ -0,0 +1,172 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "autocfg"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c08606f8c3cbf4ce6ec8e28fb0014a2c086708fe954eaa885384a6165172e7e8"
[[package]]
name = "heck"
version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
[[package]]
name = "indoc"
version = "2.0.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "79cf5c93f93228cf8efb3ba362535fb11199ac548a09ce117c9b1adc3030d706"
dependencies = [
"rustversion",
]
[[package]]
name = "libc"
version = "0.2.177"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2874a2af47a2325c2001a6e6fad9b16a53b802102b528163885171cf92b15976"
[[package]]
name = "memoffset"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a"
dependencies = [
"autocfg",
]
[[package]]
name = "once_cell"
version = "1.21.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "42f5e15c9953c5e4ccceeb2e7382a716482c34515315f7b03532b8b4e8393d2d"
[[package]]
name = "portable-atomic"
version = "1.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f84267b20a16ea918e43c6a88433c2d54fa145c92a811b5b047ccbe153674483"
[[package]]
name = "proc-macro2"
version = "1.0.103"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ee95bc4ef87b8d5ba32e8b7714ccc834865276eab0aed5c9958d00ec45f49e8"
dependencies = [
"unicode-ident",
]
[[package]]
name = "pyo3"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "37a6df7eab65fc7bee654a421404947e10a0f7085b6951bf2ea395f4659fb0cf"
dependencies = [
"indoc",
"libc",
"memoffset",
"once_cell",
"portable-atomic",
"pyo3-build-config",
"pyo3-ffi",
"pyo3-macros",
"unindent",
]
[[package]]
name = "pyo3-build-config"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f77d387774f6f6eec64a004eac0ed525aab7fa1966d94b42f743797b3e395afb"
dependencies = [
"target-lexicon",
]
[[package]]
name = "pyo3-ffi"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2dd13844a4242793e02df3e2ec093f540d948299a6a77ea9ce7afd8623f542be"
dependencies = [
"libc",
"pyo3-build-config",
]
[[package]]
name = "pyo3-macros"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eaf8f9f1108270b90d3676b8679586385430e5c0bb78bb5f043f95499c821a71"
dependencies = [
"proc-macro2",
"pyo3-macros-backend",
"quote",
"syn",
]
[[package]]
name = "pyo3-macros-backend"
version = "0.27.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70a3b2274450ba5288bc9b8c1b69ff569d1d61189d4bff38f8d22e03d17f932b"
dependencies = [
"heck",
"proc-macro2",
"pyo3-build-config",
"quote",
"syn",
]
[[package]]
name = "quote"
version = "1.0.42"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a338cc41d27e6cc6dce6cefc13a0729dfbb81c262b1f519331575dd80ef3067f"
dependencies = [
"proc-macro2",
]
[[package]]
name = "rustversion"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b39cdef0fa800fc44525c84ccb54a029961a8215f9619753635a9c0d2538d46d"
[[package]]
name = "syn"
version = "2.0.111"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "390cc9a294ab71bdb1aa2e99d13be9c753cd2d7bd6560c77118597410c4d2e87"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "target-lexicon"
version = "0.13.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df7f62577c25e07834649fc3b39fafdc597c0a3527dc1c60129201ccfcbaa50c"
[[package]]
name = "unicode-ident"
version = "1.0.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9312f7c4f6ff9069b165498234ce8be658059c6728633667c526e27dc2cf1df5"
[[package]]
name = "unindent"
version = "0.2.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7264e107f553ccae879d21fbea1d6724ac785e8c3bfc762137959b5802826ef3"
[[package]]
name = "zilch-pyo3-example"
version = "0.1.0"
dependencies = [
"pyo3",
]

7
examples/rust/Cargo.toml Normal file
View file

@ -0,0 +1,7 @@
[package]
name = "zilch-pyo3-example"
version = "0.1.0"
edition = "2024"
[dependencies]
pyo3 = { version = "0.27.1", features = ["auto-initialize"] }

17
examples/rust/src/main.rs Normal file
View file

@ -0,0 +1,17 @@
use pyo3::prelude::*;
use pyo3::types::IntoPyDict;
use pyo3::ffi::c_str;
fn main() -> PyResult<()> {
Python::attach(|py| {
let sys = py.import("sys")?;
let version: String = sys.getattr("version")?.extract()?;
let locals = [("os", py.import("os")?)].into_py_dict(py)?;
let code = c_str!("os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'");
let user: String = py.eval(code, None, Some(&locals))?.extract()?;
println!("Hello {}, I'm Python {}", user, version);
Ok(())
})
}