96 lines
4.1 KiB
Text
96 lines
4.1 KiB
Text
|
= Content-addressed derivations and the Zilch build scheduler
|
||
|
|
:page-pagination: next
|
||
|
|
|
||
|
|
Zilch implements its own content-addressed derivations, along with a subset of
|
||
|
|
the Nix build scheduler. This brings a few important features with it, such as
|
||
|
|
derivation fallbacks and post-build hooks. It also means that content-addressed
|
||
|
|
derivations can be experimented with inside Zilch without worrying about
|
||
|
|
changes to Nix breaking it, and without enabling any `experimental-features`.
|
||
|
|
|
||
|
|
== Content-addressed derivations
|
||
|
|
Currently, the content-addressed derivation format used by Zilch closely
|
||
|
|
matches that of Nix, with a few subtleties. content-addressed derivations are
|
||
|
|
never written to the Nix store, as that would require Nix support them.
|
||
|
|
|
||
|
|
If a Zilch CA derivation is built, it is translated to an input-addressed
|
||
|
|
derivation and built. Each output of this derivation is then queried and the
|
||
|
|
NAR hash is used to copy the output path to its final, content-addressed,
|
||
|
|
destination.footnote:[This means self-references aren't supported; though as
|
||
|
|
these are not correctly handled in Nix either this problem has been punted for
|
||
|
|
now.] These store paths are then used to substitute the placeholder in the
|
||
|
|
generated derivation for any derivation that depends on it.
|
||
|
|
|
||
|
|
The output of these content-addressed derivations has the Deriver set properly,
|
||
|
|
so it's possible to use the Nix CLI to quickly query the (input-addressed
|
||
|
|
equivalent) derivation used to build them, using a shell script:
|
||
|
|
|
||
|
|
[source,bash]
|
||
|
|
----
|
||
|
|
nix derivation show "$(nix path-info --json /nix/store/mjs27ix6ig2bkbi3s3sm470vrv4lf7ic-hello | jq -r '.[].deriver')
|
||
|
|
----
|
||
|
|
|
||
|
|
|
||
|
|
== Build hooks
|
||
|
|
By implementing the content-addressed derivation logic itself, Zilch is able to
|
||
|
|
provide a few features Nix is unable to: Build hooks. When a content-addressed
|
||
|
|
derivation fails to build, it is possible to swap out the derivation, and
|
||
|
|
instead build a different derivation. This is done using xref:generated:zilch.magic.adoc#store-path-register-fallback[`store-path-register-fallback`],
|
||
|
|
which takes a thunk that is evaluated when the build fails, and returns a new
|
||
|
|
derivation to build.
|
||
|
|
|
||
|
|
[source,scheme]
|
||
|
|
----
|
||
|
|
(define fails-to-build
|
||
|
|
(cdar (store-path-for-ca-drv
|
||
|
|
"hello" "x86_64-linux"
|
||
|
|
'("/bin/sh" "-c" "echo i fail to build")
|
||
|
|
'() '("out")))
|
||
|
|
;; #<store path /0qln6zx68yai0x802a804hvwszvddjpkhv87phsdy0snvlnmr3q8
|
||
|
|
;; (ca~ /nix/store/7cp7gl3sdi6k9kvrg0b628fq4khlk5py-hello.drv!out)>
|
||
|
|
|
||
|
|
(define fallback
|
||
|
|
(cdar (store-path-for-ca-drv
|
||
|
|
"hello" "x86_64-linux"
|
||
|
|
'("/bin/sh" "-c" "echo hi > $out")
|
||
|
|
'() '("out")))
|
||
|
|
;; #<store path /106ghhc6jy33ycylgj3ndzwb1l6sdkm75likgd7fm5pr7fjfx5cv
|
||
|
|
;; (ca~ /nix/store/bl3mp0i3kd2rssjgynga84gwzx5cj653-hello.drv!out)>
|
||
|
|
|
||
|
|
(store-path-register-fallback fails-to-build
|
||
|
|
(lambda () fallback))
|
||
|
|
;; #<store path /0qln6zx68yai0x802a804hvwszvddjpkhv87phsdy0snvlnmr3q8
|
||
|
|
;; (ca~ /nix/store/7cp7gl3sdi6k9kvrg0b628fq4khlk5py-hello.drv!out)>
|
||
|
|
|
||
|
|
(store-path-realised fails-to-build)
|
||
|
|
;> [..building "/nix/store/qmx86vqz7pl191vwpcc8zrpnh98rils8-hello.drv"]
|
||
|
|
;> [0/1 builds, 1 running]
|
||
|
|
;> i fail to build
|
||
|
|
;>
|
||
|
|
;> Error: daemon logger received error: (error "Error" 0 "builder for '\x1b[35;1m/nix/store/qmx86vqz7pl191vwpcc8zrpnh98r...")
|
||
|
|
;> [..building "/nix/store/76w21n1f03fs5kw8fnffphx7qrqffw6r-hello.drv"]
|
||
|
|
;> [0/1 builds, 1 running]
|
||
|
|
;; "/nix/store/a5izqk5bgpxcrrnrm1m8n1fvyq2jlc52-hello"
|
||
|
|
----
|
||
|
|
|
||
|
|
The post-build hook works similarly, but calls the callback with an `alist` of output name to output store path:
|
||
|
|
|
||
|
|
[source,scheme]
|
||
|
|
----
|
||
|
|
(define example
|
||
|
|
(cdar (store-path-for-ca-drv
|
||
|
|
"hello" "x86_64-linux"
|
||
|
|
'("/bin/sh" "-c" "echo hi > $out")
|
||
|
|
'() '("out")))
|
||
|
|
;; #<store path /106ghhc6jy33ycylgj3ndzwb1l6sdkm75likgd7fm5pr7fjfx5cv
|
||
|
|
;; (ca~ /nix/store/bl3mp0i3kd2rssjgynga84gwzx5cj653-hello.drv!out)>
|
||
|
|
|
||
|
|
(store-path-register-post-build example
|
||
|
|
(lambda (vals) (write vals) (newline)))
|
||
|
|
;; #<store path /106ghhc6jy33ycylgj3ndzwb1l6sdkm75likgd7fm5pr7fjfx5cv
|
||
|
|
;; (ca~ /nix/store/bl3mp0i3kd2rssjgynga84gwzx5cj653-hello.drv!out)>
|
||
|
|
|
||
|
|
(store-path-realised example)
|
||
|
|
;> (("out" . "/nix/store/a5izqk5bgpxcrrnrm1m8n1fvyq2jlc52-hello"))
|
||
|
|
;; "/nix/store/a5izqk5bgpxcrrnrm1m8n1fvyq2jlc52-hello"
|
||
|
|
----
|