docs: Document core concepts

Change-Id: I6a6a6964d6bded229cd640463eaac70fd52df233

docs/modules/ROOT/pages/core/derivations.adoc

@@ -0,0 +1,95 @@
+= Content-addressed derivations and the Zilch build scheduler
+:page-pagination: next
+
+Zilch implements its own content-addressed derivations, along with a subset of
+the Nix build scheduler. This brings a few important features with it, such as
+derivation fallbacks and post-build hooks. It also means that content-addressed
+derivations can be experimented with inside Zilch without worrying about
+changes to Nix breaking it, and without enabling any `experimental-features`.
+
+== Content-addressed derivations
+Currently, the content-addressed derivation format used by Zilch closely
+matches that of Nix, with a few subtleties. content-addressed derivations are
+never written to the Nix store, as that would require Nix support them.
+
+If a Zilch CA derivation is built, it is translated to an input-addressed
+derivation and built. Each output of this derivation is then queried and the
+NAR hash is used to copy the output path to its final, content-addressed,
+destination.footnote:[This means self-references aren't supported; though as
+these are not correctly handled in Nix either this problem has been punted for
+now.] These store paths are then used to substitute the placeholder in the
+generated derivation for any derivation that depends on it.
+
+The output of these content-addressed derivations has the Deriver set properly,
+so it's possible to use the Nix CLI to quickly query the (input-addressed
+equivalent) derivation used to build them, using a shell script:
+
+[source,bash]
+----
+nix derivation show "$(nix path-info --json /nix/store/mjs27ix6ig2bkbi3s3sm470vrv4lf7ic-hello | jq -r '.[].deriver')
+----
+
+
+== Build hooks
+By implementing the content-addressed derivation logic itself, Zilch is able to
+provide a few features Nix is unable to: Build hooks. When a content-addressed
+derivation fails to build, it is possible to swap out the derivation, and
+instead build a different derivation. This is done using xref:generated:zilch.magic.adoc#store-path-register-fallback[`store-path-register-fallback`],
+which takes a thunk that is evaluated when the build fails, and returns a new
+derivation to build.
+
+[source,scheme]
+----
+(define fails-to-build
+ (cdar (store-path-for-ca-drv
+        "hello" "x86_64-linux"
+        '("/bin/sh" "-c" "echo i fail to build")
+        '() '("out")))
+;; #<store path /0qln6zx68yai0x802a804hvwszvddjpkhv87phsdy0snvlnmr3q8
+;;   (ca~ /nix/store/7cp7gl3sdi6k9kvrg0b628fq4khlk5py-hello.drv!out)>
+
+(define fallback
+ (cdar (store-path-for-ca-drv
+        "hello" "x86_64-linux"
+        '("/bin/sh" "-c" "echo hi > $out")
+        '() '("out")))
+;; #<store path /106ghhc6jy33ycylgj3ndzwb1l6sdkm75likgd7fm5pr7fjfx5cv
+;;   (ca~ /nix/store/bl3mp0i3kd2rssjgynga84gwzx5cj653-hello.drv!out)>
+
+(store-path-register-fallback fails-to-build
+ (lambda () fallback))
+;; #<store path /0qln6zx68yai0x802a804hvwszvddjpkhv87phsdy0snvlnmr3q8
+;;   (ca~ /nix/store/7cp7gl3sdi6k9kvrg0b628fq4khlk5py-hello.drv!out)>
+
+(store-path-realised fails-to-build)
+;> [..building "/nix/store/qmx86vqz7pl191vwpcc8zrpnh98rils8-hello.drv"]
+;> [0/1 builds, 1 running]
+;> i fail to build
+;> 
+;> Error: daemon logger received error: (error "Error" 0 "builder for '\x1b[35;1m/nix/store/qmx86vqz7pl191vwpcc8zrpnh98r...")
+;> [..building "/nix/store/76w21n1f03fs5kw8fnffphx7qrqffw6r-hello.drv"]
+;> [0/1 builds, 1 running]
+;; "/nix/store/a5izqk5bgpxcrrnrm1m8n1fvyq2jlc52-hello"
+----
+
+The post-build hook works similarly, but calls the callback with an `alist` of output name to output store path:
+
+[source,scheme]
+----
+(define example
+ (cdar (store-path-for-ca-drv
+        "hello" "x86_64-linux"
+        '("/bin/sh" "-c" "echo hi > $out")
+        '() '("out")))
+;; #<store path /106ghhc6jy33ycylgj3ndzwb1l6sdkm75likgd7fm5pr7fjfx5cv
+;;   (ca~ /nix/store/bl3mp0i3kd2rssjgynga84gwzx5cj653-hello.drv!out)>
+
+(store-path-register-post-build example
+ (lambda (vals) (write vals) (newline)))
+;; #<store path /106ghhc6jy33ycylgj3ndzwb1l6sdkm75likgd7fm5pr7fjfx5cv
+;;   (ca~ /nix/store/bl3mp0i3kd2rssjgynga84gwzx5cj653-hello.drv!out)>
+
+(store-path-realised example)
+;> (("out" . "/nix/store/a5izqk5bgpxcrrnrm1m8n1fvyq2jlc52-hello"))
+;; "/nix/store/a5izqk5bgpxcrrnrm1m8n1fvyq2jlc52-hello"
+----