docs: Document core concepts

Change-Id: I6a6a6964d6bded229cd640463eaac70fd52df233

docs/modules/ROOT/pages/core/zilch-as-lib.adoc

@@ -0,0 +1,193 @@
+= Using Zilch as a library
+:page-pagination: prev
+
+Zilch is able to be used as a library; provided you have knowledge of Chicken
+Scheme. 
+
+Once the Zilch libraries are installed (or inside a `nix-shell` inside the
+Zilch repo), open an interpreter (or an editor).
+
+These examples assume you are running in an interpreter with
+`(scheme base)`, `(zilch zexpr)`, and `(zilch magic)` in scope.
+
+== Creating and building derivations
+
+To start off, creating an arbitrary derivation can be done with one call:
+
+[,scheme,line-comment=;]
+----
+(define normal-derivation
+ (store-path-for-drv
+  "hello" ; <1>
+  "x86_64-linux" ; <2>
+  '("/bin/sh" "-c" "echo hi > $out") ; <3>
+  '() ; <4>
+  '("out"))) ; <5>
+;; (("out" . #<store path /nix/store/mjs27ix6ig2bkbi3s3sm470vrv4lf7ic-hello
+;;             (/nix/store/76w21n1f03fs5kw8fnffphx7qrqffw6r-hello.drv!out)>))
+----
+<1> Name for the derivation (used in output store paths)
+<2> Architecture that this derivation needs to be built on.
+<3> Derivation builder and its arguments (unlike Nix, these are considered one list)
+<4> Any environment variables to set whilst executing the builder
+<5> The list of output names to use for this derivation.
+
+This procedure returns an alist of `<store-path>` records, which refer to
+derivation store paths. These derivations may or may not exist in the Nix
+store, however, as Zilch only writes them to the store when necessary, or when
+requested. To do this manually, call `store-path-materialize`: 
+
+[,scheme]
+----
+(store-path-materialize (cdar normal-derivation))
+----
+
+After materialization, it's now possible to build this derivation, both
+inside Zilch and outside it.
+
+[,console]
+----
+$ nix-build '/nix/store/76w21n1f03fs5kw8fnffphx7qrqffw6r-hello.drv!out'
+this derivation will be built:
+  /nix/store/76w21n1f03fs5kw8fnffphx7qrqffw6r-hello.drv
+building '/nix/store/76w21n1f03fs5kw8fnffphx7qrqffw6r-hello.drv'...
+/nix/store/mjs27ix6ig2bkbi3s3sm470vrv4lf7ic-hello
+
+$ cat /nix/store/mjs27ix6ig2bkbi3s3sm470vrv4lf7ic-hello
+hi
+----
+
+[,scheme]
+----
+(store-path-build (cdar normal-derivation))
+;> [..building "/nix/store/76w21n1f03fs5kw8fnffphx7qrqffw6r-hello.drv"]
+;> [0/1 builds, 1 running]
+;; 1
+----
+
+These store paths are a type of ``<zexp>``; a format used to describe data that
+depends on paths from the Nix store.
+
+== Zexpressions
+While it's possible to manually build every store paths you need, and handle
+them as plain strings, this is cumbersome and error-prone. It also doesn't work
+when you have to deal with placeholders, which are used for content-addressed
+derivations. So, to handle these, the `store-path` concept is extended to
+generic Scheme expressions. These use `zexp` (or `++#~++`) and you can use
+`zexp-unquote` (or `++#$++`) to read out the contents of one `zexp` inside another.
+Finally, to read out a `zexp` outside another `zexp`, you can use `zexp-unwrap`:
+
+[,scheme]
+----
+(define example-zexp
+ (zexp (zexp-unquote (cdar normal-derivation))))
+;; #<zexp val: (zexp-unquote (cdar normal-derivation))>
+
+(define unwrapped (zexp-unwrap example-zexp))
+;; #<zexp-evaluation val: "/nix/store/mjs27ix6ig2bkbi3s3sm470vrv4lf7ic-hello",
+;;  drvs: ((#<derivation "hello" …> "out")),
+;;  srcs: ()>
+----
+
+A `zexp` represents an arbitrary S-expression. It's also possible to run code
+when a `zexp` is used, similarly to `quasiquote`. Like `quasiquote`, this uses
+`unquote` (or its syntax sugar). Doing this also retains the context of any
+``zexp-unquote``d value:
+
+[,scheme]
+----
+(define more-complex
+ (zexp ,(string-append
+         "The store path is at "
+         (zexp-unquote example-zexp))))
+
+(zexp-unwrap more-complex)
+;; #<zexp-evaluation
+;;   val: "The store path is at /nix/store/mjs27ix6ig2bkbi3s3sm470vrv4lf7ic-hello",
+;;   drvs: ((#<derivation "hello" …> "out")),
+;;   srcs: ()>
+----
+
+These expressions can then be used in derivations (e.g. through
+`store-path-for-drv`) too:
+
+[,scheme]
+----
+(define another-derivation
+ (store-path-for-drv "hello2" "x86_64-linux"
+  '("/bin/sh" "-c" "echo \"$foo\" > $out")
+  (list (cons "foo" more-complex))
+  '("out")))
+;; (("out" . #<store path /nix/store/nz1fv397b3cix74d58i8kh2nj8knvb72-hello2
+;;             (/nix/store/gvglnwmalgdnri3zwzmkscg61ll8nas1-hello2.drv!out)>))
+
+(store-path-materialize (cdar another-derivation))
+
+(store-path-build (cdar another-derivation))
+;! /nix/store/nz1fv397b3cix74d58i8kh2nj8knvb72-hello2 contains
+;! "The store path is at /nix/store/mjs27ix6ig2bkbi3s3sm470vrv4lf7ic-hello"
+----
+
+As ``zexp``s aren't limited to strings, it's possible to write expressive
+derivations relatively easy.
+
+[,scheme]
+----
+(define another-derivation
+ (store-path-for-drv "hello2" "x86_64-linux"
+  #~("/bin/sh" "-c" ,(string-append "echo \"" #$more-complex "\" > $out"))
+  '()
+  '("out")))
+;; (("out" . #<store path /nix/store/91v91bmyyjl04ccirp9x1bb1ync3c0f5-hello2
+;;             (/nix/store/a2315xkzjssicyxgf0ji0j8a3y085hkm-hello2.drv!out)>))
+
+(store-path-materialize (cdar another-derivation))
+----
+
+== Content-addressed derivations
+Zilch implements xref:./derivations.adoc[its own content-addressed derivations],
+which can be used almost as easily as input-addressed derivations. However,
+they have an important caveat: Their output store paths are only known at build
+time. To improve performance, the store paths render as a "placeholder", which
+gets substituted at build time.
+
+[,scheme]
+----
+(define content-addressed-derivation
+ (store-path-for-ca-drv "hello" "x86_64-linux"
+  '("/bin/sh" "-c" "echo hi > $out")
+  '()
+  '("out")))
+;; (("out" . #<store path /106ghhc6jy33ycylgj3ndzwb1l6sdkm75likgd7fm5pr7fjfx5cv
+;;             (ca~ /nix/store/bl3mp0i3kd2rssjgynga84gwzx5cj653-hello.drv!out)>))
+----
+
+Content addressed store paths can be determined by the output path not starting
+with the Nix store, and the `ca~` marker in their representation. These
+derivations also cannot be materialized; `store-path-materialize` will ignore
+them.
+
+When these store paths are used in other content-addressed derivations, they
+will work as intended. Their output, when ``zexp-unwrap``ed, however, will
+still contain placeholders:
+
+[,scheme]
+----
+(zexp-unwrap (cdar content-addressed-derivation))
+;; #<zexp-evaluation val: "/106ghhc6jy33ycylgj3ndzwb1l6sdkm75likgd7fm5pr7fjfx5cv",
+;;   drvs: ((#<derivation "hello" …> "out")),
+;;   srcs: ()>
+----
+
+To resolve their placeholders, you need to use `store-path-realised`. This will
+evaluate the `zexp` or `store-path`, and resolve placeholders:
+
+[,scheme]
+----
+(store-path-realised (cdar content-addressed-derivation))
+;; "/nix/store/a5izqk5bgpxcrrnrm1m8n1fvyq2jlc52-hello"
+----
+
+Alternatively, it's possible to use `store-path-devirtualise`. This does the
+same thing, but returns a `zexp`, which can then be used in an input-addressed
+derivation.