zilch/docs/modules/ROOT/pages/ninja/usage.adoc

= Usage
:page-pagination: next

`zilch-cli-ninja` allows you to use Zilch to compile a Ninja-based nixpkgs
derivation reproducibly, and to quickly modify it and rebuild it reproducibly
as well. To use the tool, you first have to write a configuration file. This
document will use this example, for `gtest`:

.`gtest.scm`
[,scheme]
----
(environment: "pkgs.gtest"
 depfile-path: "gtest-deps.scm")
----

This can then be used with the various subcommands of `zilch-cli-ninja`:

== Building
To build a derivation with `zilch-cli-ninja`, use the `build` subcommand.
[,console]
----
$ zilch-cli-ninja -f gtest.scm build <1>
…
dev	-> #<store path /nix/store/lmdgblqm0prkx3hcs2q76rzcvg1xm6cm-gtest-1.16.0-dev (/nix/store/ksrbqg2nkgm8zq3g59gzydf5976griis-gtest-1.16.0.drv!dev)>
out	-> #<store path /nix/store/b2ylcnfsaf2wmzk51whgq5r86xb955xq-gtest-1.16.0 (/nix/store/ksrbqg2nkgm8zq3g59gzydf5976griis-gtest-1.16.0.drv!out)>

$ zilch-cli-ninja -f gtest.scm build googlemock/CMakeFiles/gmock_main.dir/src/gmock_main.cc.o <2>
…
googlemock/CMakeFiles/gmock_main.dir/src/gmock_main.cc.o	-> "/nix/store/c96j8sqk08xj15pififz5myjzskg0cqw-Building-CXX-object-googlemock-CMakeFiles-gmock_main.dir-src-gmock_main.cc.o"
----
<1> Build the full derivation
<2> Build a specific output of the Ninja file

== Source editing
Zilch makes it very easy to edit the source of any derivation built in this
manner. With one command, you can export the source code used for building.
This source can then be used to rebuild parts of the derivation that have changed.

[,console]
----
$ zilch-cli-ninja -f gtest.scm source gtest-src <1>
…

$ zilch-cli-ninja -f gtest.scm -s gtest-src diff <2>
…

$ zilch-cli-ninja -f gtest.scm -s gtest-src build <3>
…
dev	-> #<store path /nix/store/lmdgblqm0prkx3hcs2q76rzcvg1xm6cm-gtest-1.16.0-dev (/nix/store/ksrbqg2nkgm8zq3g59gzydf5976griis-gtest-1.16.0.drv!dev)>
out	-> #<store path /nix/store/b2ylcnfsaf2wmzk51whgq5r86xb955xq-gtest-1.16.0 (/nix/store/ksrbqg2nkgm8zq3g59gzydf5976griis-gtest-1.16.0.drv!out)>
----

It's also possible to use `override-source: "<path>"` inside the configuration
file, which allows skipping the `-s` arguments. This is also used when
<<_rewrite,using cross-project builds>>.

[#source-editing-warning]
WARNING: Zilch will not regenerate the build files when editing source in this
manner, for performance and logistical reasons. To change the build files,
override the `src` attribute of the environment's derivation.

== Limitations
To make the Nixpkgs integration work, the Zilch code breaks up the derivation's
phases into two steps: unpack + patch + configure, and check + install. This
can cause issues with derivations that expect to share variables between these
phases. If `meson` is present, it is assumed to be used, otherwise it is
assumed the derivation is `cmake`-based. Other build systems are not yet
supported.

The final store path of the derivation is not known at configuration time,
unlike normal derivations. Meson and CMake are configured to install into a
prefix, and are then rewritten (using `sed`) to the final output store path.
This may cause issues with programs that e.g. compress a file containing these
paths during their build.footnote:[The only one to have known issues is Node.]

== Patches
In some cases, the code may not handle running in Zilch's environment. In this
case, it's possible to add extra code to run before running the command of a
target.

These patches are defined as part of the configuration file, and are arbitrary
Scheme code:

[,scheme]
----
(environment: "import ./sample.nix"
 patch:
  (lambda (edge)
   (if (string=? (build-edge-rule edge) "foo")
     "rm src/foo; echo 'replacement file' > src/foo"
     #f))) 
----

Patches are arbitrary shell scripts, run with GNU Coreutils in `$PATH`. The
environment variable `$_ZILCH_ROOT` points to the root of the source + build
file tree, and can be used to copy non-symlinked files into the working
directory.

[#_rewrite]
== Cross-project incremental builds
Zilch supports rewriting the input of a Ninja project with another Ninja
project. To do this, you will need the name of the derivation, and a project
configuration:

[,scheme]
----
(environment: "import ./sample.nix"
 override-source: "sample-src"
 rewrite:
 ("input-1.2.3"
  environment: "import ../input.nix"
  override-source: "input-src"
  #<...more configuration here>))
----

WARNING: The <<source-editing-warning,warning under "Source editing">> applies
here too. As addition, the boundary between projects is determined by running
the `installPhase` of the project to rewrite. If the installPhase is complex
(e.g. by using compression, rather than just copying inputs to outputs) this
will not work properly. Any code that assumes it can run outputs from the
rewritten project will also not function, as any shared libraries are stubbed
out.

== Automatic file elision
To implement incremental builds efficiently, Zilch automatically slims down the
part of the source directory that is accessible to each target. If a filename
ends in `.cc` or `.c` and is directly depended upon as in input file to a
target, it will be elided, and only the targets that depend on it directly will
be able to read its contents.

The `build/meson-private` directory is fully elided, except for derivations
that directly depend upon it. This is to limit the amount of impurities that
are part of the configuration process. 

=== Dependency tracking
When a depfile cache is provided, and the output being built supports
dependency files, any files ending in `.h` and `.hh` that aren't a known
dependency are elided. However, if the build fails, it will be retried with all
header files visible to the client.

In some cases, this dependency tracking is imprecise, and causes some header
files to not be properly visible to targets that need it. In this case, you can
add an extra entry to the configuration file:

.`lix.scm`
[,scheme]
----
(environment: "(pkgs.lix.override { enableDocumentation = false; }).overrideAttrs (a: { doCheck = false; doInstallCheck = false; })"
 depfile-path: "lix-depfile.scm"
 disallow-elide: (lambda (path) (string=? path "config.h")))
----

This example ensures that `config.h` will be available to all targets being
built for the derivation.

== Library stubs
When using Meson, any libraries that are linked will also simultaneously
generate a stub `.so` file, using `llvm-ifs`. This stub library is valid for
linking, but contains no code, and is generated deterministically. When linking
any other libraries or binaries that depend on this library, the stub is
automatically used, which improves early cutoff between library builds.