Commit graph

75 commits

Author SHA1 Message Date
c4ec32eac3 (zilch nix daemon): generate new logger instance per startWorking
In the case of CA handling, a large quantity (32+) of daemon
connections may exist at once. Handling this with one single derivation
build counter is counterintuitive, and is why the Ninja CLI did not yet
use the counter.

Change-Id: Ia88b4f014ae8df8b1a900b881ac821ff6a6a6964
2025-11-21 14:37:15 +00:00
222b0eb5a3 (zilch statusbar): Fix resilience during shutdown
When the (current-error-port) is closed, other threads in the middle
of using the port are likely to error out with a "port is closed"
error. When this occurs, the Chicken Scheme runtime will either
deadlock (when this happens on the output port), or worse, _hang_.
As the barf procedure usess the current error port, it itself hits this
same issue.

We can't perfectly protect against this, as there's no mutex on the current
output/error port; but we can improve the chances of hitting it massively,
to the point where a simple program won't hit the deadlock.

Change-Id: I3ccec0073b375ca21e8b46861c2f8c146a6a6964
2025-11-21 14:33:36 +00:00
1247097de2 core, (zilch lang rust): fix egg definitions
This is enough to make them all compile under native `chicken-install',
as opposed to Nix-only.

Change-Id: I40e480f0b98be40aaf6cc0fe6f0eedc26a6a6964
2025-11-19 12:39:12 +00:00
56b114ef2c support building with chicken-install too
Change-Id: I47510ca9d5892f1def55ab14f610ba796a6a6964
2025-11-19 12:25:55 +00:00
933b942a62 (zilch planner): drop
Turns out this idea was kind of a dead end.

Change-Id: Id90b2249191ad66b0691471aa1721d726a6a6964
2025-11-14 13:13:33 +00:00
42ea5fd576 (zilch lib rewrite): introduce bytevector/string rewrite helper
Change-Id: I6a6a69645a253c8bc0cfd0919c280c20a6404c1e
2025-11-14 13:01:04 +00:00
feeb14eed5 (zilch nixpkgs): add dataflow tracking support
Dataflow tracking runs the installPhase of a derivation, and then
does its best to track build input -> store path output pathways.
This allows for substituting the installPhase of a derivation as much
as possible with relatively quick store path logic, allowing for cross-
project build input tracking. 

Change-Id: I6a6a69642530b32edbc2c521a3f584703731b6e1
2025-11-14 13:01:04 +00:00
e76c57a388 (zilch nix daemon): Upgrade daemon version to 2.18-like
This is necessary for support with recent Lix.

Change-Id: I6a6a69648db16e23ee82a5107cec858ccc06fb42
2025-11-14 13:01:04 +00:00
a80266d9d8 (zilch): document most exported symbols
Change-Id: I6a6a6964d3be7b8c6306a21d810c639f30253d38
2025-11-14 13:01:04 +00:00
6a1efc6a92 (zilch zexpr): print zexpr stack when unquoting fails 2025-11-14 13:01:04 +00:00
933f46a385 (zilch zexpr): remove zexp-with-context 2025-11-14 13:01:04 +00:00
c7acae633f (zilch nix drv): remove impure derivation support 2025-11-14 13:01:04 +00:00
c9e48ff1ba (zilch nix drv): expose drv-is-fod 2025-11-14 13:01:04 +00:00
32673728f9 (zilch nix drv): deduplicate input sources 2025-11-14 13:01:04 +00:00
99bb2501e3 (zilch magic): fix off-by-one when replacing the end of the string 2025-11-14 13:01:04 +00:00
6e4d191145 (zilch magic): close the daemon connection when we're done with it 2025-11-14 13:01:04 +00:00
c7a92cb052 (zilch magic): store resolved paths for CA drvs in meta
The non-Ninja implementations have a lot more derivation-realising
churn; we don't need to realise every single "extract a file from a zip
file" helper again for each file.
2025-11-14 13:01:04 +00:00
78b41236ab (zilch magic): handle FODs with CA dependencies properly
Whilst not CA themselves, they can safely depend on CA inputs. Thus,
they need rewrites.
2025-11-14 13:01:04 +00:00
c192be2cf8 (zilch magic): remove impure drv support
Change-Id: I6a6a6964785281e054f986bfc29909228b8e0e2d
2025-11-14 13:01:04 +00:00
ff58095797 (zilch file): fix unneccessary double-serialization in zfile
Change-Id: I6a6a69640707ff155f425f05aa857f03bad2650d
2025-11-14 13:01:04 +00:00
31bdc68f8c (zilch magic): add post-build hook
..This is really starting to look like a build scheduler, huh?

Change-Id: I6a6a696433e0a13f8de759061d47e737d48f4342
2025-11-14 13:01:04 +00:00
9b61f4df0a (zilch core magic): use bytevectors to calculate rewrites, use mapping
This massively improves performance. Turns out strings are slow!

Change-Id: I6a6a6964c48e10fe046298eaf29c4e86f7d30cf1
2025-11-14 13:01:04 +00:00
9382082702 (zilch magic): support fallback CA drvs
A derivation can now be tagged with a fallback, which is substituted in
place of the failed derivation.

Change-Id: I6a6a696418b5d9ced3ba16ce458f55f23813c32b
2025-11-14 13:01:04 +00:00
a2ec3ded0f (zilch nix drv): add derivation metadata slot
This will be used to store information for CA derivation fallbacks,
for e.g. handling header dependencies.

Change-Id: I6a6a696467311fcfbb0f01fb13998153b0cc04a7
2025-11-14 13:01:04 +00:00
85895d2e2a (zilch nix daemon): hide "0/1 builds, 0 running"
Change-Id: I6a6a696471170e79b255de57a14e90c48f6479fa
2025-11-14 13:01:04 +00:00
e08558456a (zilch vfs): support symlinks
Change-Id: I6a6a69647b11d85f5440262c8e5e27cc220aefb8
2025-11-14 13:01:04 +00:00
db3b2316f0 (zilch magic): configurable CA build job count
Change-Id: I6a6a6964f157a33b35dae7245c344be7907c5221
2025-11-14 13:01:04 +00:00
9cd5bfb8dd (zilch nixpkgs): fix reproducibility of environment-for-derivation
Change-Id: I6a6a6964095f727498ed30d06a7236c9b241fdd0
2025-11-14 13:01:04 +00:00
0c0c4b5d22 (zilch magic): support devirtualising zexpr without losing zexpr-ness
Change-Id: I6a6a6964787b3a1fcd3223df258e34a8daba5dc8
2025-11-14 13:01:04 +00:00
4008106946 (zilch nixpkgs): remove other load-bearing derivation attrs on drv-to-env
Change-Id: I6a6a6964cda47ecb852e71490412e6236398fe60
2025-11-14 13:01:04 +00:00
83c6d94de3 (zilch magic): re-raise the first CA error
Change-Id: I6a6a6964a748b10ec12aa5cbc1cd736bffe077ff
2025-11-14 13:01:04 +00:00
f3618358aa (zilch magic): fix ca build error conditions more 2025-11-14 13:01:04 +00:00
55dd6a8483 (zilch magic): handle errors properly in CA builds
This properly stops the rest of the builders, instead of causing a
deadlock.
2025-11-14 13:01:04 +00:00
84748541c7 (zilch magic): purposely dangle self-references in IA->CA handling
This is to workaround an issue I don't know how to fix right now.
In effect: Some of the Nix code contains the placeholder for $out,
which then gets translated to $out for that store path. Converting that
fails, for obvious reasons. However, rewriting those to self-references
is more complicated than I want to get into right now, and as the
behavior of self-references is vaguely broken (and this only affects
one test and a comment), I'm going to take the easy way out and make
it generate a dangling reference.
2025-11-14 13:01:04 +00:00
8e2f0f3014 (zilch magic): refuse to handle self-references 2025-11-14 13:01:04 +00:00
9b03914ec1 (zilch magic): Replace daemon-handled CA derivations with Zilch
Lix is deprecating CA derivations, and it's not too difficult to have
this be handled entirely by Zilch. This also improves stability and
future handling of the feature, as Zilch does not depend on how the
daemon handles CA derivations. Though right now, the behavior of
CA derivations is identical to that of the experimental feature, at
this time.

Building CA derivations requires a small scheduler to bypass the Nix
behavior. Each CA derivation is recorded, and then all derivations that
have no CA dependencies get built. These are used to resolve the
realisations for the derivations that depend upon them, and the the
cycle continues.
2025-11-14 13:01:04 +00:00
83a4a0569d (zilch magic): add helper to turn a CA drv into a non-CA equivalent
This reuses the exact NAR, so it cannot depend on the exact path of
$out. That is, however, good enough for the usecases Zilch has.
2025-11-14 13:01:04 +00:00
8859253253 (zilch magic): fix non-CA store path building 2025-11-14 13:01:04 +00:00
7e8e504f89 (zilch magic): whitespace fixes 2025-11-14 13:01:04 +00:00
ec41674598 (zilch magic): add helper to turn a store path into its FOD equivalent 2025-11-14 13:01:04 +00:00
9d72e8a246 (zilch nix path): support content-addressed paths with references 2025-11-14 13:01:04 +00:00
f8dd673609 (zilch nix daemon): add more daemon calls 2025-11-14 13:01:04 +00:00
355af66489 (zilch nix drv): finally fix overlapping environment variables
Having an `out` environment variable now correctly gets it ignored
entirely, rather than behave unexpectedly.
2025-11-14 13:01:04 +00:00
3f2e90543e (zilch nix drv): fix failed derivation output logging 2025-11-14 13:01:04 +00:00
5b4bb8cd8a (zilch nix drv): fix serialized derivation format caching
This was broken at some point, and I'm not sure if this is a load-bearing
bug. Let's find out.
2025-11-14 13:01:04 +00:00
fc2d2551fe (zilch vfs): add vfs-to-string 2025-11-14 13:01:04 +00:00
8080da3ba0 (zilch vfs): add parent directories when using vfs-append-file 2025-11-14 13:01:04 +00:00
8eb1934d03 (zilch vfs): add vfs-append-file 2025-11-14 13:01:04 +00:00
80883d3206 (zilch core nixpkgs): add "eval raw in nixpkgs" function
This allows evaluating code that isn't as obviously "one derivation";
e.g. `pkgs.mkShell'.
2025-11-14 13:01:04 +00:00
5176533e0a (zilch core nix daemon): fix error handling 2025-11-14 13:01:04 +00:00