| Age | Commit message (Expand) | Author |
|---|
| 2022-06-05 | daemon: Quote consistently within a string....* nix/libstore/build.cc (DerivationGoal::registerOutput): ‘’ → `'.
| Tobias Geerinckx-Rice |
| 2022-05-29 | daemon: Clarify ‘--check’ error when outputs are missing....Drop the confusing ‘invalid’ jargon and display a hint like we do
for ‘--fallback’.
* nix/libstore/build.cc (DerivationGoal::outputsSubstituted): Rewrite error message.
| Tobias Geerinckx-Rice |
| 2022-05-18 | daemon: runChild() is forbidden to talk during environment set up...DerivationGoal::startBuilder() is waiting for an empty line as a check that
the environment set up is fine.
Fixes <https://issues.guix.gnu.org/55324>.
* nix/libstore/build.cc (DerivationGoal::runChild): Remove 'debug'
statement corresponding to bind mounts.
Signed-off-by: Ludovic Courtès <ludo@gnu.org>
| yarl-baudig@mailoo.org |
| 2022-04-14 | daemon: Support systemd-style socket activation....* nix/nix-daemon/guix-daemon.cc (SD_LISTEN_FDS_START): New macro.
(systemd_activation_sockets): New function.
(main): Use it. Remove obsolete 'printMsg' call.
* doc/guix.texi (Invoking guix-daemon): Document socket activation.
| Ludovic Courtès |
| 2022-01-18 | daemon: Always default to gzip for log compression....* nix/libstore/globals.cc (Settings::Settings): Have 'logCompression'
default to COMPRESSION_GZIP unconditionally.
* gnu/services/base.scm (<guix-configuration>)[log-compression]: Default
to 'gzip.
* doc/guix.texi (Invoking guix-daemon, Base Services): Adjust accordingly.
| Ludovic Courtès |
| 2021-11-27 | daemon: Print the line whence we expect an integer....* nix/libstore/local-store.cc (LocalStore::getLineFromSubstituter):
Include the malformed substituter stream line in the error message.
| Tobias Geerinckx-Rice |
| 2021-11-25 | daemon: Read substitute nar size as 'unsigned long long'....Fixes <https://issues.guix.gnu.org/46212>.
Reported by Christopher Baines <mail@cbaines.net>.
Previously, the nar size returned by 'guix substitute' would be read as
an 'int'; thus, values above 2^31 - 1 would be read and then stored as
negative integers in the database.
Regression introduced in 9dfa20a22ae0be3d3b01a7b3d422af97428c627e.
* nix/libstore/build.cc (SubstitutionGoal::finished): Use templatized
'string2Int' instead of 'std::atoi' to get an 'unsigned long long',
which is the type of 'hash.second'.
* tests/store.scm ("substitute and large size"): New test.
| Ludovic Courtès |
| 2021-11-25 | daemon: Read unsigned nar size and download size from substituter....Fixes <https://issues.guix.gnu.org/51983>.
Reported by Tobias Geerinckx-Rice <me@tobias.gr>.
* nix/libstore/local-store.cc (LocalStore::querySubstitutablePathInfos):
Expect 'unsigned long long' for 'downloadSize' and 'narSize'.
* tests/store.scm ("substitute query and large size"): New test.
| Ludovic Courtès |
| 2021-11-19 | daemon: Micro-optimize 'deletePath'....'remove' calls 'unlink' first and falls back to 'rmdir' upon EISDIR.
This change gets rid of the 'unlink' call for every directory being
removed.
* nix/libutil/util.cc (_deletePath): Call 'unlink' or 'rmdir' depending
on 'st.st_mode', rather than call 'remove'.
| Ludovic Courtès |
| 2021-11-16 | daemon: Do not deduplicate files smaller than 8 KiB....Files smaller than 8 KiB typically represent ~70% of the entries in
/gnu/store/.links but only contribute to ~4% of the space savings
afforded by deduplication.
Not considering these files for deduplication speeds up file insertion
in the store and, more importantly, leaves 'removeUnusedLinks' with
fewer entries to traverse, thereby speeding it up proportionally.
Partly fixes <https://issues.guix.gnu.org/24937>.
* config-daemon.ac: Remove symlink hard link check and CAN_LINK_SYMLINK
definition.
* guix/store/deduplication.scm (%deduplication-minimum-size): New
variable.
(deduplicate)[loop]: Do not recurse when FILE's size is below
%DEDUPLICATION-MINIMUM-SIZE.
(dump-port): New procedure.
(dump-file/deduplicate)[hash]: Turn into...
[dump-and-compute-hash]: ... this thunk.
Call 'deduplicate' only when SIZE is greater than
%DEDUPLICATION-MINIMUM-SIZE; otherwise call 'dump-port'.
* nix/libstore/gc.cc (LocalStore::removeUnusedLinks): Drop files where
st.st_size < deduplicationMinSize.
* nix/libstore/local-store.hh (deduplicationMinSize): New declaration.
* nix/libstore/optimise-store.cc (deduplicationMinSize): New variable.
(LocalStore::optimisePath_): Return when PATH is a symlink or smaller
than 'deduplicationMinSize'.
* tests/derivations.scm ("identical files are deduplicated"): Produce
files bigger than %DEDUPLICATION-MINIMUM-SIZE.
* tests/nar.scm ("restore-file-set with directories (signed, valid)"):
Likewise.
* tests/store-deduplication.scm ("deduplicate, below %deduplication-minimum-size"):
New test.
("deduplicate", "deduplicate, ENOSPC"): Produce files bigger than
%DEDUPLICATION-MINIMUM-SIZE.
* tests/store.scm ("substitute, deduplication"): Likewise.
| Ludovic Courtès |
| 2021-10-01 | build: Don’t delete ‘guix-gc.timer’ file....Without this invoking ‘make clean’ would remove ‘guix-gc.timer’, and ‘make’
would fail with.
make[2]: *** No rule to make target 'etc/guix-gc.timer', needed by 'all-am'. Stop.
* nix/local.mk (nodist_systemdservice_DATA): Remove ‘guix-gc.timer’.
| Xinglu Chen |
| 2021-09-20 | etc: Add systemd files for running ‘guix gc’ periodically...* etc/guix-gc.service.in: New file.
* etc/guix-gc.timer: Likewise.
* .gitignore: Ignore generated ‘guix-gc.service’.
* nix/local.mk (nodist_systemdservice_DATA): Add ‘guix-gc.service’ and
‘guix-gc.timer’.
(EXTRA_DIST): Add ‘guix-gc.service.in’ and ‘guix-gc.timer’.
* doc/guix.texi (Binary Installation): Mention the new systemd files.
Signed-off-by: Mathieu Othacehe <othacehe@gnu.org>
| Thiago Jung Bauermann |
| 2021-04-09 | daemon: 'guix substitute' replies on FD 4....This avoids the situation where error messages would unintentionally go
to stderr and be wrongfully interpreted as a reply by the daemon.
Fixes <https://bugs.gnu.org/46362>.
This is a followup to ee3226e9d54891c7e696912245e4904435be191c.
* guix/scripts/substitute.scm (display-narinfo-data): Add 'port'
parameter and honor it.
(process-query): Likewise.
(process-substitution): Likewise.
(%error-to-file-descriptor-4?, with-redirected-error-port): Remove.
(%reply-file-descriptor): New variable.
(guix-substitute): Remove use of 'with-redirected-error-port'. Define
'reply-port' and pass it to 'process-query' and 'process-substitution'.
* nix/libstore/build.cc (SubstitutionGoal::handleChildOutput): Swap
'builderOut' and 'fromAgent'.
* nix/libstore/local-store.cc (LocalStore::getLineFromSubstituter):
Likewise.
* tests/substitute.scm <top level>: Set '%reply-file-descriptor'
rather than '%error-to-file-descriptor-4?'.
| Ludovic Courtès |
| 2021-04-03 | daemon: Remove dead code....Reported by Noisytoot on #guix.
* nix/nix-daemon/shared.hh (showManPage): Remove.
* nix/nix-daemon/nix-daemon.cc (printHelp, programId): Remove.
| Ludovic Courtès |
| 2021-03-30 | gnu: guix: Fix openrc init scripts....Fixes <https://bugs.gnu.org/46871>.
* gnu/packages/package-management.scm (guix)[arguments]: Fix openrc init.
* nix/local.mk (openrcservicedir): Likewise.
Signed-off-by: Efraim Flashner <efraim@flashner.co.il>
| zimoun |
| 2021-03-18 | daemon: Prevent privilege escalation with '--keep-failed' [security]....Fixes <https://bugs.gnu.org/47229>.
Reported by Nathan Nye of WhiteBeam Security.
* nix/libstore/build.cc (DerivationGoal::startBuilder): When 'useChroot'
is true, add "/top" to 'tmpDir'.
(DerivationGoal::deleteTmpDir): Adjust accordingly. When
'settings.keepFailed' is true, chown in two steps: first the "/top"
sub-directory, and then rename "/top" to its parent.
| Ludovic Courtès |
| 2021-03-17 | daemon: Correctly handle '--discover' with no value....Previously, we'd get:
$ guix-daemon --discover
error: basic_string::_M_construct null not valid
* nix/nix-daemon/guix-daemon.cc (parse_opt): Change second argument to
'settings.set' to properly handle case where ARG is NULL.
| Ludovic Courtès |
| 2020-12-19 | daemon: Delegate deduplication to 'guix substitute'....This removes the main source of latency between subsequent downloads.
* nix/libstore/build.cc (SubstitutionGoal::tryToRun): Add a
"deduplicate" key to ENV.
(SubstitutionGoal::finished): Remove call to 'optimisePath'.
* guix/scripts/substitute.scm (process-substitution)[destination-in-store?]
[dump-file/deduplicate*]: New variables.
Pass #:dump-file to 'restore-file'.
* guix/scripts/substitute.scm (guix-substitute)[deduplicate?]: New
variable.
Pass #:deduplicate? to 'process-substitution'.
* guix/serialization.scm (dump-file): Export and augment 'dump-file'.
| Ludovic Courtès |
| 2020-12-19 | daemon: Do not reset timestamps and permissions on substituted items....'guix substitute' now takes care of it via 'restore-file'.
* nix/libstore/build.cc (SubstitutionGoal::finished): Remove call to
'canonicalisePathMetaData'.
| Ludovic Courtès |
| 2020-12-19 | daemon: Let 'guix substitute' perform hash checks....This way, the hash of the store item can be computed as it is restored,
thereby avoiding an additional file tree traversal ('hashPath' call)
later on in the daemon. Consequently, it should reduce latency between
subsequent substitute downloads.
This is a followup to 5ff521452b9ec2aae9ed8e4bb7bdc250a581f203.
* guix/scripts/substitute.scm (narinfo-hash-algorithm+value): New
procedure.
(process-substitution): Wrap INPUT into a hash input port, 'hashed', and
read from it. Compare the actual and expected hashes, and print a
"hash-mismatch" status line when they differ. When they match, print
not just "success" but also the nar hash and size.
* nix/libstore/build.cc (class SubstitutionGoal)[expectedHashStr]:
Remove.
(SubstitutionGoal::finished): Tokenize 'status'. Parse it and handle
"success" and "hash-mismatch" accordingly. Call 'hashPath' only when
the returned hash is not SHA256.
(SubstitutionGoal::handleChildOutput): Remove 'expectedHashStr'
handling.
* tests/substitute.scm ("substitute, invalid hash"): Rename to...
("substitute, invalid narinfo hash"): ... this.
("substitute, invalid hash"): New test.
| Ludovic Courtès |
| 2020-12-08 | daemon: Raise an error if substituter doesn't send the expected hash....It was already impossible in practice for 'expectedHashStr' to be empty
if 'status' == "success".
* nix/libstore/build.cc (SubstitutionGoal::finished): Throw 'SubstError'
when 'expectedHashStr' is empty.
| Ludovic Courtès |
| 2020-12-08 | substitute: Cache and reuse connections while substituting....That way, when fetching a series of substitutes from the same server(s),
the connection is reused instead of being closed/opened for each
substitutes, which saves on network round trips and TLS handshakes.
* guix/http-client.scm (http-fetch): Add #:keep-alive? and honor it.
* guix/progress.scm (progress-report-port): Add #:close? parameter and
honor it.
* guix/scripts/substitute.scm (at-most): Return the tail as a second
value.
(fetch): Add #:port and #:keep-alive? and honor them.
(%max-cached-connections): New variable.
(open-connection-for-uri/cached, call-with-cached-connection): New
procedures.
(with-cached-connection): New macro.
(process-substitution): Wrap 'fetch' call in 'with-cached-connection'.
Pass #:close? to 'progress-report-port'.
| Ludovic Courtès |
| 2020-12-08 | daemon: Run 'guix substitute --substitute' as an agent....This avoids spawning one substitute process per substitution.
* nix/libstore/build.cc (class Worker)[substituter]: New field.
[outPipe, logPipe, pid]: Remove.
(class SubstitutionGoal)[expectedHashStr, status, substituter]: New fields.
(SubstitutionGoal::timedOut): Adjust to check 'substituter'.
(SubstitutionGoal::tryToRun): Remove references to 'outPipe' and
'logPipe'. Run "guix substitute --substitute" as an 'Agent'. Send the
request with 'writeLine'.
(SubstitutionGoal::finished): Likewise.
(SubstitutionGoal::handleChildOutput): Change to fill in
'expectedHashStr' and 'status'.
(SubstitutionGoal::handleEOF): Call 'wakeUp' unconditionally.
(SubstitutionGoal::~SubstitutionGoal): Adjust to check 'substituter'.
* guix/scripts/substitute.scm (process-substitution): Write "success\n"
to stdout upon success.
(%error-to-file-descriptor-4?): New variable.
(guix-substitute): Set 'current-error-port' to file descriptor 4
unless (%error-to-file-descriptor-4?) is false.
Remove "--substitute" arguments. Loop reading line from stdin.
* tests/substitute.scm <top level>: Call '%error-to-file-descriptor-4?'.
(request-substitution): New procedure.
("substitute, no signature")
("substitute, invalid hash")
("substitute, unauthorized key")
("substitute, authorized key")
("substitute, unauthorized narinfo comes first")
("substitute, unsigned narinfo comes first")
("substitute, first narinfo is unsigned and has wrong hash")
("substitute, first narinfo is unsigned and has wrong refs")
("substitute, two invalid narinfos")
("substitute, narinfo with several URLs"): Adjust to new "guix
substitute --substitute" calling convention.
| Ludovic Courtès |
| 2020-12-08 | daemon: Factorize substituter agent spawning....* nix/libstore/local-store.hh (class LocalStore)[substituter]: New
method.
[runningSubstituter]: Turn into a shared_ptr.
* nix/libstore/local-store.cc (LocalStore::querySubstitutablePaths):
Call 'substituter' instead of using inline code.
(LocalStore::querySubstitutablePathInfos): Likewise.
(LocalStore::substituter): New method.
| Ludovic Courtès |
| 2020-12-08 | daemon: Use 'Agent' to spawn 'guix substitute --query'....* nix/libstore/local-store.hh (RunningSubstituter): Remove.
(LocalStore)[runningSubstituter]: Change to unique_ptr<Agent>.
[setSubstituterEnv, didSetSubstituterEnv]: Remove.
[getLineFromSubstituter, getIntLineFromSubstituter]: Take an 'Agent'.
* nix/libstore/local-store.cc (LocalStore::~LocalStore): Remove
reference to 'runningSubstituter'.
(LocalStore::setSubstituterEnv, LocalStore::startSubstituter): Remove.
(LocalStore::getLineFromSubstituter): Adjust to 'run' being an 'Agent'.
(LocalStore::querySubstitutablePaths): Spawn substituter agent if
needed. Adjust to 'Agent' interface.
(LocalStore::querySubstitutablePathInfos): Likewise.
* nix/libstore/build.cc (SubstitutionGoal::tryToRun): Remove call to
'setSubstituterEnv' and add 'setenv' call for "_NIX_OPTIONS" instead.
(SubstitutionGoal::finished): Remove 'readLine' call for 'dummy'.
* guix/scripts/substitute.scm (%allow-unauthenticated-substitutes?):
Remove second argument to 'make-parameter'.
(process-query): Call 'warn-about-missing-authentication'
when (%allow-unauthenticated-substitutes?) is #t.
(guix-substitute): Wrap body in 'parameterize'. Set 'guix-warning-port'
too. No longer exit when 'substitute-urls' returns the empty list. No
longer print newline initially.
* tests/substitute.scm (test-quit): Parameterize 'current-error-port' to
account for the port changes in 'guix-substitute'.
| Ludovic Courtès |
| 2020-12-08 | daemon: 'Agent' constructor takes a list of environment variables....* nix/libutil/util.hh (struct Agent)[Agent]: Add 'env' parameter.
* nix/libutil/util.cc (Agent::Agent): Honor it.
| Ludovic Courtès |
| 2020-12-01 | daemon: Remove unneeded forward declaration....This is a followup to ee9dff34f9317509cb2b833d07a0d5e01a36a4ae.
* nix/libstore/build.cc: Remove 'struct Agent' forward declaration.
| Ludovic Courtès |
| 2020-11-29 | daemon: Remove pre-Guix hack....* nix/libstore/build.cc (DerivationGoal::startBuilder): Remove
"NIX_OUTPUT_CHECKED" hack.
| Ludovic Courtès |
| 2020-11-29 | Use substitute servers on the local network....* guix/scripts/discover.scm: New file.
* Makefile.am (MODULES): Add it.
* nix/nix-daemon/guix-daemon.cc (options): Add "discover" option,
(parse-opt): parse it,
(main): start "guix discover" process when the option is set.
* guix/scripts/substitute.scm (%local-substitute-urls): New variable,
(substitute-urls): add it.
* gnu/services/base.scm (<guix-configuration>): Add "discover?"
field,
(guix-shepherd-service): honor it.
* doc/guix.texi (Invoking guix-daemon): Document "discover" option,
(Base Services): ditto.
| Mathieu Othacehe |
