| Age | Commit message (Expand) | Author |
|---|
| 2021-03-18 | daemon: Prevent privilege escalation with '--keep-failed' [security]....Fixes <https://bugs.gnu.org/47229>.
Reported by Nathan Nye of WhiteBeam Security.
* nix/libstore/build.cc (DerivationGoal::startBuilder): When 'useChroot'
is true, add "/top" to 'tmpDir'.
(DerivationGoal::deleteTmpDir): Adjust accordingly. When
'settings.keepFailed' is true, chown in two steps: first the "/top"
sub-directory, and then rename "/top" to its parent.
| Ludovic Courtès |
| 2020-12-19 | daemon: Delegate deduplication to 'guix substitute'....This removes the main source of latency between subsequent downloads.
* nix/libstore/build.cc (SubstitutionGoal::tryToRun): Add a
"deduplicate" key to ENV.
(SubstitutionGoal::finished): Remove call to 'optimisePath'.
* guix/scripts/substitute.scm (process-substitution)[destination-in-store?]
[dump-file/deduplicate*]: New variables.
Pass #:dump-file to 'restore-file'.
* guix/scripts/substitute.scm (guix-substitute)[deduplicate?]: New
variable.
Pass #:deduplicate? to 'process-substitution'.
* guix/serialization.scm (dump-file): Export and augment 'dump-file'.
| Ludovic Courtès |
| 2020-12-19 | daemon: Do not reset timestamps and permissions on substituted items....'guix substitute' now takes care of it via 'restore-file'.
* nix/libstore/build.cc (SubstitutionGoal::finished): Remove call to
'canonicalisePathMetaData'.
| Ludovic Courtès |
| 2020-12-19 | daemon: Let 'guix substitute' perform hash checks....This way, the hash of the store item can be computed as it is restored,
thereby avoiding an additional file tree traversal ('hashPath' call)
later on in the daemon. Consequently, it should reduce latency between
subsequent substitute downloads.
This is a followup to 5ff521452b9ec2aae9ed8e4bb7bdc250a581f203.
* guix/scripts/substitute.scm (narinfo-hash-algorithm+value): New
procedure.
(process-substitution): Wrap INPUT into a hash input port, 'hashed', and
read from it. Compare the actual and expected hashes, and print a
"hash-mismatch" status line when they differ. When they match, print
not just "success" but also the nar hash and size.
* nix/libstore/build.cc (class SubstitutionGoal)[expectedHashStr]:
Remove.
(SubstitutionGoal::finished): Tokenize 'status'. Parse it and handle
"success" and "hash-mismatch" accordingly. Call 'hashPath' only when
the returned hash is not SHA256.
(SubstitutionGoal::handleChildOutput): Remove 'expectedHashStr'
handling.
* tests/substitute.scm ("substitute, invalid hash"): Rename to...
("substitute, invalid narinfo hash"): ... this.
("substitute, invalid hash"): New test.
| Ludovic Courtès |
| 2020-12-08 | daemon: Raise an error if substituter doesn't send the expected hash....It was already impossible in practice for 'expectedHashStr' to be empty
if 'status' == "success".
* nix/libstore/build.cc (SubstitutionGoal::finished): Throw 'SubstError'
when 'expectedHashStr' is empty.
| Ludovic Courtès |
| 2020-12-08 | substitute: Cache and reuse connections while substituting....That way, when fetching a series of substitutes from the same server(s),
the connection is reused instead of being closed/opened for each
substitutes, which saves on network round trips and TLS handshakes.
* guix/http-client.scm (http-fetch): Add #:keep-alive? and honor it.
* guix/progress.scm (progress-report-port): Add #:close? parameter and
honor it.
* guix/scripts/substitute.scm (at-most): Return the tail as a second
value.
(fetch): Add #:port and #:keep-alive? and honor them.
(%max-cached-connections): New variable.
(open-connection-for-uri/cached, call-with-cached-connection): New
procedures.
(with-cached-connection): New macro.
(process-substitution): Wrap 'fetch' call in 'with-cached-connection'.
Pass #:close? to 'progress-report-port'.
| Ludovic Courtès |
| 2020-12-08 | daemon: Run 'guix substitute --substitute' as an agent....This avoids spawning one substitute process per substitution.
* nix/libstore/build.cc (class Worker)[substituter]: New field.
[outPipe, logPipe, pid]: Remove.
(class SubstitutionGoal)[expectedHashStr, status, substituter]: New fields.
(SubstitutionGoal::timedOut): Adjust to check 'substituter'.
(SubstitutionGoal::tryToRun): Remove references to 'outPipe' and
'logPipe'. Run "guix substitute --substitute" as an 'Agent'. Send the
request with 'writeLine'.
(SubstitutionGoal::finished): Likewise.
(SubstitutionGoal::handleChildOutput): Change to fill in
'expectedHashStr' and 'status'.
(SubstitutionGoal::handleEOF): Call 'wakeUp' unconditionally.
(SubstitutionGoal::~SubstitutionGoal): Adjust to check 'substituter'.
* guix/scripts/substitute.scm (process-substitution): Write "success\n"
to stdout upon success.
(%error-to-file-descriptor-4?): New variable.
(guix-substitute): Set 'current-error-port' to file descriptor 4
unless (%error-to-file-descriptor-4?) is false.
Remove "--substitute" arguments. Loop reading line from stdin.
* tests/substitute.scm <top level>: Call '%error-to-file-descriptor-4?'.
(request-substitution): New procedure.
("substitute, no signature")
("substitute, invalid hash")
("substitute, unauthorized key")
("substitute, authorized key")
("substitute, unauthorized narinfo comes first")
("substitute, unsigned narinfo comes first")
("substitute, first narinfo is unsigned and has wrong hash")
("substitute, first narinfo is unsigned and has wrong refs")
("substitute, two invalid narinfos")
("substitute, narinfo with several URLs"): Adjust to new "guix
substitute --substitute" calling convention.
| Ludovic Courtès |
| 2020-12-08 | daemon: Use 'Agent' to spawn 'guix substitute --query'....* nix/libstore/local-store.hh (RunningSubstituter): Remove.
(LocalStore)[runningSubstituter]: Change to unique_ptr<Agent>.
[setSubstituterEnv, didSetSubstituterEnv]: Remove.
[getLineFromSubstituter, getIntLineFromSubstituter]: Take an 'Agent'.
* nix/libstore/local-store.cc (LocalStore::~LocalStore): Remove
reference to 'runningSubstituter'.
(LocalStore::setSubstituterEnv, LocalStore::startSubstituter): Remove.
(LocalStore::getLineFromSubstituter): Adjust to 'run' being an 'Agent'.
(LocalStore::querySubstitutablePaths): Spawn substituter agent if
needed. Adjust to 'Agent' interface.
(LocalStore::querySubstitutablePathInfos): Likewise.
* nix/libstore/build.cc (SubstitutionGoal::tryToRun): Remove call to
'setSubstituterEnv' and add 'setenv' call for "_NIX_OPTIONS" instead.
(SubstitutionGoal::finished): Remove 'readLine' call for 'dummy'.
* guix/scripts/substitute.scm (%allow-unauthenticated-substitutes?):
Remove second argument to 'make-parameter'.
(process-query): Call 'warn-about-missing-authentication'
when (%allow-unauthenticated-substitutes?) is #t.
(guix-substitute): Wrap body in 'parameterize'. Set 'guix-warning-port'
too. No longer exit when 'substitute-urls' returns the empty list. No
longer print newline initially.
* tests/substitute.scm (test-quit): Parameterize 'current-error-port' to
account for the port changes in 'guix-substitute'.
| Ludovic Courtès |
| 2020-12-01 | daemon: Remove unneeded forward declaration....This is a followup to ee9dff34f9317509cb2b833d07a0d5e01a36a4ae.
* nix/libstore/build.cc: Remove 'struct Agent' forward declaration.
| Ludovic Courtès |
| 2020-11-29 | daemon: Remove pre-Guix hack....* nix/libstore/build.cc (DerivationGoal::startBuilder): Remove
"NIX_OUTPUT_CHECKED" hack.
| Ludovic Courtès |
| 2020-10-09 | nix: Honor '--rounds' when also using '--check'....Fixes <https://issues.guix.gnu.org/40144>.
Until now, the '--rounds' option, when also using '--check', was ignored.
This change makes it possible to use both, so that an item that has already
been built once can be rebuilt as many times as desired.
* nix/libstore/build.cc: Remove the conditionals causing the daemon to
complete a build task early when 'buildMode' is equal to 'nix::bmCheck'.
Reported-by: Brice Waegeneire <brice@waegenei.re>
| Maxim Cournoyer |
| 2020-10-01 | daemon: Try to execute derivation builders only for matching OS kernels....Fixes <https://bugs.gnu.org/43668>.
Previously, guix-daemon would try to run GNU/Hurd executables on
GNU/Linux. execve(2) would succeed, but the executable would
immediately crash.
This change prevents it from attempting to execute "i586-gnu" code on
"*-linux", while preserving the binfmt_misc-friendly behavior
implemented in commit 7bf2a70a4ffd976d50638d3b9f2ec409763157df.
* nix/libstore/build.cc (sameOperatingSystemKernel): New function.
(DerivationGoal::runChild): Call 'execve' only when
'sameOperatingSystemKernel' returns true.
| Ludovic Courtès |
| 2020-09-14 | daemon: Move 'Agent' to libutil....* nix/libstore/build.cc (DerivationGoal::tryBuildHook): Add "offload" to
'args' and pass settings.guixProgram as the first argument to
Agent::Agent.
(pathNullDevice, commonChildInit, Agent, Agent::Agent)
(Agent::~Agent): Move to...
* nix/libutil/util.cc: ... here.
* nix/libutil/util.hh (struct Agent, commonChildInit): New
declarations.
| Ludovic Courtès |
| 2020-09-14 | daemon: Generalize 'HookInstance' to 'Agent'....* nix/libstore/build.cc (HookInstance): Rename to...
(Agent): ... this. Rename 'toHook' and 'fromHook' similarly and update
users. Change constructor to require a command and an argument list.
(DerivationGoal::tryBuildHook): Pass arguments to the 'Agent'
constructor.
| Ludovic Courtès |
| 2020-03-26 | daemon: Do not use clone on the Hurd....Checking for CLONE_NEWNS is only needed for using tha Linux specific clone(2),
otherwise we can use fork(2). Using clone on the Hurd needs some work, only
support LINUX for now. See
https://lists.gnu.org/archive/html/guix-devel/2020-03/msg00190.html
* nix/libstore/build.cc (CHROOT_ENABLED): Break into CHROOT_ENABLED
and CLONE_ENABLED.
(DerivationGoal::startBuilder): Replace CHROOT_ENABLED with __linux__.
(DerivationGoal::runChild): Only define pivot_root() if SYS_pivot_root is
defined.
Co-authored-by: Jan Nieuwenhuizen <janneke@gnu.org>
| Manolis Ragkousis |
| 2019-11-13 | daemon: Don't include <linux/fs.h>....As of GNU libc 2.29, <sys/mount.h> declares all the constants and
functions we need, so there's no use in including <linux/fs.h> anymore.
This silences annoying warnings like this one:
In file included from nix/libstore/local-store.cc:32:0:
/gnu/store/…-linux-libre-headers-4.19.56/include/linux/fs.h:108:0: warning: "MS_RDONLY" redefined
#define MS_RDONLY 1 /* Mount read-only */
In file included from nix/libstore/local-store.cc:28:0:
/gnu/store/…-glibc-2.29/include/sys/mount.h:36:0: note: this is the location of the previous definition
#define MS_RDONLY MS_RDONLY
* config-daemon.ac: Remove check for <linux/fs.h>.
* nix/libstore/build.cc: Remove conditional inclusion of <linux/fs.h>.
* nix/libstore/local-store.cc: Remove "#if HAVE_LINUX_FS_H" and
inclusion of <linux/fs.h>.
| Ludovic Courtès |
| 2019-11-04 | daemon: Unregister build hook from the worker's children upon build failure....Fixes <https://bugs.gnu.org/38062>.
This is a followup to ada9a19a2dca74feafcf24df1152abd685d4142f.
* nix/libstore/build.cc (DerivationGoal::killChild): Add conditional
call to 'worker.childTerminated' for 'hook->pid'.
| Ludovic Courtès |
| 2019-09-28 | daemon: Strictly respect timeouts for 'guix offload'....Until now it was up to 'guix offload' to honor timeouts. Unfortunately
it would sometimes fail to do that, for example due to the libssh bug at
<https://bugs.libssh.org/T33>. With this change, 'guix offload' is
automatically killed by the daemon when one of the timeouts expires.
Thus, data transfers performed by 'guix offload' now count as part of
the timeouts, rather than just actual build time.
* nix/libstore/build.cc (DerivationGoal::tryBuildHook): Pass true as the
'respectTimeouts' argument to 'childStarted'.
| Ludovic Courtès |
| 2019-09-08 | daemon: Run 'guix substitute' directly and assume a single substituter....The daemon had a mechanism that allows it to handle a list of
substituters and try them sequentially; this removes it.
* nix/scripts/substitute.in: Remove.
* nix/local.mk (nodist_pkglibexec_SCRIPTS): Remove.
* config-daemon.ac: Don't output 'nix/scripts/substitute'.
* nix/libstore/build.cc (SubstitutionGoal)[subs, sub, hasSubstitute]:
Remove.
[tryNext]: Make private.
(SubstitutionGoal::SubstitutionGoal, SubstitutionGoal::init): Remove now
unneeded initializers.
(SubstitutionGoal::tryNext): Adjust to assume a single substituter: call
'amDone' upfront when we couldn't find substitutes.
(SubstitutionGoal::tryToRun): Adjust to run 'guix substitute' via
'settings.guixProgram'.
(SubstitutionGoal::finished): Call 'amDone(ecFailed)' upon failure
instead of setting 'state' to 'tryNext'.
* nix/libstore/globals.hh (Settings)[substituters]: Remove.
* nix/libstore/local-store.cc (LocalStore::~LocalStore): Adjust to
handle a single substituter.
(LocalStore::startSubstituter): Remove 'path' parameter. Adjust to
invoke 'settings.guixProgram'. Don't refer to 'run.program', which no
longer exists.
(LocalStore::querySubstitutablePaths): Adjust for 'runningSubstituters'
being a singleton instead of a list.
(LocalStore::querySubstitutablePathInfos): Likewise, and remove
'substituter' parameter.
* nix/libstore/local-store.hh (RunningSubstituter)[program]: Remove.
(LocalStore)[runningSubstituters]: Remove.
[runningSubstituter]: New field.
[querySubstitutablePathInfos]: Remove 'substituter' parameter.
[startSubstituter]: Remove 'substituter' parameter.
* nix/nix-daemon/guix-daemon.cc (main): Remove references to
'settings.substituters'.
* nix/nix-daemon/nix-daemon.cc (performOp): Ignore the user's
"build-use-substitutes" value when 'settings.useSubstitutes' is false.
| Ludovic Courtès |
| 2019-09-08 | daemon: Run 'guix offload' directly....* nix/scripts/offload.in: Remove.
* nix/local.mk (nodist_pkglibexec_SCRIPTS) [BUILD_DAEMON_OFFLOAD]:
Remove 'scripts/offload'.
* config-daemon.ac: Don't output 'nix/scripts/offload'.
* build-aux/pre-inst-env.in: Don't set 'NIX_BUILD_HOOK'.
* nix/libstore/build.cc (HookInstance::HookInstance): Run 'guix
offload'.
(DerivationGoal::tryBuildHook): Remove reference to 'NIX_BUILD_HOOK'.
* nix/nix-daemon/guix-daemon.cc (main) [HAVE_DAEMON_OFFLOAD_HOOK]: Don't
set 'NIX_BUILD_HOOK'.
* nix/nix-daemon/nix-daemon.cc (performOp) [!HAVE_DAEMON_OFFLOAD_HOOK]:
Leave 'settings.useBuildHook' unchanged.
| Ludovic Courtès |
| 2019-06-13 | daemon: Replace "illegal" by "invalid" in error messages....* nix/libstore/build.cc (parseReferenceSpecifiers): Replace "illegal" by
"invalid".
* nix/libstore/globals.cc (Settings::pack): Likewise.
* nix/libstore/store-api.cc (checkStoreName): Likewise.
| Ludovic Courtès |
| 2019-02-06 | daemon: Emit a 'build-succeeded' event in check mode....Until now, something like "guix build sed -v1 --check" would not get a
'build-succeeded' event, which in turn meant that the spinner would not
be erased upon build completion.
* nix/libstore/build.cc (DerivationGoal::registerOutputs): When
'buildMode' is bmCheck and 'settings.printBuildTrace' emit a
"@ build-succeeded" trace upon success.
* tests/store.scm ("build-succeeded trace in check mode"): New test.
| Ludovic Courtès |
| 2018-12-16 | daemon: Use unbranded phrases in comments and messages....* nix/libstore/build.cc, nix/libstore/globals.cc,
nix/libstore/gc.cc, nix/libstore/local-store.cc,
nix/libstore/optimise-store.cc, nix/libstore/store-api.cc,
nix/libutil/archive.cc, nix/nix-daemon/nix-daemon.cc: Replace "Nix
store" by "store", and "Nix daemon" by "build daemon".
| Ludovic Courtès |
| 2018-10-15 | daemon: Support multiplexed build output....This allows clients to tell whether output comes from the daemon or, if
it comes from a builder, from which builder it comes. The latter is
particularly useful when MAX-BUILD-JOBS > 1.
* nix/libstore/build.cc (DerivationGoal::tryBuildHook)
(DerivationGoal::startBuilder): Print the child's PID in "@ build-started"
traces.
(DerivationGoal::handleChildOutput): Define 'prefix', pass it to
'writeToStderr'.
* nix/libstore/globals.cc (Settings:Settings): Initialize
'multiplexedBuildOutput'.
(Settings::update): Likewise.
* nix/libstore/globals.hh (Settings)[multiplexedBuildOutput]: New field.
Update 'printBuildTrace' documentation.
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0.163.
* nix/nix-daemon/nix-daemon.cc (performOp) <wopSetOptions>: Special-case
"multiplexed-build-output" and remove "use-ssh-substituter".
* guix/store.scm (set-build-options): Add #:multiplexed-build-output?
and honor it.
(%protocol-version): Bump to #x163.
* tests/store.scm ("multiplexed-build-output"): New test.
fixlet
| Ludovic Courtès |
| 2018-09-27 | Add (guix status) and use it for pretty colored output....* guix/progress.scm (progress-reporter/trace): New procedure.
(%progress-interval): New variable.
(progress-reporter/file): Use it.
* guix/scripts/build.scm (set-build-options-from-command-line): Pass
#:print-extended-build-trace?.
(%default-options): Add 'print-extended-build-trace?'.
(guix-build): Parameterize CURRENT-TERMINAL-COLUMNS. Use
'build-status-updater'.
* guix/scripts/environment.scm (%default-options): Add
'print-extended-build-trace?'.
(guix-environment): Wrap body in 'with-status-report'.
* guix/scripts/pack.scm (%default-options): Add 'print-build-trace?' and
'print-extended-build-trace?'.
(guix-pack): Wrap body in 'with-status-report'.
* guix/scripts/package.scm (%default-options, guix-package): Likewise.
* guix/scripts/system.scm (%default-options, guix-system): Likewise.
* guix/scripts/pull.scm (%default-options, guix-pull): Likewise.
* guix/scripts/substitute.scm (progress-report-port): Don't call STOP
when TOTAL is zero.
(process-substitution): Add #:print-build-trace? and honor it.
(guix-substitute)[print-build-trace?]: New variable.
Pass #:print-build-trace? to 'process-substitution'.
* guix/status.scm: New file.
* guix/store.scm (set-build-options): Add #:print-extended-build-trace?;
pass it into PAIRS.
(%protocol-version): Bump.
(protocol-version, nix-server-version): New procedures.
(current-store-protocol-version): New variable.
(with-store, build-things): Parameterize it.
* guix/ui.scm (build-output-port): Remove.
(colorize-string): Export.
* po/guix/POTFILES.in: Add guix/status.scm.
* tests/status.scm: New file.
* Makefile.am (SCM_TESTS): Add it.
* nix/libstore/worker-protocol.hh (PROTOCOL_VERSION): Bump to 0x162.
* nix/libstore/build.cc (DerivationGoal::registerOutputs)
(SubstitutionGoal::finished): Print a "@ hash-mismatch" trace before
throwing.
| Ludovic Courtès |
