aboutsummaryrefslogtreecommitdiff


 
                         
generated by
                             
                                 cgit
                             
                         

                        
 


AgeCommit message (Expand)Author
2021-11-16daemon: Do not deduplicate files smaller than 8 KiB....Files smaller than 8 KiB typically represent ~70% of the entries in
/gnu/store/.links but only contribute to ~4% of the space savings
afforded by deduplication.

Not considering these files for deduplication speeds up file insertion
in the store and, more importantly, leaves 'removeUnusedLinks' with
fewer entries to traverse, thereby speeding it up proportionally.

Partly fixes <https://issues.guix.gnu.org/24937>.

* config-daemon.ac: Remove symlink hard link check and CAN_LINK_SYMLINK
definition.
* guix/store/deduplication.scm (%deduplication-minimum-size): New
variable.
(deduplicate)[loop]: Do not recurse when FILE's size is below
%DEDUPLICATION-MINIMUM-SIZE.
(dump-port): New procedure.
(dump-file/deduplicate)[hash]: Turn into...
[dump-and-compute-hash]: ... this thunk.
Call 'deduplicate' only when SIZE is greater than
%DEDUPLICATION-MINIMUM-SIZE; otherwise call 'dump-port'.
* nix/libstore/gc.cc (LocalStore::removeUnusedLinks): Drop files where
st.st_size < deduplicationMinSize.
* nix/libstore/local-store.hh (deduplicationMinSize): New declaration.
* nix/libstore/optimise-store.cc (deduplicationMinSize): New variable.
(LocalStore::optimisePath_): Return when PATH is a symlink or smaller
than 'deduplicationMinSize'.
* tests/derivations.scm ("identical files are deduplicated"): Produce
files bigger than %DEDUPLICATION-MINIMUM-SIZE.
* tests/nar.scm ("restore-file-set with directories (signed, valid)"):
Likewise.
* tests/store-deduplication.scm ("deduplicate, below %deduplication-minimum-size"):
New test.
("deduplicate", "deduplicate, ENOSPC"): Produce files bigger than
%DEDUPLICATION-MINIMUM-SIZE.
* tests/store.scm ("substitute, deduplication"): Likewise.
Ludovic Courtès
2021-06-18Start enabling substitutes from bordeaux.guix.gnu.org....In addition to substitutes from ci.guix.gnu.org.  There are more changes that
can be made in the future, but these changes seem like a good start.

* config-daemon.ac (guix_substitute_urls): Add https://bordeaux.guix.gnu.org.
* guix/scripts/substitute.scm (%default-substitute-urls): Add
http://bordeaux.guix.gnu.org.
* guix/store.scm (%default-substitute-urls): Add bordeaux.guix.gnu.org.
* doc/guix.texi: Adjust accordingly.
* doc/contributing.texi: Adjust accordingly.
Christopher Baines
2020-12-11maint: Remove unused 'NIX_VERSION' macro....* config-daemon.ac: Do not define NIX_VERSION.
Ludovic Courtès
2020-12-11maint: Avoid macros obsolete in Autoconf 2.70....* configure.ac: Require Autoconf 2.69.  Use 'AS_HELP_STRING' instead of
'AC_HELP_STRING'.
* m4/guix.m4: Likewise.
* config-daemon.ac: Use 'AC_CONFIG_HEADERS' instead of the singular
variant.
Ludovic Courtès
2019-11-27daemon: GC remove-unused-links phase uses 'statx' when available....* config-daemon.ac: Check for 'statx'.
* nix/libstore/gc.cc (LocalStore::removeUnusedLinks) [HAVE_STATX]: Use
'statx' instead of 'lstat'.
Ludovic Courtès
2019-11-13daemon: Don't include <linux/fs.h>....As of GNU libc 2.29, <sys/mount.h> declares all the constants and
functions we need, so there's no use in including <linux/fs.h> anymore.
This silences annoying warnings like this one:

  In file included from nix/libstore/local-store.cc:32:0:
  /gnu/store/…-linux-libre-headers-4.19.56/include/linux/fs.h:108:0: warning: "MS_RDONLY" redefined
   #define MS_RDONLY  1 /* Mount read-only */

  In file included from nix/libstore/local-store.cc:28:0:
  /gnu/store/…-glibc-2.29/include/sys/mount.h:36:0: note: this is the location of the previous definition
   #define MS_RDONLY MS_RDONLY

* config-daemon.ac: Remove check for <linux/fs.h>.
* nix/libstore/build.cc: Remove conditional inclusion of <linux/fs.h>.
* nix/libstore/local-store.cc: Remove "#if HAVE_LINUX_FS_H" and
inclusion of <linux/fs.h>.
Ludovic Courtès
2019-09-08daemon: Run 'guix substitute' directly and assume a single substituter....The daemon had a mechanism that allows it to handle a list of
substituters and try them sequentially; this removes it.

* nix/scripts/substitute.in: Remove.
* nix/local.mk (nodist_pkglibexec_SCRIPTS): Remove.
* config-daemon.ac: Don't output 'nix/scripts/substitute'.
* nix/libstore/build.cc (SubstitutionGoal)[subs, sub, hasSubstitute]:
Remove.
[tryNext]: Make private.
(SubstitutionGoal::SubstitutionGoal, SubstitutionGoal::init): Remove now
unneeded initializers.
(SubstitutionGoal::tryNext): Adjust to assume a single substituter: call
'amDone' upfront when we couldn't find substitutes.
(SubstitutionGoal::tryToRun): Adjust to run 'guix substitute' via
'settings.guixProgram'.
(SubstitutionGoal::finished): Call 'amDone(ecFailed)' upon failure
instead of setting 'state' to 'tryNext'.
* nix/libstore/globals.hh (Settings)[substituters]: Remove.
* nix/libstore/local-store.cc (LocalStore::~LocalStore): Adjust to
handle a single substituter.
(LocalStore::startSubstituter): Remove 'path' parameter.  Adjust to
invoke 'settings.guixProgram'.  Don't refer to 'run.program', which no
longer exists.
(LocalStore::querySubstitutablePaths): Adjust for 'runningSubstituters'
being a singleton instead of a list.
(LocalStore::querySubstitutablePathInfos): Likewise, and remove
'substituter' parameter.
* nix/libstore/local-store.hh (RunningSubstituter)[program]: Remove.
(LocalStore)[runningSubstituters]: Remove.
[runningSubstituter]: New field.
[querySubstitutablePathInfos]: Remove 'substituter' parameter.
[startSubstituter]: Remove 'substituter' parameter.
* nix/nix-daemon/guix-daemon.cc (main): Remove references to
'settings.substituters'.
* nix/nix-daemon/nix-daemon.cc (performOp): Ignore the user's
"build-use-substitutes" value when 'settings.useSubstitutes' is false.
Ludovic Courtès
2019-09-08daemon: Run 'guix offload' directly....* nix/scripts/offload.in: Remove.
* nix/local.mk (nodist_pkglibexec_SCRIPTS) [BUILD_DAEMON_OFFLOAD]:
Remove 'scripts/offload'.
* config-daemon.ac: Don't output 'nix/scripts/offload'.
* build-aux/pre-inst-env.in: Don't set 'NIX_BUILD_HOOK'.
* nix/libstore/build.cc (HookInstance::HookInstance): Run 'guix
offload'.
(DerivationGoal::tryBuildHook): Remove reference to 'NIX_BUILD_HOOK'.
* nix/nix-daemon/guix-daemon.cc (main) [HAVE_DAEMON_OFFLOAD_HOOK]: Don't
set 'NIX_BUILD_HOOK'.
* nix/nix-daemon/nix-daemon.cc (performOp) [!HAVE_DAEMON_OFFLOAD_HOOK]:
Leave 'settings.useBuildHook' unchanged.
Ludovic Courtès
2019-09-08daemon: Run 'guix perform-download' directly....* nix/scripts/download.in: Remove.
* nix/local.mk (nodist_pkglibexec_SCRIPTS): Remove 'scripts/download'.
* config-daemon.ac: Don't output 'nix/scripts/download'.
* nix/libstore/builtins.cc (builtinDownload): Invoke 'guix
perform-download' directly.
Ludovic Courtès
2019-09-08daemon: Run 'guix authenticate' directly....* nix/scripts/authenticate.in: Remove.
* nix/local.mk (nodist_pkglibexec_SCRIPTS): Remove
scripts/authenticate.
* config-daemon.ac: Don't output 'nix/scripts/authenticate'.
* nix/libstore/local-store.cc (runAuthenticationProgram): Run 'guix
authenticate'.
Ludovic Courtès
2019-09-08daemon: Invoke 'guix gc --list-busy' instead of 'list-runtime-roots'....* nix/scripts/list-runtime-roots.in: Remove.
* guix/store/roots.scm (%proc-directory): New variable.
(proc-file-roots, proc-exe-roots, proc-cwd-roots)
(proc-fd-roots, proc-maps-roots, proc-environ-roots)
(referenced-files, canonicalize-store-item, busy-store-items): New
procedures, taken from 'list-runtime-roots.in'.
* nix/libstore/globals.hh (Settings)[guixProgram]: New field.
* nix/libstore/globals.cc (Settings::processEnvironment): Initialize
'guixProgram'.
* nix/libstore/gc.cc (addAdditionalRoots): Drop code related to
'NIX_ROOT_FINDER'.  Run "guix gc --list-busy".
* nix/local.mk (nodist_pkglibexec_SCRIPTS): Remove
'scripts/list-runtime-roots'.
* config-daemon.ac: Don't output nix/scripts/list-runtime-roots.
* build-aux/pre-inst-env.in: Don't set 'NIX_ROOT_FINDER'.
Set 'GUIX'.
* doc/guix.texi (Invoking guix gc): Document '--list-busy'.
* guix/scripts/gc.scm (show-help, %options): Add "--list-busy".
(guix-gc)[list-busy]: New procedure.
Handle the 'list-busy' action.
Ludovic Courtès
2019-05-01build: Change default substitute server to "ci.guix.gnu.org"....* config-daemon.ac: Replace "ci.guix.info" with "ci.guix.gnu.org".
* doc/guix.texi (SUBSTITUTE-SERVER): Likewise.
* etc/substitutes/ci.guix.gnu.org.pub: New file.
* Makefile.am (dist_pkgdata_DATA): Add it.
* guix/scripts/build.scm (%default-log-urls): Update.
* guix/scripts/substitute.scm (%default-substitute-urls): Likewise.
* guix/store.scm (%default-substitute-urls): Likewise.
* guix/self.scm (miscellaneous-files): Add "ci.guix.gnu.org".
Ludovic Courtès