This patch is needed in order to build Guix on a Btrfs file system. From da922703282b0d3b8837a99a9c7fdd32f1d20d49 Mon Sep 17 00:00:00 2001 From: Mark H Weaver Date: Tue, 9 Jan 2018 20:16:14 -0500 Subject: [PATCH] Remove nonportable check for files containing only zeroes. This check benefitted only one unlikely case (large files containing only zeroes, on systems that do not support SEEK_HOLE) and was based on an assumption about file system behavior that is not mandated by POSIX and no longer holds in practice, namely that for sufficiently large files, (st_blocks == 0) implies that the file contains only zeroes. Examples of file systems that violate this assumption include Linux's /proc file system and Btrfs. * src/sparse.c (sparse_scan_file_wholesparse): Remove this function. (sparse_scan_file_seek): Remove the initial check for files containing only zeroes. --- src/sparse.c | 24 ------------------------ 1 file changed, 24 deletions(-) diff --git a/src/sparse.c b/src/sparse.c index d41c0ea..3de6560 100644 --- a/src/sparse.c +++ b/src/sparse.c @@ -261,26 +261,6 @@ sparse_scan_file_raw (struct tar_sparse_file *file) return tar_sparse_scan (file, scan_end, NULL); } -static bool -sparse_scan_file_wholesparse (struct tar_sparse_file *file) -{ - struct tar_stat_info *st = file->stat_info; - struct sp_array sp = {0, 0}; - - /* Note that this function is called only for truly sparse files of size >= 1 - block size (checked via ST_IS_SPARSE before). See the thread - http://www.mail-archive.com/bug-tar@gnu.org/msg04209.html for more info */ - if (ST_NBLOCKS (st->stat) == 0) - { - st->archive_file_size = 0; - sp.offset = st->stat.st_size; - sparse_add_map (st, &sp); - return true; - } - - return false; -} - #ifdef SEEK_HOLE /* Try to engage SEEK_HOLE/SEEK_DATA feature. */ static bool @@ -343,10 +323,6 @@ sparse_scan_file_seek (struct tar_sparse_file *file) static bool sparse_scan_file (struct tar_sparse_file *file) { - /* always check for completely sparse files */ - if (sparse_scan_file_wholesparse (file)) - return true; - switch (hole_detection) { case HOLE_DETECTION_DEFAULT: -- 2.15.1 ref='/guix/log/nix/libstore?id=e994428066a1eda975b5b6c0c4b54eea18b05caf'>libstore/local-store.hh
AgeCommit message (Expand)Author
2021-11-16daemon: Do not deduplicate files smaller than 8 KiB....Files smaller than 8 KiB typically represent ~70% of the entries in /gnu/store/.links but only contribute to ~4% of the space savings afforded by deduplication. Not considering these files for deduplication speeds up file insertion in the store and, more importantly, leaves 'removeUnusedLinks' with fewer entries to traverse, thereby speeding it up proportionally. Partly fixes <https://issues.guix.gnu.org/24937>. * config-daemon.ac: Remove symlink hard link check and CAN_LINK_SYMLINK definition. * guix/store/deduplication.scm (%deduplication-minimum-size): New variable. (deduplicate)[loop]: Do not recurse when FILE's size is below %DEDUPLICATION-MINIMUM-SIZE. (dump-port): New procedure. (dump-file/deduplicate)[hash]: Turn into... [dump-and-compute-hash]: ... this thunk. Call 'deduplicate' only when SIZE is greater than %DEDUPLICATION-MINIMUM-SIZE; otherwise call 'dump-port'. * nix/libstore/gc.cc (LocalStore::removeUnusedLinks): Drop files where st.st_size < deduplicationMinSize. * nix/libstore/local-store.hh (deduplicationMinSize): New declaration. * nix/libstore/optimise-store.cc (deduplicationMinSize): New variable. (LocalStore::optimisePath_): Return when PATH is a symlink or smaller than 'deduplicationMinSize'. * tests/derivations.scm ("identical files are deduplicated"): Produce files bigger than %DEDUPLICATION-MINIMUM-SIZE. * tests/nar.scm ("restore-file-set with directories (signed, valid)"): Likewise. * tests/store-deduplication.scm ("deduplicate, below %deduplication-minimum-size"): New test. ("deduplicate", "deduplicate, ENOSPC"): Produce files bigger than %DEDUPLICATION-MINIMUM-SIZE. * tests/store.scm ("substitute, deduplication"): Likewise. Ludovic Courtès
2020-12-08daemon: Factorize substituter agent spawning....* nix/libstore/local-store.hh (class LocalStore)[substituter]: New method. [runningSubstituter]: Turn into a shared_ptr. * nix/libstore/local-store.cc (LocalStore::querySubstitutablePaths): Call 'substituter' instead of using inline code. (LocalStore::querySubstitutablePathInfos): Likewise. (LocalStore::substituter): New method. Ludovic Courtès
2020-12-08daemon: Use 'Agent' to spawn 'guix substitute --query'....* nix/libstore/local-store.hh (RunningSubstituter): Remove. (LocalStore)[runningSubstituter]: Change to unique_ptr<Agent>. [setSubstituterEnv, didSetSubstituterEnv]: Remove. [getLineFromSubstituter, getIntLineFromSubstituter]: Take an 'Agent'. * nix/libstore/local-store.cc (LocalStore::~LocalStore): Remove reference to 'runningSubstituter'. (LocalStore::setSubstituterEnv, LocalStore::startSubstituter): Remove. (LocalStore::getLineFromSubstituter): Adjust to 'run' being an 'Agent'. (LocalStore::querySubstitutablePaths): Spawn substituter agent if needed. Adjust to 'Agent' interface. (LocalStore::querySubstitutablePathInfos): Likewise. * nix/libstore/build.cc (SubstitutionGoal::tryToRun): Remove call to 'setSubstituterEnv' and add 'setenv' call for "_NIX_OPTIONS" instead. (SubstitutionGoal::finished): Remove 'readLine' call for 'dummy'. * guix/scripts/substitute.scm (%allow-unauthenticated-substitutes?): Remove second argument to 'make-parameter'. (process-query): Call 'warn-about-missing-authentication' when (%allow-unauthenticated-substitutes?) is #t. (guix-substitute): Wrap body in 'parameterize'. Set 'guix-warning-port' too. No longer exit when 'substitute-urls' returns the empty list. No longer print newline initially. * tests/substitute.scm (test-quit): Parameterize 'current-error-port' to account for the port changes in 'guix-substitute'. Ludovic Courtès