;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2013 Andreas Enge ;;; Copyright © 2014, 2015 Mark H Weaver ;;; Copyright © 2016, 2017, 2018 Efraim Flashner ;;; Copyright © 2018 Tobias Geerinckx-Rice ;;; Copyright © 2018 Efraim Flashner ;;; Copyright © 2020 Michael Rohleder ;;; Copyright © 2021 Marius Bakke ;;; ;;; This file is part of GNU Guix. ;;; ;;; GNU Guix is free software; you can redistribute it and/or modify it ;;; under the terms of the GNU General Public License as published by ;;; the Free Software Foundation; either version 3 of the License, or (at ;;; your option) any later version. ;;; ;;; GNU Guix is distributed in the hope that it will be useful, but ;;; WITHOUT ANY WARRANTY; without even the implied warranty of ;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the ;;; GNU General Public License for more details. ;;; ;;; You should have received a copy of the GNU General Public License ;;; along with GNU Guix. If not, see . (define-module (gnu packages file) #:use-module (gnu packages) #:use-module (guix licenses) #:use-module (guix packages) #:use-module (guix download) #:use-module (guix build-system gnu)) (define-public file (package (name "file") (version "5.39") (source (origin (method url-fetch) (uri (string-append "ftp://ftp.astron.com/pub/file/file-" version ".tar.gz")) (sha256 (base32 "1lgs2w2sgamzf27kz5h7pajz7v62554q21fbs11n4mfrfrm2hpgh")))) (build-system gnu-build-system) ;; When cross-compiling, this package depends upon a native install of ;; itself. (native-inputs (if (%current-target-system) `(("self" ,this-package)) '())) (properties `((release-monitoring-url . "http://ftp.astron.com/pub/file/"))) (synopsis "File type guesser") (description "The file command is a file type guesser, a command-line tool that tells you in words what kind of data a file contains. It does not rely on filename extensions to tell you the type of a file, but looks at the actual contents of the file. This package provides the libmagic library.") (license bsd-2) (home-page "https://www.darwinsys.com/file/"))) (define-public file-next (package (inherit file) (version "5.41") (source (origin (method url-fetch) (uri (string-append "http://ftp.astron.com/pub/file/file-" version ".tar.gz")) (sha256 (base32 "0gv027jgdr0hdkw7m9ck0nwhq583f4aa7vnz4dzdbxv4ng3k5r8k")))))) fcd8d7c5ea092e'>deduplicate: Use 'sendfile' for small file copies....* guix/store/deduplication.scm (dump-file/deduplicate): Use 'sendfile' instead of 'dump-port'. * tests/store-deduplication.scm ("copy-file/deduplicate, below %deduplication-minimum-size"): New test. Ludovic Courtès 2021-11-16daemon: Do not deduplicate files smaller than 8 KiB....Files smaller than 8 KiB typically represent ~70% of the entries in /gnu/store/.links but only contribute to ~4% of the space savings afforded by deduplication. Not considering these files for deduplication speeds up file insertion in the store and, more importantly, leaves 'removeUnusedLinks' with fewer entries to traverse, thereby speeding it up proportionally. Partly fixes <https://issues.guix.gnu.org/24937>. * config-daemon.ac: Remove symlink hard link check and CAN_LINK_SYMLINK definition. * guix/store/deduplication.scm (%deduplication-minimum-size): New variable. (deduplicate)[loop]: Do not recurse when FILE's size is below %DEDUPLICATION-MINIMUM-SIZE. (dump-port): New procedure. (dump-file/deduplicate)[hash]: Turn into... [dump-and-compute-hash]: ... this thunk. Call 'deduplicate' only when SIZE is greater than %DEDUPLICATION-MINIMUM-SIZE; otherwise call 'dump-port'. * nix/libstore/gc.cc (LocalStore::removeUnusedLinks): Drop files where st.st_size < deduplicationMinSize. * nix/libstore/local-store.hh (deduplicationMinSize): New declaration. * nix/libstore/optimise-store.cc (deduplicationMinSize): New variable. (LocalStore::optimisePath_): Return when PATH is a symlink or smaller than 'deduplicationMinSize'. * tests/derivations.scm ("identical files are deduplicated"): Produce files bigger than %DEDUPLICATION-MINIMUM-SIZE. * tests/nar.scm ("restore-file-set with directories (signed, valid)"): Likewise. * tests/store-deduplication.scm ("deduplicate, below %deduplication-minimum-size"): New test. ("deduplicate", "deduplicate, ENOSPC"): Produce files bigger than %DEDUPLICATION-MINIMUM-SIZE. * tests/store.scm ("substitute, deduplication"): Likewise. Ludovic Courtès 2020-12-15deduplicate: Create the '.links' directory lazily....This avoids repeated (mkdir-p "/gnu/store/.links") calls when deduplicating lots of files. * guix/store/deduplication.scm (deduplicate): Remove initial call to 'mkdir-p'. Add ENOENT case in 'link' exception handler. Reindent. * tests/store-deduplication.scm ("deduplicate, ENOSPC"): Check for (<= links 4) to account for the initial 'link' call. Ludovic Courtès 2020-12-15store-copy: 'populate-store' can optionally deduplicate files....Until now deduplication was performed as an additional pass after copying files, which involve re-traversing all the files that had just been copied. * guix/store/deduplication.scm (copy-file/deduplicate): New procedure. * tests/store-deduplication.scm ("copy-file/deduplicate"): New test. * guix/build/store-copy.scm (populate-store): Add #:deduplicate? parameter and honor it. * tests/gexp.scm ("gexp->derivation, store copy"): Pass #:deduplicate? #f to 'populate-store'. * gnu/build/image.scm (initialize-root-partition): Pass #:deduplicate? to 'populate-store'. Pass #:deduplicate? #f to 'register-closure'. * gnu/build/vm.scm (root-partition-initializer): Likewise. * gnu/build/install.scm (populate-single-profile-directory): Pass #:deduplicate? #f to 'populate-store'. * gnu/build/linux-initrd.scm (build-initrd): Likewise. * guix/scripts/pack.scm (self-contained-tarball)[import-module?]: New procedure. [build]: Pass it as an argument to 'source-module-closure'. * guix/scripts/pack.scm (squashfs-image)[build]: Wrap in 'with-extensions'. * gnu/system/linux-initrd.scm (expression->initrd)[import-module?]: New procedure. [builder]: Pass it to 'source-module-closure'. * gnu/system/install.scm (cow-store-service-type)[import-module?]: New procedure. Pass it to 'source-module-closure'. Ludovic Courtès