From 3cc41c05fad5601c0dd1832f64a6e9efca017727 Mon Sep 17 00:00:00 2001 From: Maxim Cournoyer Date: Mon, 1 Apr 2019 11:36:04 -0400 Subject: [PATCH] robottime: Honor the SOURCE_DATE_EPOCH environment variable. Honoring the SOURCE_DATE_EPOCH environment variable allows building the documentation using libdoc reproducibly, by setting the generated timestamp to a fixed value. For more background on reproducible builds and the SOURCE_DATE_EPOCH environment variable, see: https://reproducible-builds.org/specs/source-date-epoch/. * src/robot/utils/robottime.py: import `os'. (TimestampCache._get_epoch): Retrieve date from SOURCE_DATE_EPOCH if it is defined, otherwise from time.time(). * utest/output/test_logger.py (TestLogger.test_write_to_one_logger): Check for the existance of a timestamp attribute instead of checking for its content as the later is easy to break when using the SOURCE_DATE_EPOCH environment variable. --- src/robot/utils/robottime.py | 3 +++ utest/output/test_logger.py | 2 +- 2 files changed, 4 insertions(+), 1 deletion(-) diff --git a/src/robot/utils/robottime.py b/src/robot/utils/robottime.py index 06432a4a6..91526f826 100644 --- a/src/robot/utils/robottime.py +++ b/src/robot/utils/robottime.py @@ -14,6 +14,7 @@ # limitations under the License. import datetime +import os import time import re @@ -395,6 +396,8 @@ class TimestampCache(object): # Seam for mocking def _get_epoch(self): + if os.getenv('SOURCE_DATE_EPOCH'): + return float(os.getenv('SOURCE_DATE_EPOCH')) return time.time() def _use_cache(self, secs, *separators): diff --git a/utest/output/test_logger.py b/utest/output/test_logger.py index 92fe6d77d..e980227aa 100644 --- a/utest/output/test_logger.py +++ b/utest/output/test_logger.py @@ -46,7 +46,7 @@ class TestLogger(unittest.TestCase): logger = LoggerMock(('Hello, world!', 'INFO')) self.logger.register_logger(logger) self.logger.write('Hello, world!', 'INFO') - assert_true(logger.msg.timestamp.startswith('20')) + assert_true(hasattr(logger.msg, 'timestamp')) def test_write_to_one_logger_with_trace_level(self): logger = LoggerMock(('expected message', 'TRACE')) -- 2.20.1 /bdw-gc.scm?id=2dfb16150e11f273fd6f991bb563bf02a8a69402'>bdw-gc.scm
te files smaller than 8 KiB....Files smaller than 8 KiB typically represent ~70% of the entries in /gnu/store/.links but only contribute to ~4% of the space savings afforded by deduplication. Not considering these files for deduplication speeds up file insertion in the store and, more importantly, leaves 'removeUnusedLinks' with fewer entries to traverse, thereby speeding it up proportionally. Partly fixes <https://issues.guix.gnu.org/24937>. * config-daemon.ac: Remove symlink hard link check and CAN_LINK_SYMLINK definition. * guix/store/deduplication.scm (%deduplication-minimum-size): New variable. (deduplicate)[loop]: Do not recurse when FILE's size is below %DEDUPLICATION-MINIMUM-SIZE. (dump-port): New procedure. (dump-file/deduplicate)[hash]: Turn into... [dump-and-compute-hash]: ... this thunk. Call 'deduplicate' only when SIZE is greater than %DEDUPLICATION-MINIMUM-SIZE; otherwise call 'dump-port'. * nix/libstore/gc.cc (LocalStore::removeUnusedLinks): Drop files where st.st_size < deduplicationMinSize. * nix/libstore/local-store.hh (deduplicationMinSize): New declaration. * nix/libstore/optimise-store.cc (deduplicationMinSize): New variable. (LocalStore::optimisePath_): Return when PATH is a symlink or smaller than 'deduplicationMinSize'. * tests/derivations.scm ("identical files are deduplicated"): Produce files bigger than %DEDUPLICATION-MINIMUM-SIZE. * tests/nar.scm ("restore-file-set with directories (signed, valid)"): Likewise. * tests/store-deduplication.scm ("deduplicate, below %deduplication-minimum-size"): New test. ("deduplicate", "deduplicate, ENOSPC"): Produce files bigger than %DEDUPLICATION-MINIMUM-SIZE. * tests/store.scm ("substitute, deduplication"): Likewise.
AgeCommit message (Expand)Author
Ludovic Courtès
2020-12-15nar: Deduplicate files right as they are restored....This avoids having to traverse and re-read the files that we have just restored, thereby reducing I/O. * guix/serialization.scm (dump-file): New procedure. (restore-file): Add #:dump-file parameter and honor it. * guix/store/deduplication.scm (tee, dump-file/deduplicate): New procedures. * guix/nar.scm (restore-one-item): Pass #:dump-file to 'restore-file'. (finalize-store-file): Pass #:deduplicate? #f to 'register-items'. * tests/nar.scm <top level>: Call 'setenv' to set "NIX_STORE". Ludovic Courtès
2020-12-15serialization: 'restore-file' sets canonical timestamp and permissions....* guix/serialization.scm (restore-file): Set the permissions and mtime of FILE. * guix/nar.scm (finalize-store-file): Pass #:reset-timestamps? #f to 'register-items'. * tests/nar.scm (rm-rf): Add 'chmod' calls to ensure files are writable. ("write-file + restore-file with symlinks"): Ensure every file in OUTPUT passes 'canonical-file?'. * tests/guix-archive.sh: Run "chmod -R +w" before "rm -rf". Ludovic Courtès
2020-12-15serialization: 'fold-archive' notifies about directory processing completion....* guix/serialization.scm (fold-archive): Call PROC with a 'directory-complete tag when done with a directory. (restore-file): Handle it. * guix/scripts/archive.scm (list-contents): Likewise. * guix/scripts/challenge.scm (archive-contents): Likewise. * tests/nar.scm ("write-file-tree + fold-archive"): Adjust accordingly. Ludovic Courtès