diff options
| -rw-r--r-- | README.md | 184 | ||||
| -rw-r--r-- | hydrilla.scm (renamed from guix.scm) | 43 |
2 files changed, 168 insertions, 59 deletions
@@ -3,74 +3,156 @@ Haketilo is a tool to modify pages being opened in a web browser. It can block pages' scripts and optionally inject user-specified ones. -Haketilo started as a browser extension (a WebExtension) with a dedicated -user scripts repository server, Hydrilla. Haketilo is now being rewritten as an -HTTP proxy. This repository contains combined sources of Hydrilla and Haketilo -which are going to be distributed together. +Haketilo started as a browser extension (a WebExtension) with a dedicated user +scripts repository server, Hydrilla. It has since been rewritten as an HTTP +proxy. This repository contains the Python code of Hydrilla and Haketilo. -This source tree is a work in progress. The server part has simlar functionality -as in earlier Hydrilla releases and should be mostly usable at this point. The -proxy part is still being developed, but you can play with it nevertheless if -you're brave 😉 +## Getting started -You're going to need at least Python 3.7 plus the following Python libraries -* mitmproxy 8 (sadly, not in Debian yet) -* immutables (also not in Debian but if you are familiar with git-buildpackage, - you can utilize [my own packaging of it](https://git.koszko.org/immutables/)) -* flask -* jsonschema (>= 3.0) -* beautifulsoup4 -* html5lib -* setuptools -* setuptools_scm -* wheel -* babel (Python library) -* build (a PEP517 package builder) +At the moment the recommended method of using Haketilo and Hydrilla is through +the [GNU Guix](https://guix.gnu.org/) package manager. Installation from Python +wheel is also possible but not described here. -This project also utilizes reuse, pytest and mypy but those are either -development-only dependencies or just not required for the actual Haketilo -proxy. +Build instructions have been most recently tested with Guix version +1.3.0-26.fd00ac7. + +### Building locally (from release tarball) + +Assuming GNU Guix is already installed and working, we can execute the following +command in the project root to spawn a shell with Hydrilla & Haketilo available +inside. -Make sure to run ``` -./setup.py build +guix environment -L . --ad-hoc -e '(@ (hydrilla) hydrilla)' ``` -before running anything. Otherwise, missing files can make things break. -Create directory ~/.haketilo/store/temporary_malcontent/ and put there all the -[built Haketilo packages](https://hydrillabugs.koszko.org/projects/hydrilla/wiki/Hydrilla_on-disk_data_format) that you'd like to use. You can use -[hydrilla builder](https://hydrillabugs.koszko.org/projects/hydrilla/wiki/Releases) -and the [repository of fixes for JS-encumbered websites](https://git.koszko.org/hydrilla-fixes-bundle/). +We can then make a test by running one of the programs, e.g. +``` +haketilo --version +``` + +To instead install Haketilo & Hydrilla into the current Guix profile, treat your +terminal with the following incantation -Now, run ``` -PYTHONPATH=./src/ python3 -m mitmproxy_launcher +guix package -L . -e '(@ (hydrilla) hydrilla)' ``` -Next, open your browser, go to settings and import the certificate that got -generated under ~/.haketilo/mitmproxy/mitmproxy-ca-cert.pem. +### Building locally (from git checkout) -Under (at least) Mozilla browsers, certificate pinning may cause mitmproxy's -generated certificates to still be rejected. This generally happens on Google -and Mozilla websites. To be able to access these through proxy, go to -about:config and set the security.cert_pinning.enforcement_level preference to -0. +Due to some nuances of the setuptools-scm tool we're using, it is necessary to +generate a project source tarball under `dist/` before building the Guix +package. -You can now browse to some site for which you configured a Haketilo user script -to run. I usually visit https://opencores.org/projects for which the first -Haketilo script was written... +``` +guix environment -L . -e '(@ (hydrilla) hydrilla)' -- python3 -m build -s +``` -There's some work to do before any of this will be usable to normal human -beings. Help appreciated. +After that, we can start a shell with Hydrilla & Haketilo installed -## Copying +``` +guix environment -L . --ad-hoc -e '(@ (hydrilla) hydrilla-dist-tarball)' +``` -Hydrilla is Copyright (C) 2021-2022 Wojtek Kosior and contributors, entirely available under the GNU Affero General Public License version 3 or later. Some files might also give you broader permissions, see comments inside them. +or just install to Guix profile -*I, Wojtek Kosior, thereby promise not to sue for violation of this project's license. Although I request that you do not make use of this code in a proprietary program, I am not going to enforce this in court.* +``` +guix package -L . -e '(@ (hydrilla) hydrilla-dist-tarball)' +``` + +or build a binary package suitable for distribution to other GNU/Linux users + +``` +guix pack -L . -RR \ + -S /hydrilla=bin/hydrilla \ + -S /hydrilla-builder=bin/hydrilla-builder \ + -S /hydrilla-server=bin/hydrilla-server \ + -S /haketilo=bin/haketilo \ + -e '(@ (hydrilla) hydrilla-dist-tarball)' +``` + +### Running from source + +During development, it is convenient to run the tools being worked on without +putting them in a package. To spawn a shell with all development dependencies +installed, run + +``` +guix environment -L . -e '(@ (hydrilla) hydrilla)' +``` + +For software to run, we first need to compile message catalogs and make sure the +relevant metadata has been extracted from git by setuptools-scm. Inside the +shell we just spawned, we run + +``` +# The following command, besides building a source tarball, generates the +# src/hydrilla/_version.py file we need. +python3 -m build -s +# Generate .mo file(s) for gettext. +./setup.py compile_catalog +``` + +Tools can be manually tested by telling Python interpreter tu execute the +relevant module, e.g -## Contributing +``` +PYTHONPATH=./src/ python3 -m hydrilla.mitmproxy_launcher --version +``` + +#### Running tests + +Hydrilla uses pytest. Tests can be run with + +``` +pytest +``` + +Please refer to the +[pytest documentation](https://docs.pytest.org/en/stable/how-to/usage.html) for +more details. + +#### Working on message catalogs + +There are 3 commands we'll want to use. + +``` +# Generate a message catalog template (src/hydrilla/locales/messages.pot) +./setup.py extract_messages +# Merge the generated template into existing .po catalog file(s) +./setup.py update_catalog +# Generate .mo file(s) that will be loaded by gettext +./setup.py compile_catalog +``` + +Please refer to the +[Babel documentation](https://babel.pocoo.org/en/latest/messages.html#message-extraction) +for more details. -Please visit our Redmine instance at https://hydrillabugs.koszko.org. +#### Exiting Guix environment -You can also write an email to koszko@koszko.org. +Once we're done hacking on the project, we can type + +``` +exit +``` + +in the shell... or just hit `Ctrl+d`. + +### User documentation + +Please look at +[our wiki](https://hydrillabugs.koszko.org/projects/haketilo/wiki) for +instructions on how to operate Haketilo and Hydrilla. + +## Contributing, asking for help, giving feedback, reporting bugs + +Development occurs on +[our issue tracker](https://hydrillabugs.koszko.org/projects/haketilo). You can +also write directly to [Wojtek](mailto:koszko@koszko.org) if you prefer. + +## Copying + +Hydrilla is Copyright (C) 2021-2022 Wojtek Kosior and contributors, entirely available under the GNU Affero General Public License version 3 or later. Some files might also give you broader permissions, see comments inside them. + +*I, Wojtek Kosior, thereby promise not to sue for violation of this project's license. Although I request that you do not make use of this code in a proprietary program, I am not going to enforce this in court.* @@ -4,6 +4,28 @@ ;; ;; Available under the terms of Creative Commons Zero v1.0 Universal. +(define-module (hydrilla)) + +(use-modules + (ice-9 rdelim) + (ice-9 regex)) + +(define %source-dir (dirname (current-filename))) + +;; The PKG-INFO file is generated when running `python3 -m build -s` or similar. +;; It is also automatically included in the generated tarballs. +(define %hydrilla-version + (if (access? "src/hydrilla.egg-info/PKG-INFO" R_OK) + (let* ((port (open-input-file "src/hydrilla.egg-info/PKG-INFO")) + (process-line + (lambda (self-ref) + (let ((match-result + (string-match "^Version: (.*)" (read-line port)))) + (if match-result (match:substring match-result 1) + (self-ref self-ref)))))) + (process-line process-line)) + "unknown")) + (use-modules (guix packages) (guix download) @@ -26,8 +48,6 @@ (gnu packages license) (gnu packages gnupg)) -(define %source-dir (dirname (current-filename))) - (define-public python-kaitaistruct (package (name "python-kaitaistruct") @@ -261,14 +281,12 @@ SSL/TLS-protected protocols.") static types.") (license license:asl2.0))) +;; Use this variant when building from a downloaded release tarball. (define-public hydrilla (package (name "hydrilla") - ;; Change the following to the version for which you have generated a - ;; tarball under dist/. - (version "3.0b0") - (source (local-file - (string-append %source-dir "/dist/hydrilla-" version ".tar.gz"))) + (version %hydrilla-version) + (source (local-file %source-dir #:recursive? #t)) (build-system python-build-system) (arguments `(#:phases @@ -303,4 +321,13 @@ combines the functionalities of content blocker and user script manager. It can be used with its script repository, Hydrilla.") (license (list license:agpl3+ license:gpl3+ license:cc0)))) -hydrilla +;; Use this variant when building from a tarball generated under dist/. This +;; can be used to build from a git checkout after running `python3 -m build -s` +;; or similar. +(define-public hydrilla-dist-tarball + (let ((base hydrilla) + (filename (string-append "hydrilla-" %hydrilla-version ".tar.gz"))) + (package + (inherit base) + (source (local-file + (string-append %source-dir "/dist/" filename)))))) |