From b59e421b5fdd3a7728212b6b09adcaa40709fe73 Mon Sep 17 00:00:00 2001 From: Wojtek Kosior Date: Wed, 12 Oct 2022 08:26:16 +0200 Subject: [builder][server][proxy] replace guix.scm with a loadable scheme module; document Haketilo+Guix usage in the README --- README.md | 184 ++++++++++++++++++++++++--------- guix.scm | 306 ------------------------------------------------------ hydrilla.scm | 333 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 466 insertions(+), 357 deletions(-) delete mode 100644 guix.scm create mode 100644 hydrilla.scm diff --git a/README.md b/README.md index ad3434b..fc1bda1 100644 --- a/README.md +++ b/README.md @@ -3,74 +3,156 @@ Haketilo is a tool to modify pages being opened in a web browser. It can block pages' scripts and optionally inject user-specified ones. -Haketilo started as a browser extension (a WebExtension) with a dedicated -user scripts repository server, Hydrilla. Haketilo is now being rewritten as an -HTTP proxy. This repository contains combined sources of Hydrilla and Haketilo -which are going to be distributed together. +Haketilo started as a browser extension (a WebExtension) with a dedicated user +scripts repository server, Hydrilla. It has since been rewritten as an HTTP +proxy. This repository contains the Python code of Hydrilla and Haketilo. -This source tree is a work in progress. The server part has simlar functionality -as in earlier Hydrilla releases and should be mostly usable at this point. The -proxy part is still being developed, but you can play with it nevertheless if -you're brave 😉 +## Getting started -You're going to need at least Python 3.7 plus the following Python libraries -* mitmproxy 8 (sadly, not in Debian yet) -* immutables (also not in Debian but if you are familiar with git-buildpackage, - you can utilize [my own packaging of it](https://git.koszko.org/immutables/)) -* flask -* jsonschema (>= 3.0) -* beautifulsoup4 -* html5lib -* setuptools -* setuptools_scm -* wheel -* babel (Python library) -* build (a PEP517 package builder) +At the moment the recommended method of using Haketilo and Hydrilla is through +the [GNU Guix](https://guix.gnu.org/) package manager. Installation from Python +wheel is also possible but not described here. -This project also utilizes reuse, pytest and mypy but those are either -development-only dependencies or just not required for the actual Haketilo -proxy. +Build instructions have been most recently tested with Guix version +1.3.0-26.fd00ac7. + +### Building locally (from release tarball) + +Assuming GNU Guix is already installed and working, we can execute the following +command in the project root to spawn a shell with Hydrilla & Haketilo available +inside. -Make sure to run ``` -./setup.py build +guix environment -L . --ad-hoc -e '(@ (hydrilla) hydrilla)' ``` -before running anything. Otherwise, missing files can make things break. -Create directory ~/.haketilo/store/temporary_malcontent/ and put there all the -[built Haketilo packages](https://hydrillabugs.koszko.org/projects/hydrilla/wiki/Hydrilla_on-disk_data_format) that you'd like to use. You can use -[hydrilla builder](https://hydrillabugs.koszko.org/projects/hydrilla/wiki/Releases) -and the [repository of fixes for JS-encumbered websites](https://git.koszko.org/hydrilla-fixes-bundle/). +We can then make a test by running one of the programs, e.g. +``` +haketilo --version +``` + +To instead install Haketilo & Hydrilla into the current Guix profile, treat your +terminal with the following incantation -Now, run ``` -PYTHONPATH=./src/ python3 -m mitmproxy_launcher +guix package -L . -e '(@ (hydrilla) hydrilla)' ``` -Next, open your browser, go to settings and import the certificate that got -generated under ~/.haketilo/mitmproxy/mitmproxy-ca-cert.pem. +### Building locally (from git checkout) -Under (at least) Mozilla browsers, certificate pinning may cause mitmproxy's -generated certificates to still be rejected. This generally happens on Google -and Mozilla websites. To be able to access these through proxy, go to -about:config and set the security.cert_pinning.enforcement_level preference to -0. +Due to some nuances of the setuptools-scm tool we're using, it is necessary to +generate a project source tarball under `dist/` before building the Guix +package. -You can now browse to some site for which you configured a Haketilo user script -to run. I usually visit https://opencores.org/projects for which the first -Haketilo script was written... +``` +guix environment -L . -e '(@ (hydrilla) hydrilla)' -- python3 -m build -s +``` -There's some work to do before any of this will be usable to normal human -beings. Help appreciated. +After that, we can start a shell with Hydrilla & Haketilo installed -## Copying +``` +guix environment -L . --ad-hoc -e '(@ (hydrilla) hydrilla-dist-tarball)' +``` -Hydrilla is Copyright (C) 2021-2022 Wojtek Kosior and contributors, entirely available under the GNU Affero General Public License version 3 or later. Some files might also give you broader permissions, see comments inside them. +or just install to Guix profile -*I, Wojtek Kosior, thereby promise not to sue for violation of this project's license. Although I request that you do not make use of this code in a proprietary program, I am not going to enforce this in court.* +``` +guix package -L . -e '(@ (hydrilla) hydrilla-dist-tarball)' +``` + +or build a binary package suitable for distribution to other GNU/Linux users + +``` +guix pack -L . -RR \ + -S /hydrilla=bin/hydrilla \ + -S /hydrilla-builder=bin/hydrilla-builder \ + -S /hydrilla-server=bin/hydrilla-server \ + -S /haketilo=bin/haketilo \ + -e '(@ (hydrilla) hydrilla-dist-tarball)' +``` + +### Running from source + +During development, it is convenient to run the tools being worked on without +putting them in a package. To spawn a shell with all development dependencies +installed, run + +``` +guix environment -L . -e '(@ (hydrilla) hydrilla)' +``` + +For software to run, we first need to compile message catalogs and make sure the +relevant metadata has been extracted from git by setuptools-scm. Inside the +shell we just spawned, we run + +``` +# The following command, besides building a source tarball, generates the +# src/hydrilla/_version.py file we need. +python3 -m build -s +# Generate .mo file(s) for gettext. +./setup.py compile_catalog +``` + +Tools can be manually tested by telling Python interpreter tu execute the +relevant module, e.g -## Contributing +``` +PYTHONPATH=./src/ python3 -m hydrilla.mitmproxy_launcher --version +``` + +#### Running tests + +Hydrilla uses pytest. Tests can be run with + +``` +pytest +``` + +Please refer to the +[pytest documentation](https://docs.pytest.org/en/stable/how-to/usage.html) for +more details. + +#### Working on message catalogs + +There are 3 commands we'll want to use. + +``` +# Generate a message catalog template (src/hydrilla/locales/messages.pot) +./setup.py extract_messages +# Merge the generated template into existing .po catalog file(s) +./setup.py update_catalog +# Generate .mo file(s) that will be loaded by gettext +./setup.py compile_catalog +``` + +Please refer to the +[Babel documentation](https://babel.pocoo.org/en/latest/messages.html#message-extraction) +for more details. -Please visit our Redmine instance at https://hydrillabugs.koszko.org. +#### Exiting Guix environment -You can also write an email to koszko@koszko.org. +Once we're done hacking on the project, we can type + +``` +exit +``` + +in the shell... or just hit `Ctrl+d`. + +### User documentation + +Please look at +[our wiki](https://hydrillabugs.koszko.org/projects/haketilo/wiki) for +instructions on how to operate Haketilo and Hydrilla. + +## Contributing, asking for help, giving feedback, reporting bugs + +Development occurs on +[our issue tracker](https://hydrillabugs.koszko.org/projects/haketilo). You can +also write directly to [Wojtek](mailto:koszko@koszko.org) if you prefer. + +## Copying + +Hydrilla is Copyright (C) 2021-2022 Wojtek Kosior and contributors, entirely available under the GNU Affero General Public License version 3 or later. Some files might also give you broader permissions, see comments inside them. + +*I, Wojtek Kosior, thereby promise not to sue for violation of this project's license. Although I request that you do not make use of this code in a proprietary program, I am not going to enforce this in court.* diff --git a/guix.scm b/guix.scm deleted file mode 100644 index 308596a..0000000 --- a/guix.scm +++ /dev/null @@ -1,306 +0,0 @@ -;; SPDX-License-Identifier: CC0-1.0 - -;; Copyright (C) 2022 Wojtek Kosior -;; -;; Available under the terms of Creative Commons Zero v1.0 Universal. - -(use-modules - (guix packages) - (guix download) - (guix git-download) - (guix build-system python) - (guix gexp) - ((guix licenses) #:prefix license:) - (gnu packages python-build) - (gnu packages python-xyz) - (gnu packages python-crypto) - (gnu packages compression) - (gnu packages python-compression) - (gnu packages xdisorg) - (gnu packages serialization) - (gnu packages protobuf) - (gnu packages python-web) - (gnu packages check) - (gnu packages sphinx) - (gnu packages python-check) - (gnu packages license) - (gnu packages gnupg)) - -(define %source-dir (dirname (current-filename))) - -(define-public python-kaitaistruct - (package - (name "python-kaitaistruct") - (version "0.10") - (source - (origin - (method url-fetch) - (uri (pypi-uri "kaitaistruct" version)) - (sha256 - (base32 "0ap5ka51gnc2mc4s1kqqsi6nb6zqv8wsrg17ryxazmkkj7idwi50")))) - (build-system python-build-system) - (home-page "https://kaitai.io") - (native-inputs (list python-wheel)) - (synopsis - "Declarative parser generator for binary data: runtime library for Python") - (description - "Kaitai Struct is a declarative language used for describing various binary -data structures, laid out in files or in memory - i.e. binary file formats, -network stream packet formats, etc.") - (license license:expat))) - -(define-public python-parver - (package - (name "python-parver") - (version "0.3.1") - (source - (origin - (method url-fetch) - (uri (pypi-uri "parver" version)) - (sha256 - (base32 "1lyzqp8bz0n2kzabzl7k7g7cn90rlnrxjzva2p62gsfc7djy00n9")))) - (build-system python-build-system) - (arguments - `(#:phases - (modify-phases %standard-phases - (add-after 'unpack 'relax-requirements - (lambda _ - (substitute* "setup.py" - (("arpeggio[^']*") "arpeggio")))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest"))))))) - (propagated-inputs (list python-arpeggio python-attrs python-six)) - (native-inputs - (list python-hypothesis - python-pretend - python-pytest)) - (home-page "https://github.com/RazerM/parver") - (synopsis "Parse and manipulate version numbers") - (description "Parver facilitates parsing and manipulation of -@url{https://www.python.org/dev/peps/pep-0440/,PEP 440} version numbers.") - (license license:expat))) - -(define-public python-pyopenssl-for-haketilo - (let ((base python-pyopenssl)) - (package - (inherit base) - (version "22.0.0") - (source - (origin - (inherit (package-source base)) - (uri (pypi-uri "pyOpenSSL" version)) - (sha256 - (base32 - "1gzihw09sqi71lwx97c69hab7w4rbnl6hhfrl6za3i5a4la1n2v6")))) - (propagated-inputs - (modify-inputs (package-propagated-inputs base) - (replace "python-cryptography" python-cryptography-next)))))) - -(define-public python-urllib3-for-haketilo - (let ((base python-urllib3)) - (package - (inherit base) - (propagated-inputs - (modify-inputs (package-propagated-inputs base) - (replace "python-cryptography" python-cryptography-next) - (replace "python-pyopenssl" python-pyopenssl-for-haketilo)))))) - -(define-public python-requests-for-haketilo - (let ((base python-requests)) - (package - (inherit base) - (propagated-inputs - (modify-inputs (package-propagated-inputs base) - (replace "python-urllib3" python-urllib3-for-haketilo)))))) - -(define-public python-werkzeug-for-haketilo - (let ((base python-werkzeug)) - (package - (inherit base) - (propagated-inputs - (modify-inputs (package-propagated-inputs base) - (replace "python-requests" python-requests-for-haketilo)))))) - -(define-public python-flask-for-haketilo - (let ((base python-flask)) - (package - (inherit base) - (propagated-inputs - (modify-inputs (package-propagated-inputs base) - (replace "python-werkzeug" python-werkzeug-for-haketilo)))))) - -(define-public mitmproxy - (package - (name "mitmproxy") - (version "8.1.1") - (source - (origin - (method git-fetch) - (uri (git-reference - (url "https://github.com/mitmproxy/mitmproxy") - (commit (string-append "v" version)))) - (sha256 - (base32 "0kpzk8ci02vyjg9nqnpnadmgyaxxrpdydgfnm2xmxf1s4rzdcvwx")) - (snippet - '(begin - ;; The player contains some minified JS. It would be possible to find - ;; player sources elsewhere on the internet but there's no point in - ;; doing do since we're not building the docs anyway. - (delete-file "docs/src/assets/asciinema-player.js") - #t)))) - (build-system python-build-system) - (arguments - `(#:phases - (modify-phases %standard-phases - (add-after 'unpack 'relax-requirements - (lambda _ - (substitute* "setup.py" - (("kaitaistruct>=0\\.7[^\"]*") "kaitaistruct") - ;; The ">=2.8" req was there because older ldap3 lacked a crucial - ;; ">=0.4.8" req for its dep, pyasn. It's not an issue for Guix - ;; which ships with pyasn 4.8 anyway. - (("ldap3>=2\\.8[^\"]*") "ldap3") - (("protobuf>=3\\.14,<5") "protobuf") - (("sortedcontainers>=2\\.3[^\"]*") "sortedcontainers") - (("wsproto>=1\\.0[^\"]*") "wsproto") - (("pytest-timeout[^\"]*<2[^\"]*") "pytest-timeout") - (("pytest-asyncio[^\"]*<0.14[^\"]*") "pytest-asyncio")) - (substitute* "test/mitmproxy/proxy/layers/http/test_http.py" - (("isinstance\\(x, HTTPFlow\\)") - "issubclass(type(x), HTTPFlow)")))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (setenv "HOME" "/tmp") - (invoke "pytest" "--timeout" "60"))))))) - (propagated-inputs - (list python-asgiref - python-blinker - python-brotli - python-cryptography-next - python-flask-for-haketilo - python-h11 - python-h2 - python-hyperframe - python-kaitaistruct - python-ldap3 - python-msgpack - python-passlib - python-protobuf - python-pyopenssl-for-haketilo - python-pyparsing - python-pyperclip - python-ruamel.yaml - python-sortedcontainers - python-tornado-6 - python-urwid - python-wsproto - python-publicsuffix2 - python-zstandard)) - (native-inputs - (list python-parver - python-pytest - python-pytest-asyncio - python-pytest-timeout)) - (home-page "https://mitmproxy.org/") - (synopsis "A free interactive HTTPS proxy") - (description - "An interactive TLS-capable intercepting HTTP proxy for penetration testers -and software developers. It can be used to intercept, inspect, modify and -replay web traffic such as HTTP/1, HTTP/2, WebSockets, or any other -SSL/TLS-protected protocols.") - (license license:expat))) - -(define-public python-immutables-for-haketilo - (let ((base python-immutables)) - (package - (inherit base) - (version "0.19") - (source - (origin - ;; Old version tarballs seem to be getting deleted from PyPI each time - ;; a new version comes out. - (method git-fetch) - (uri (git-reference - (url "https://github.com/MagicStack/immutables") - (commit (string-append "v" version)))) - (sha256 - (base32 - "1awjylp4pl0jknwgnabg7kkj8f8883fkf99spsdrw1pj1acajvy9")))) - (arguments - `(#:phases - (modify-phases %standard-phases - (add-after 'unpack 'fix-expected-mypy-types - (lambda _ - (substitute* "tests/test-data/check-immu.test" - (("builtins.str") "builtins.str*")))) - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest"))))))) - (native-inputs - (list python-pytest python-mypy))))) - -(define-public python-types-requests - (package - (name "python-types-requests") - (version "2.26.0") - (source (origin - (method url-fetch) - (uri (pypi-uri "types-requests" version)) - (sha256 - (base32 - "10sq8jarr642vhw53k6zbf3hn2b8xfyrckwfngml4fj19g1whpnz")))) - (build-system python-build-system) - (home-page "https://github.com/python/typeshed") - (synopsis "Typing stubs for requests") - (description - "This package provides a collection of library stubs for Python, with -static types.") - (license license:asl2.0))) - -(define-public hydrilla - (package - (name "hydrilla") - ;; Change the following to the version for which you have generated a - ;; tarball under dist/. - (version "3.0b0") - (source (local-file - (string-append %source-dir "/dist/hydrilla-" version ".tar.gz"))) - (build-system python-build-system) - (arguments - `(#:phases - (modify-phases %standard-phases - (replace 'check - (lambda* (#:key tests? #:allow-other-keys) - (when tests? - (invoke "pytest"))))))) - (propagated-inputs - (list mitmproxy - python-beautifulsoup4 - python-click - python-flask-for-haketilo - python-gnupg - python-html5lib - python-immutables-for-haketilo - python-itsdangerous - python-jsonschema - reuse)) - (native-inputs - (list python-setuptools-scm - python-babel - python-pytest - python-pypa-build - python-mypy - python-types-requests)) - (home-page "https://hydrillabugs.koszko.org/projects/haketilo/wiki") - (synopsis "Block JavaScript and add custom logic to web pages") - (description "Haketilo HTTP proxy facilitates viewing of websites while -having their original JavaScript replaced by user-provided scripts. Haketilo -combines the functionalities of content blocker and user script manager. It can -be used with its script repository, Hydrilla.") - (license (list license:agpl3+ license:gpl3+ license:cc0)))) - -hydrilla diff --git a/hydrilla.scm b/hydrilla.scm new file mode 100644 index 0000000..6b8ee5c --- /dev/null +++ b/hydrilla.scm @@ -0,0 +1,333 @@ +;; SPDX-License-Identifier: CC0-1.0 + +;; Copyright (C) 2022 Wojtek Kosior +;; +;; Available under the terms of Creative Commons Zero v1.0 Universal. + +(define-module (hydrilla)) + +(use-modules + (ice-9 rdelim) + (ice-9 regex)) + +(define %source-dir (dirname (current-filename))) + +;; The PKG-INFO file is generated when running `python3 -m build -s` or similar. +;; It is also automatically included in the generated tarballs. +(define %hydrilla-version + (if (access? "src/hydrilla.egg-info/PKG-INFO" R_OK) + (let* ((port (open-input-file "src/hydrilla.egg-info/PKG-INFO")) + (process-line + (lambda (self-ref) + (let ((match-result + (string-match "^Version: (.*)" (read-line port)))) + (if match-result (match:substring match-result 1) + (self-ref self-ref)))))) + (process-line process-line)) + "unknown")) + +(use-modules + (guix packages) + (guix download) + (guix git-download) + (guix build-system python) + (guix gexp) + ((guix licenses) #:prefix license:) + (gnu packages python-build) + (gnu packages python-xyz) + (gnu packages python-crypto) + (gnu packages compression) + (gnu packages python-compression) + (gnu packages xdisorg) + (gnu packages serialization) + (gnu packages protobuf) + (gnu packages python-web) + (gnu packages check) + (gnu packages sphinx) + (gnu packages python-check) + (gnu packages license) + (gnu packages gnupg)) + +(define-public python-kaitaistruct + (package + (name "python-kaitaistruct") + (version "0.10") + (source + (origin + (method url-fetch) + (uri (pypi-uri "kaitaistruct" version)) + (sha256 + (base32 "0ap5ka51gnc2mc4s1kqqsi6nb6zqv8wsrg17ryxazmkkj7idwi50")))) + (build-system python-build-system) + (home-page "https://kaitai.io") + (native-inputs (list python-wheel)) + (synopsis + "Declarative parser generator for binary data: runtime library for Python") + (description + "Kaitai Struct is a declarative language used for describing various binary +data structures, laid out in files or in memory - i.e. binary file formats, +network stream packet formats, etc.") + (license license:expat))) + +(define-public python-parver + (package + (name "python-parver") + (version "0.3.1") + (source + (origin + (method url-fetch) + (uri (pypi-uri "parver" version)) + (sha256 + (base32 "1lyzqp8bz0n2kzabzl7k7g7cn90rlnrxjzva2p62gsfc7djy00n9")))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'relax-requirements + (lambda _ + (substitute* "setup.py" + (("arpeggio[^']*") "arpeggio")))) + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (when tests? + (invoke "pytest"))))))) + (propagated-inputs (list python-arpeggio python-attrs python-six)) + (native-inputs + (list python-hypothesis + python-pretend + python-pytest)) + (home-page "https://github.com/RazerM/parver") + (synopsis "Parse and manipulate version numbers") + (description "Parver facilitates parsing and manipulation of +@url{https://www.python.org/dev/peps/pep-0440/,PEP 440} version numbers.") + (license license:expat))) + +(define-public python-pyopenssl-for-haketilo + (let ((base python-pyopenssl)) + (package + (inherit base) + (version "22.0.0") + (source + (origin + (inherit (package-source base)) + (uri (pypi-uri "pyOpenSSL" version)) + (sha256 + (base32 + "1gzihw09sqi71lwx97c69hab7w4rbnl6hhfrl6za3i5a4la1n2v6")))) + (propagated-inputs + (modify-inputs (package-propagated-inputs base) + (replace "python-cryptography" python-cryptography-next)))))) + +(define-public python-urllib3-for-haketilo + (let ((base python-urllib3)) + (package + (inherit base) + (propagated-inputs + (modify-inputs (package-propagated-inputs base) + (replace "python-cryptography" python-cryptography-next) + (replace "python-pyopenssl" python-pyopenssl-for-haketilo)))))) + +(define-public python-requests-for-haketilo + (let ((base python-requests)) + (package + (inherit base) + (propagated-inputs + (modify-inputs (package-propagated-inputs base) + (replace "python-urllib3" python-urllib3-for-haketilo)))))) + +(define-public python-werkzeug-for-haketilo + (let ((base python-werkzeug)) + (package + (inherit base) + (propagated-inputs + (modify-inputs (package-propagated-inputs base) + (replace "python-requests" python-requests-for-haketilo)))))) + +(define-public python-flask-for-haketilo + (let ((base python-flask)) + (package + (inherit base) + (propagated-inputs + (modify-inputs (package-propagated-inputs base) + (replace "python-werkzeug" python-werkzeug-for-haketilo)))))) + +(define-public mitmproxy + (package + (name "mitmproxy") + (version "8.1.1") + (source + (origin + (method git-fetch) + (uri (git-reference + (url "https://github.com/mitmproxy/mitmproxy") + (commit (string-append "v" version)))) + (sha256 + (base32 "0kpzk8ci02vyjg9nqnpnadmgyaxxrpdydgfnm2xmxf1s4rzdcvwx")) + (snippet + '(begin + ;; The player contains some minified JS. It would be possible to find + ;; player sources elsewhere on the internet but there's no point in + ;; doing do since we're not building the docs anyway. + (delete-file "docs/src/assets/asciinema-player.js") + #t)))) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'relax-requirements + (lambda _ + (substitute* "setup.py" + (("kaitaistruct>=0\\.7[^\"]*") "kaitaistruct") + ;; The ">=2.8" req was there because older ldap3 lacked a crucial + ;; ">=0.4.8" req for its dep, pyasn. It's not an issue for Guix + ;; which ships with pyasn 4.8 anyway. + (("ldap3>=2\\.8[^\"]*") "ldap3") + (("protobuf>=3\\.14,<5") "protobuf") + (("sortedcontainers>=2\\.3[^\"]*") "sortedcontainers") + (("wsproto>=1\\.0[^\"]*") "wsproto") + (("pytest-timeout[^\"]*<2[^\"]*") "pytest-timeout") + (("pytest-asyncio[^\"]*<0.14[^\"]*") "pytest-asyncio")) + (substitute* "test/mitmproxy/proxy/layers/http/test_http.py" + (("isinstance\\(x, HTTPFlow\\)") + "issubclass(type(x), HTTPFlow)")))) + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (when tests? + (setenv "HOME" "/tmp") + (invoke "pytest" "--timeout" "60"))))))) + (propagated-inputs + (list python-asgiref + python-blinker + python-brotli + python-cryptography-next + python-flask-for-haketilo + python-h11 + python-h2 + python-hyperframe + python-kaitaistruct + python-ldap3 + python-msgpack + python-passlib + python-protobuf + python-pyopenssl-for-haketilo + python-pyparsing + python-pyperclip + python-ruamel.yaml + python-sortedcontainers + python-tornado-6 + python-urwid + python-wsproto + python-publicsuffix2 + python-zstandard)) + (native-inputs + (list python-parver + python-pytest + python-pytest-asyncio + python-pytest-timeout)) + (home-page "https://mitmproxy.org/") + (synopsis "A free interactive HTTPS proxy") + (description + "An interactive TLS-capable intercepting HTTP proxy for penetration testers +and software developers. It can be used to intercept, inspect, modify and +replay web traffic such as HTTP/1, HTTP/2, WebSockets, or any other +SSL/TLS-protected protocols.") + (license license:expat))) + +(define-public python-immutables-for-haketilo + (let ((base python-immutables)) + (package + (inherit base) + (version "0.19") + (source + (origin + ;; Old version tarballs seem to be getting deleted from PyPI each time + ;; a new version comes out. + (method git-fetch) + (uri (git-reference + (url "https://github.com/MagicStack/immutables") + (commit (string-append "v" version)))) + (sha256 + (base32 + "1awjylp4pl0jknwgnabg7kkj8f8883fkf99spsdrw1pj1acajvy9")))) + (arguments + `(#:phases + (modify-phases %standard-phases + (add-after 'unpack 'fix-expected-mypy-types + (lambda _ + (substitute* "tests/test-data/check-immu.test" + (("builtins.str") "builtins.str*")))) + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (when tests? + (invoke "pytest"))))))) + (native-inputs + (list python-pytest python-mypy))))) + +(define-public python-types-requests + (package + (name "python-types-requests") + (version "2.26.0") + (source (origin + (method url-fetch) + (uri (pypi-uri "types-requests" version)) + (sha256 + (base32 + "10sq8jarr642vhw53k6zbf3hn2b8xfyrckwfngml4fj19g1whpnz")))) + (build-system python-build-system) + (home-page "https://github.com/python/typeshed") + (synopsis "Typing stubs for requests") + (description + "This package provides a collection of library stubs for Python, with +static types.") + (license license:asl2.0))) + +;; Use this variant when building from a downloaded release tarball. +(define-public hydrilla + (package + (name "hydrilla") + (version %hydrilla-version) + (source (local-file %source-dir #:recursive? #t)) + (build-system python-build-system) + (arguments + `(#:phases + (modify-phases %standard-phases + (replace 'check + (lambda* (#:key tests? #:allow-other-keys) + (when tests? + (invoke "pytest"))))))) + (propagated-inputs + (list mitmproxy + python-beautifulsoup4 + python-click + python-flask-for-haketilo + python-gnupg + python-html5lib + python-immutables-for-haketilo + python-itsdangerous + python-jsonschema + reuse)) + (native-inputs + (list python-setuptools-scm + python-babel + python-pytest + python-pypa-build + python-mypy + python-types-requests)) + (home-page "https://hydrillabugs.koszko.org/projects/haketilo/wiki") + (synopsis "Block JavaScript and add custom logic to web pages") + (description "Haketilo HTTP proxy facilitates viewing of websites while +having their original JavaScript replaced by user-provided scripts. Haketilo +combines the functionalities of content blocker and user script manager. It can +be used with its script repository, Hydrilla.") + (license (list license:agpl3+ license:gpl3+ license:cc0)))) + +;; Use this variant when building from a tarball generated under dist/. This +;; can be used to build from a git checkout after running `python3 -m build -s` +;; or similar. +(define-public hydrilla-dist-tarball + (let ((base hydrilla) + (filename (string-append "hydrilla-" %hydrilla-version ".tar.gz"))) + (package + (inherit base) + (source (local-file + (string-append %source-dir "/dist/" filename)))))) -- cgit v1.2.3