2024 01 13 native binaries
AlgoKit CLI native binaries¶
- Status: Approved
- Owner: Altynbek Orumbayev (MakerX), Negar Abbasi (MakerX)
- Deciders: Alessandro (Algorand Foundation), MakerX
- Date created: 2024-01-13
- Date decided: 2024-01-25
- Date updated: 2024-01-16
Context¶
The primary motivation for this decision is to streamline the installation process of AlgoKit CLI and reduce the friction associated with installing it on various operating systems. Currently, users often encounter minor environment-specific bugs during installation, which can be a significant deterrent. By providing native binaries, we aim to speed up the installation time and eliminate these bugs by removing requirements to install python by the user, thereby improving the overall user experience.
The north star for this decision is to provide a distribution model that can be described as:
graph TD
A[GitHub Runners] -->|Windows| B[Packaging tool]
A -->|Mac| C[Packaging tool]
A -->|Linux| D[Packaging tool]
B --> E[Windows Binary]
C --> F[Mac Binary]
D --> G[Linux Binary]
E -->|Winget| H[Windows Users]
F -->|Brew| I[Mac Users]
G -->|Snap| J[Linux Users]
A -->|Wheel Build| K[Poetry]
K --> L[PyPi]
L -->|pipx| M[Python Users]
⚠️⚠️⚠️ Please note diagram above is a draft and is to be separately discussed in a follow up ADR that will focus on distribution of the binaries.
The scope of this ADR only concerns the packaging for the CLI. The distribution via snap
, winget
and etc will be handled separately/in-parallel after decision and implementation of this ADR is in place.
Requirements¶
- The native binaries should be easy to maintain and understand from a CI/CD deployment perspective.
- The solution should support a wide variety of Linux distributions, macOS (both Apple Silicon and Intel architectures), and Windows.
- The solution should integrate seamlessly with existing installation options, including Homebrew, or provide an easier alternative.
- The solution should be designed with future scalability in mind, allowing for the addition of support for other variations of architectures or else as needed.
- The solution should not significantly increase the complexity of the build process.
- The solution should provide clear error messages and debugging information to assist in troubleshooting any issues that may arise.
- You don't need to install Python on your system to use AlgoKit if you aren't creating a Python project.
Options¶
Option 1 - PyInstaller¶
Pros
- Easy to use and configure
- Supports multiple platforms and architectures
- Can handle complex packages and dependencies
- Generates a single file executable
- Active development and community support
- Fairly fast build time via ci - ~3-4 minutes
- Fairly small executable size (see benchmarking results below)
- Marginally equal executable load time in
onedir
mode compared toonefile
mode
Cons
- Occasionally requires manual configuration for more complex packages
- Requires complex build packaging matrix to support multiple platforms and architectures
- Requires minor tweaks in algokit cli to account for the fact that features relying on
sys.executable
will point to algokit cli executable instead of python interpreter. This is a minor change and can be done in a backwards compatible way however still a con to consider. - Requires minor tweaks in algokit cli to introduce
multiprocessing.freeze_support()
to avoid issues withvanity-address
task when executing via binary.
PoC¶
The PoC is available here. It outlines a simple github action with extra setup that compiles algokit cli as a single file executable on latest versions of Windows, Mac and Linux github runners.
Option 2 - Nuitka¶
Pros
- Nuitka translates Python code into C and then compiles it, which can result in performance improvements.
- Cross-Platform: Supports multiple platforms including Windows, macOS, and Linux.
- More cross compilations options than PyInstaller
- Official github action simplifies the process of building executables for different platforms.
Cons
- Compilation Time: The process of converting Python to C and then compiling can be time-consuming. Up to ~30 minutes on github with 3 parallel jobs.
- Size of Executable: The resulting executables can be larger due to the inclusion of the Python interpreter and the compiled C code (see benchmarking results below).
- Does not support Python 3.12.
- Requires minor tweaks in algokit cli to account for the fact that features relying on
sys.executable
will point to algokit cli executable instead of python interpreter. This is a minor change and can be done in a backwards compatible way however still a con to consider. - Requires minor tweaks in algokit cli to introduce
multiprocessing.freeze_support()
to avoid issues withvanity-address
task when executing via binary.
PoC¶
The PoC is available here. It outlines a simple github action with extra setup that compiles algokit cli as a single file executable on latest versions of Windows, Mac and Linux github runners.
Benchmarking pyinstaller
vs nuitka
vs pipx installed algokit
¶
Methodology¶
hyperfine
was used to benchmark 5 different executables:
- Nuitka Onefile - Nuitka compiled executable with
--onefile
flag, which produces a single file executable. - Nuitka Onedir - Nuitka compiled executable with
--onedir
flag, which produces a directory with the executable and other dependencies unzipped. - PyInstaller Onedir - PyInstaller compiled executable with
--standalone
flag, which produces a directory with the executable and other dependencies unzipped. - PyInstaller Onefile - PyInstaller compiled executable with
--onefile
flag, which produces a single file executable. - AlgoKit from
pipx
- AlgoKit CLI installed viapipx
with all dependencies frozen (current latest stable release).
The benchmarking was performed on a MacBook M2 running macOS 14.2.1 and an ARM based Ubuntu 20.04.3 LTS running on a Parallels Desktop on the same machine.
Results¶
Method | macOS M2 | Ubuntu 20 ARM Linux VM | Windows 11 ARM |
---|---|---|---|
nuitka_onefile | 3.634 | 1.465 | 3.874 |
nuitka_onedir | 0.2515 | 0.6200 | 0.5136 |
pyinstaller_onedir | 0.3228 | 0.7927 | 0.6668 |
pyinstaller_onefile | 3.031 | 1.466 | 1.875 |
algokit | 0.3126 | 0.6111 | 0.7579 |
Figure: Benchmarking results comparing the performance of Nuitka (onefile, onedir modes), PyInstaller (onefile, onedir modes), and pipx installed Algokit CLI on macOS M2, Windows 11 ARM VM, Ubuntu 20 ARM VM.
Method | Windows (MB) | Ubuntu (MB) | macOS (MB) |
---|---|---|---|
nuitka_onedir | 92.10 | 106 | 166 |
nuitka_onefile | 22.48 | 23 | 41 |
pyinstaller_onedir | 46.07 | 52 | 113 |
pyinstaller_onefile | 26.47 | 25 | 45 |
Figure: Bundle sizes of folders with executables build with Nuitka (onefile, onedir modes), PyInstaller (onefile, onedir modes).
Preliminary Observations¶
- Nuitka's warmed up execution time is fast
- Nuitka produces largest executables in
onedir
mode - Nuitka is the slowest to build (no charts for build benchmarks, this is observations based on CI build time from PoC, see links above)
- PyInstaller produces smallest executables in
onedir
mode - PyInstaller is the fastest to build (no charts for build benchmarks, this is observations based on CI build time from PoC, see links above)
Honorable Mentions¶
cx_Freeze¶
cx_Freeze is a set of scripts and modules for freezing Python scripts into executables. It is similar to PyInstaller in many ways, but PyInstaller is preferred due to its more mature and comprehensive documentation.
PyOxidizer¶
PyOxidizer is a utility for producing binaries that embed Python. However, it is no longer actively maintained, which makes it a less desirable option for our needs.
Preferred option¶
Based on observations so far we are leaning towards an Option 1. Where we would use PyInstaller to build native binaries for Windows, Mac and Linux.
While nuitka
in onedir
mode is even faster than pip installed algokit, it generates larger executables, and is the slowest option in terms of build time. Pyinstaller is only marginally slower than nuitka
or pip installed algokit in terms of execution time, has mature documentation, and is the fastest option to build (in onedir
mode) and produces smaller executables than nuitka
. Given that and the fact that nuitka
does not support Python 3.12 yet and has a lot of magical
optimizations hidden under the hood, we are leaning towards PyInstaller as the preferred option for building native binaries given its maturity and straightforwardness despite marginally slower execution time (which is not a big deal given that we are talking of deviations of 5-10 milliseconds).
Selected option¶
The team has formally pre-discussed this ADR and has agreed to proceed with Option 1 - PyInstaller.
Next Steps¶
- Finalize the decision on the preferred option.
- Expand PoC and polish the github action to build native binaries for Windows, Mac and Linux for x86, x86-64 and ARM architectures.
- Implement portability snapshot tests, expanding existing algokit cli snapshot tests by running against real executable covering main functionality to test and ensure that the native binaries are portable and behave the same way as pip installed algokit cli.
- Submit follow up ADR to discuss strategies on how to distribute the binaries in most accessible, user friendly and secure way.