algokit generate command
- Status: Approved
- Owner: Altynbek Orumbayev, Inaie Ignacio
- Deciders: Rob Moore, Daniel McGregor, Alessandro Cappellato
- Date created: 2023-07-19
- Date decided: 2023-07-24
- Date updated: 2023-07-24
Context¶
The Frontend Templates ADR introduced and expanded on AlgoKit's principles of Modularity and Maintainability by introducing a new set of official templates for quickly bootstrapping standalone react
and fullstack
projects showcasing best practices and patterns for building frontend and fullstack applications with Algorand. As a logical next step, we want to enable developers to extend existing projects instantiated from official templates with new files and features.
Requirements¶
1. AlgoKit user should be able to use generate command to extend existing algokit compliant projects with new files
of any kind¶
This implies scenarios like:
- Adding new contracts into existing algokit compliant projects.
Algokit compliant projects are projects that were instantiated from official or community templates and follow the same structure and conventions.
- Overriding existing files with new ones.
- Adding new files into existing projects.
Overall, we want to introduce a notion of generators
which can be viewed as a modular self-sufficient template units that are hosted within template repositories and describe how to create or update files within projects instantiated from AlgoKit templates.
Ruby on Rails has a similar concept of generators which are used to create or update files within Rails projects. This can be used as a reference for inspiration.
2. Template builder should be able to access a clear guideline and refer to official templates for examples on how to create generators
¶
This implies extension of existing starter guidelines available for template builders on AlgoKit Docs and using one or several official templates as a reference point.
Principles¶
- Modularity: Artifacts dependant on
advanced algokit generate
command capabilities embedded into templates should follow guiding AlgoKit principles and expand on approaches already utilized inreact
,fullstack
andbeaker
templates. This implies that giving developers flexibility to define any extra templating logic, allowing to create or update any files within projects instantiated from algokit templates. - Maintainability: The
advanced algokit generate
capabilities onalgokit-cli
and related artifacts on respective official templates should be easy to maintain and extend. - Seamless onramp: Great developer experience for template builders to create their own
generators
and user experience to use them viaadvanced algokit generate
command should be a priority.
All of the aforementioned requirements should be met in a way that is consistent with the guiding principles of AlgoKit or attempt to find a balanced trade of between the principles that satisfies the requirements. Refer to AlgoKit Guiding Principles for detailed reference on the principles.
Considered Options¶
Based on preliminary research, all of the options below assume that:
A generator
is a self contained copier/jinja template that is hosted within a template repository and describes how to create or update files within projects instantiated from algokit templates. Hosting it along with the template is a necessity given that community based templates can follow different conventions, patterns and structure making it hard to attempt to generalize the logic of generators
and make them work for all templates.
Option 1: Wrapping generators into self contained copier templates hidden within algokit templates¶
This option implies that generators
are self contained copier templates that are hidden within algokit templates and are not exposed to the end user. This option is inspired by Ruby on Rails generators and Yeoman generators.
The main idea is to rely on _templates_suffix
in copier.yamls to define 2 separate types of suffixes for templates
and for generators
:
- Existing templates under all official algokit templates are already prefixed with
.jinja
hence we just need to explicitly prefix it with.jinja
on root copier - The new generators jinja templates can be prefixed (for example) with alternative file extension for jinja files such as
.j2
. Which is also a common convention for jinja templates. -
- This only works for files though for regular folders and cases like
{% if %}folder_name{% endif %}.j2
we need to wrap them into {% raw %} to that first pass when template initially initialized unwraps the content allowing second pass via generator to then use them as jinja templates. The only downside here is slightly longer file names for folders, but I think it's a reasonable tradeoff considering simplicity of the solution.
- This only works for files though for regular folders and cases like
Overview of the proposal can be summarized via the following diagram:
graph TB
T["Template Folder"]
T --> C["copier.yaml"]
T --> C1["..."]
T --> G[".algokit/generators"]
G --> G1["Generator 1"]
G1 --> E1["copier.yaml"]
G1 --> E2["..."]
G --> G2["..."]
G2 --> E3["..."]
G --> G3["Generator N"]
G3 --> E["copier.yaml"]
G3 --> E4["..."]
Pros¶
- Generators are hidden within algokit templates and are not exposed to the end user. When user runs
algokit generate
command, cli presents a list of available generators to choose from. This makes it easier for user to understand what generators are available and what they do. - Generators are self contained copier templates giving template builders flexibility to do any kind of templating logic similar to what they can already do with regular templates.
- Majority of implementation complexity is reduced by relying on copier as a backbone for generators feature.
Cons¶
- Generators are somewhat tightly coupled with individual algokit templates, which implies its not necessarily a matter of copy pasting generators from one template to another. This can be a problem for community template builders who want to reuse generators from official templates. However, this can be mitigated by providing clear guidelines on how to create generators and referring to official templates as a reference point. Additionally, it seems like a reasonable tradeoff given that templates can vastly differ in type, structure, conventions and patterns and it's significantly harder to generalize the logic of generators to make them work for all templates.
Implementation details¶
1. Adjusting templates structure
For instance, if we assume existing beaker
template, the new file/folder structure can look as follows:
template_content/.algokit # alternatively could be just `.algokit-generators`
└── generators
└── {generator_name} # generator name can be anything
├── copier.yaml # copier config for generator
└── smart_contracts # logic for adding new contracts to beaker template
└── {% raw %}{{ contract_name }}{% endraw %}
├── contract.py.j2
├── {% raw %}{% if language == 'python' %}deploy_config.py{% endif %}{% endraw %}.j2
└── {% raw %}{% if language == 'typescript' %}deploy-config.ts{% endif %}{% endraw %}.j2
...rest of the template is left as is
copier.yml
The index.ts
and config.py
files on beaker template need to be updated to auto import all contracts from sub folders at smart_contracts
to eliminate the need for developers to manually import them after running the smart contract generator.
Please note, above is just an example that assumes the generator for adding new contracts, but the proposal is generic enough to support any kind of jinja-based templating logic.
2. Adjusting .algokit.toml
The proposal for new structure for defining generators in root algokit toml is as follows:
[generators.create_contract] # [generators.<generator_name>]
description = "Adds new smart contract to existing project" # description of the generator, can appear in cli for extra info
path = ".algokit/generators/create_contract" # path that cli should grab to forward to copier copy
3. Adjusting algokit generate
command on cli
Next step in implementation part of the proposal is adjusting the generator
command on algokit-cli
to make sure it knows how to look for generators. The available generators can be provided to user via list picker (a.k.a ruby on rails style) by letting algokit scan contents of algokit.toml
and look for .generators
folder.
a) Has no generators If algokit-cli can't find any generator configured then nothing change from current implementation.
b) Has generators and user runs algokit generate
command to see the list of available generators
A new click
command per each generator in .algokit.toml
is added to the list of generate commands, that will list all available generators for the user to choose from.
algokit-cli generate
---
Usage: algokit generate [OPTIONS] COMMAND [ARGS]...
Generate code for an Algorand project.
Options:
-h, --help Show this message and exit.
Commands:
client Create a typed ApplicationClient from an ARC-32 application.json
smart-contract Adds new smart contract to existing project
Then, to invoke the interactive flow via copier user runs:
algokit-cli generate smart-contract `name of the contract`
c) Has generators, user knows what to pick and wants to run it non interactively
Using the generator configuration the generator
will automatically add new click
commands to the existing list of generate commands allowing the user to run them in the command line.
algokit-cli generate smart-contract -a contract_name=`name of the contract` -a language=python # passing extra arguments to generator similar to algokit init
4. Testing and documentation
Lastly we need to make sure that the new feature is properly tested and documented. This includes:
- Testing the new feature works as expected on all official templates that will host generators. This will imply adding a separate test suite that picks some default template state and run generators on top of them confirming that created artifacts are placed in expected locations and have expected content. Tests should be easy to follow and expand on existing tests suite on templates without introducing extra complexity.
- Introduce new tutorial sections on AlgoKit Docs on how to create generators and refer to official templates as a reference point.
Option 2: Wrapping generators into self contained copier templates hosted on separate repositories¶
This option proposes to host generators
on separate set of repositories and use them as a dependency for algokit templates.
algokit.toml
can be extended on template repositories to list generators they depend on.
The only distinction between this option and option 1 is that generators are hosted on separate repositories and are not hidden within algokit templates. Implying that they are not tightly coupled with algokit templates and can be reused/forked by community template builders to build their own generators.
Pros¶
- Generators are not tightly coupled with algokit templates and can be reused/forked by community template builders to build their own generators.
- Generators can be versioned and updated independently from algokit templates.
Cons¶
- Developing and maintaining generators is significantly more complex due to the need to maintain separate repositories and versioning.
- Official maintainers and community template builders need to be put extra effort at keeping generators generic enough to be reused by other templates. Given that templates can vastly differ in type, structure, conventions and patterns, this can be a challenge.
- Given that copier is being considered as a backbone for generators feature, drawbacks outlined for hosting templates on monorepos in the previous adr apply here as well.
Open questions for reviewers¶
- What kinds of generators other than
add new contract
do we want to support on initial release (if any)? - Are there any other template repositories that we want to integrate with generators other than
beaker
(fullstack
will contain those as well as it usesbeaker
as a dependency)?
Please note an MVP PoC is already implemented and available to play around on algokit-cli and beaker template repo under
advanced-generate-command
andgenerators
branches respectively. To test it out checkout the branche on cli dopipx install . --force
, navigate to beaker template repo and checkout the branch as well, then navigate to any of the sub folders intests_generated
. Lastly doalgokit bootstrap all
, build the contract and executealgokit generate
from root of that folder to play around with the feature based on the implementation proposal from Option 1.
Final Decision¶
The team approved the proposal for Option 1: Wrapping generators into self contained copier templates hidden within algokit templates.
Next steps¶
- Polishing the PoC on algokit-cli and adding tests
- Polishing the PoC on beaker template and adding tests
- Adding documentation for new capabilities of the generate command
- Adding documentation for template builders on how to create generators