Building a release

Before you build

• Python 3.11.4 or later (the deploy-bundle export reads foundry.toml with tomllib, new in 3.11, and the verification extracts the archive with tarfile's data filter, new in 3.11.4), Foundry, Pillow, and pycryptodome or OpenSSL 3.2 or later. No Node.js.
• Merge any CSS parts first: python3 tools/merge_css.py. The build stops while web/assets/css/app.css still imports parts/*.css.
• The contracts carry their own version string: DAOFactory.VERSION and the record script/Deploy.s.sol writes, "COLLAB DAO v2.2.0". A server's admin service accepts only records of its release's major and minor version.

python3 tools/merge_css.py
python3 tools/build_release.py --version 2.2.0
FORGE_FLAGS="--offline --use /path/to/solc-0.8.28" python3 tools/build_release.py --version 2.2.0   # without network

In the working tree

1. Refresh the generated blocks. tools/shell_html.py --write writes the portal shell and the preload block of every entry. tools/build_docs.py --write writes the documentation pages. The docs renderer's unit tests run (python3 -m unittest discover -s tools/tests -p "test_*.py"), then tools/build_docs.py --check --strict fails on any problem or placeholder page or a sitemap that lists other docs pages, and tools/docs_figures.py --check on a missing figure.
2. Check the site. tools/make_assets.py --check (the icons are renders of favicon.svg), tools/lint_web.py (0 errors), tools/prerender.py --static-pages --check (the /pools and /locks copies).
3. Test the server kit. python3 -m unittest discover -s server/tests -p "test_*.py": the admin service, the site settings, the records and config.js.
4. Compile from scratch. forge build --force --skip test --skip script builds the contracts; then the files of script/ and test/ are built FORGE_BATCH at a time. One via-IR solc run over everything can need more than 5 GB of memory; in batches it stays below 2 GB, with the same artifacts. The build always starts from scratch, because Foundry's cache does not notice every settings change and a stale artifact must never reach the deploy bundle.
5. Run the suite. forge test (331 tests in 92 suites for 2.2.0).
6. Export. tools/export_deploy_artifacts.py --release <version> writes web/assets/deploy/collab-protocol.json, the bundle the admin page's Deploy tab sends; tools/export_abis.py writes abi/ and web/js/core/abi.js; tools/abi_summary.py writes docs/CONTRACT_API.md.

Steps 1 and 6 rewrite generated files in the working tree itself; when they change anything, commit those files with the release.

In the stage

7. Copy the tree to dist/collab-<version>/: README.md, LICENSE, THIRD_PARTY_NOTICES.md, foundry.toml, remappings.txt, .gitignore, and contracts/, test/, script/, externals/, abi/, docs/, tools/, web/, deployments/, deploy/ and server/. The libraries are trimmed to what COLLAB compiles and their licences: OpenZeppelin's contracts/ folders, Solady's src/utils/SSTORE2.sol, forge-std's src/. Left out: __pycache__, config.local.js, the local chain's files in deployments/ (local.json, demo.json, anvil.pid, anvil.log, integrations.local.json), web/assets/css/parts, and the per-deployment web/dao/ and web/og/.
8. Version the assets. tools/version_assets.py web --version <version> --no-gzip appends one release-wide ?v=<12 hex> to every module import (static and dynamic), every /js/ string, the stylesheets, fonts, the icon sprite and CSS url()s, and writes web/BUILD.json (the build stamp). It fails when a reference is left unversioned or a module would be reached under two URLs, which would load it twice.
9. Write the static pages again from the versioned tree: tools/prerender.py --static-pages (/pools and /locks) and tools/build_docs.py --write (the docs pages, which now carry the release's ?v=).
10. Precompress. Every text file of 1 KB or more (.js, .css, .html, .svg, .json, .webmanifest, .txt, .xml, .map, .md), except config.js, gets a .gz sibling for nginx's gzip_static, at level 9 and without a timestamp, so the same file always compresses to the same bytes.
11. Checksum. SHA256SUMS lists every file of the stage as <sha256> <path>.

The archive

12. Pack. A PAX tar of the stage under the folder collab-<version>/: entries sorted, every modification time set to SOURCE_DATE_EPOCH, owner and group 0 with no names, mode 0755 for folders and executables and 0644 for the rest. It is gzipped at level 9 with no file name or time in the header, as dist/collab-<version>.tar.gz, and its SHA-256 goes to dist/collab-<version>.tar.gz.sha256. The build prints the size, the number of files and the digest.

Nothing in the archive depends on when or where it was built beyond the tools' versions: the same tree with the same Foundry, solc, Python and zlib gives the same archive.

Verification from a clean extraction

13. The archive is extracted into a temporary folder, and every line of SHA256SUMS must verify.
14. In the extracted tree: tools/lint_web.py, tools/version_assets.py web --check (every asset URL carries the version), tools/shell_html.py --check, tools/prerender.py --static-pages --check and tools/build_docs.py --check --strict.
15. The contracts are compiled from the archive alone (forge build --force --skip test --skip script), and tools/export_deploy_artifacts.py --release <version> --check confirms the deploy bundle matches them byte for byte, so the archive builds the exact code the admin page deploys.

The build ends with "release verified: checksums match, lint passes, the docs pages are current, contracts compile from the archive and the deploy page's bundle matches them". Any failing step stops the build with that step's error.

Checking a release yourself

sha256sum -c collab-2.2.0.tar.gz.sha256
tar xzf collab-2.2.0.tar.gz && cd collab-2.2.0
sha256sum -c --quiet SHA256SUMS
python3 tools/lint_web.py && python3 tools/version_assets.py web --check && python3 tools/build_docs.py --check --strict
forge build --force --skip test --skip script && python3 tools/export_deploy_artifacts.py --release 2.2.0 --check

To run the whole suite on the release, build the test files in batches and run forge test in the extracted folder; the end-to-end suites of a release run against its built tree (Testing).

Related

• Testing: every check the build runs.
• Data formats: BUILD.json, the deploy bundle and config.js.
• The web app: the docs build.
• Upgrades, rollbacks and backups: installing a new release on a server.
• Release notes: what changed in 2.2.0.