Deploying COLLAB: reference
Deployment has three parts:
- The contracts, once per network.
- The website: static files that can be published again whenever you like, on any web server.
- An optional job on that server that gives every DAO its own page for search engines and link
previews, and caches the L1 price and the token data under
/data/.
Nothing has to be operated afterwards. Without the third part, every page, every read from the chain and every transaction stay the same. Search engines and link previews then see generic DAO pages, amounts are shown in L1 unless a visitor chooses a currency, and token pages lack holder counts, the 24-hour change and the 7-day line (what you lose).
On your own server, server/install.sh does parts 2 and 3 for you. It installs and configures
nginx with HTTPS and the redirecting names, and the admin page from which you deploy the contracts
(part 1) and publish them on the site. It also installs the prerender job. See
Installing COLLAB on a server and Deploy the protocol from /admin.
The rest of this document is the reference behind them, and the way to publish web/ yourself, on
any host (Hosting without the server kit is the short version).
1. Contracts on GenesisL1
Requirements: Foundry, and an account funded with L1 for gas, imported into Foundry's encrypted keystore. Never put a private key in an environment variable or a file in this repository.
cast wallet import deployer --interactive # once; stores an encrypted key
forge build
forge test # must pass before any deploymentWith via-IR, compiling every test at once can take more than 5 GB of memory; see Building a release for the batched build the release uses.
External contracts and integrations
COLLAB builds on one live GenesisL1 contract, wL1 (0x59a153c0fD47C6c1F305Abdb8030A90Aa3001fe2).
EXT_WL1 overrides it.
The DAO factory also lists the integrations it offers new DAOs: GL1F collections and CIPNFT
contracts. A GL1F collection is a registry, its Model NFT, market and runtime, plus the model store
GL1F front-ends upload to. The deploy script reads the lists from script/integrations.json, which
holds the live ones on chain 29:
| Contract | GL1F (id forest) | GL1F Crypto (id crypto) |
|---|---|---|
| Registry | 0x33c9844F77a07e36B98f0FFf8201B8A8b02c2a69 | 0xF93C5407B07C29cCA9E1313d587fBDF33405038B |
| Model NFT | 0x44Dc1c54B8D579B42d78cC21cf8260DC0A3279fA | 0xad381e8841AC9d4D62B532f3044a870Da20A10C4 |
| Market | 0xA9bfa0a719b7F73cE85CA1E7f23af626D383fB46 | 0xa11a8BF841f544863173bd1F98a1E8b021df68c9 |
| Runtime | 0xD2fD0cf461a6cb56Fc08d9aEc120833D8E79044E | 0x99aAA2a23Fe672e8177f7038559dB9b5160d9932 |
| Model store | 0x9CdbC23392648Bd27B4A5eD09c0fEa9452454B54 | 0x555e94a1699a77C531B59f13810E3d3B6D74E906 |
| Site | https://gl1f.com/ | https://crypto.gl1f.com/ |
| Contract | CIPNFT (id cipnft) |
|---|---|
| CIPNFT | 0x05e4F1929947bE93a853F2de155fc1d6137f8446 |
| Site | https://cipnft.com |
Both GL1F collections were checked on chain: each registry's modelNFT() and each market's nft()
is that collection's Model NFT, and each runtime's registry() is its registry.
To list other integrations, or none, name another file in the same format with
EXT_INTEGRATIONS. Every entry needs all its fields:
{"gl1f": [{"id": "forest", "name": "GL1F", "site": "https://gl1f.com/", "registry": "0x…",
"nft": "0x…", "market": "0x…", "runtime": "0x…", "store": "0x…"}],
"cipnft": [{"id": "cipnft", "name": "CIPNFT", "site": "https://cipnft.com", "address": "0x…"}]}EXT_INTEGRATIONS replaces the single-contract variables of COLLAB 2.0 (EXT_CIPNFT and
EXT_GL1F_*).
The factory keeps these lists for good (defaultGl1f(), defaultCipnfts()), but they bind no DAO.
They are only what interfaces offer first:
- At creation. Each DAO chooses its own integrations. The creation wizard preselects the deployment's lists.
- Later. A DAO can add a collection or contract of its own, or remove any, each by a critical vote.
- Checks. The DAO's policy checks every addition:
- each address is non-zero, has code and is distinct;
- none is one of the DAO's own contracts or one of the protocol's;
- a GL1F collection's contracts point at each other;
- a DAO lists at most 8 GL1F collections and 8 CIPNFT contracts.
Do a dry run first, then broadcast:
forge script script/Deploy.s.sol --rpc-url https://rpc.genesisl1.org --account deployer --sender <address>
forge script script/Deploy.s.sol --rpc-url https://rpc.genesisl1.org --broadcast --slow \
--account deployer --sender <address>The script creates 20 contracts:
- TextStore and CollabKeyring.
- The DeSciFi pair implementation, factory and router.
- The LockBox implementation and LockFactory.
- MemberDistributor, plus the DAOToken implementation and TokenLauncher.
- PolicyBuilder and the six DAO implementations: shares, governor, treasury, policy, profile and vault.
- DAOFactory and the two lens contracts.
Deploy the protocol from /admin lists them in order, with what each one is.
The deployment record
The script writes the deployment record to deployments/29.json, or to the path in DEPLOY_OUT.
Commit that file: it is the public record the website configuration is generated from. It holds:
chainId,startBlockandversion(COLLAB DAO v2.2.0);- the address of each contract by key:
textStore,keyring,desciFactory,desciRouter,lockFactory,distributor,launcher,policyBuilder,factory,daoLens,marketLensandwl1; - the factory's two lists, as given:
"gl1f": [{id, name, site, registry, nft, market, runtime, store}]and"cipnft": [{id, name, site, address}].
Its full format, and the stricter checks a server's admin API applies before publishing it, are in Data formats.
tools/write_web_config.py turns the two lists into the integrations of config.js (section 2).
The site names the integrations by them. It checks them as the site does:
- ids of at most 32 lowercase letters, digits and hyphens (
x-is kept for DAOs' own integrations); - plain names of at most 40 characters, and https sites;
- distinct, non-zero addresses;
- at most 8 entries in each list.
From a browser wallet: the Deploy tab
The admin page's Deploy tab, /admin/deploy, is an alternative to forge script. /deploy answers
301 there.
- It sends the same 20 creations as
script/Deploy.s.sol, in the same order, from the wallet you connect, one transaction each. The page never asks for or handles a key. - Before that, it shows wL1 and the two integration lists. On chain 29 they come prefilled from
script/integrations.json. You may edit, add or remove entries, and each one is checked on chain:- a GL1F collection's registry, Model NFT, market and runtime must point at each other, and its model store must have code;
- a CIPNFT contract must answer as CIPNFT does.
- It keeps its progress in the browser, so a reload resumes where it stopped. Let the transaction in flight confirm before you reload: a reload during sending once led, in testing, to one extra transaction (the deployment still completed and verified).
- It compares the deployed code and wiring with this release.
- On a server set up with
server/install.sh, its last step, Publish to this site, sends the record to the admin service, which checks it, keeps it and writesconfig.js. Its Publish by hand part downloadsdeployments/<chainId>.jsonandconfig.js, both with the lists, for any other host.
The Check tab, /admin/check, checks an existing deployment against this release, the integration
lists and the factory's defaultGl1f() and defaultCipnfts() included. 2.1's /deploy?mode=check
arrives there. Deploy the protocol from /admin describes every step.
Save the downloaded record as deployments/<chainId>.json and commit it. A record made with
Foundry is published on a server from the admin page's Publish tab
(Publish, unpublish and check).
Compiler and verification
Compiler settings are fixed in foundry.toml: solc 0.8.28, via-IR, optimizer at 200 runs and EVM
istanbul, matching the GL1F contracts already on GenesisL1. Only remappings.txt is used, so a
forge build of the release archive produces exactly the bytecode, metadata included, that the
Deploy tab sends and checks. To publish verified sources on an explorer, export the standard JSON
input of each contract with
forge verify-contract <address> <path>:<Name> --show-standard-json-input.
There are no admin functions to call after deployment and nothing to transfer: the factory and every DAO it creates are owner-less. That holds for COLLAB's own contracts only. The integrations belong to others, whose owners keep their powers. For example:
- the original GL1F registry's owner sets fees, licences and terms;
- its market's owner sets the listing fee;
- CIPNFT's admin sets fees, offer rules and terms.
The site's DAO settings show, for each integration a DAO lists, who holds those keys and what they can do to the DAO's assets. A later version of COLLAB means a new factory; DAOs created by the old one keep working unchanged.
2. Website
Configuration
Generate web/config.js from the deployment record and write the matching Content-Security-Policy
into every page. config.js carries the record's contract addresses and its integration lists
(section 1). Then copy that policy into the server configs and web/_headers:
python3 tools/write_web_config.py deployments/29.json --patch-csp --site-url https://collab.genesisl1.com
python3 deploy/sync_csp.pywrite_web_config.py sets these:
RPC presets.
--rpc NAME=URL(repeatable) lists the endpoints. The default is the site settings' list, else the GenesisL1 RPC and the GL1X JSON-RPC.Custom RPCs.
--no-custom-rpcstops visitors from typing their own endpoint.Explorers.
--explorer NAME=URL(repeatable) lists the block explorers, the first being the default, and--explorer ""lists none. The default is the site settings' explorer, else GL1X.Prices.
--price-apisets the price source, and""turns prices off.Analytics.
--ga-idcounts visits with Google Analytics 4, and only after a visitor accepts. The page policy then allows only what gtag.js needs:script-src https://www.googletagmanager.com/gtag/: gtag.js and the scripts it loads. The path keeps out Tag Manager containers, which the same host serves at/gtm.js.img-srcandconnect-src:https://*.google-analytics.com, the regional collectors such aswww.andregion1., andhttps://www.googletagmanager.com.connect-srcalso:https://*.analytics.google.com.
Advertising features and Google signals are off, so no
doubleclick.netorgoogle.comorigin is needed.deploy/sync_csp.pyrefuses pages whosescript-srcadmits a whole domain, as older versions wrote for analytics: write the pages again with--patch-csp.Site URL.
--site-urlis the public origin used in canonical links. It is also the only origin from which the landing page asks for/og/network.json(section 3), so set it to the address you actually serve the site from.Site settings.
--site FILEtakes the alpha strip, announcement, featured and hidden DAOs, verified and hidden tokens, search-engine texts, links, analytics, RPCs and explorer from asite.json(Site settings). A value given on the command line wins over the file.No deployment yet.
--no-deployment(with--chain-id N, 29 by default) writes aconfig.jswithout contracts: the site then says on every page that COLLAB is not deployed yet.Strict records.
--strictchecks the record as a server's admin API does.Output.
--out FILEwritesconfig.jssomewhere other thanweb/config.js.The policy alone.
--csp-of CONFIG_JSprints the policy of an existingconfig.js(--headerfor its response-header form), or with--patch-csp --web DIRwrites it into the pages under DIR.
Every URL must use https (http only for localhost and 127.0.0.1), with a host of ASCII
letters, digits, dots and hyphens and an optional port; write internationalized names in punycode
(xn--…). The tool refuses anything else, because each origin goes into the policy of every page
as it is.
The page policy's connect-src lists exactly what this configuration needs. deploy/sync_csp.py
refuses to go on when two pages disagree. The format of config.js is in
Data formats.
Indexing
Search engines may index the landing page, /daos, /votes, /tokens and every token's page,
/create, /swap, /pools, /locks, /docs and every documentation page, and every DAO's page.
The settings, privacy, admin and not-found pages are not indexed (the admin page also with an
X-Robots-Tag header):
- Each carries
<meta name="robots" content="noindex">in its head. - None of them is in the sitemap.
robots.txtdoes not block them, because a crawler has to read a page to see its noindex.
tools/prerender.py lists only the indexable pages in the sitemap, and leaves out any entry page
that says noindex. deploy/check_site.py checks all three points on a live site.
What to publish
Publish the web/ directory. Leave out web/DEVELOPING.md and web/config.local.js, which are for
development only; the server configs below refuse to serve them anyway.
Serve the site at the root of a domain, as at https://collab.genesisl1.com/. Every asset path
in the pages is root-absolute (/assets/…, /js/…, /config.js), so the pages work at any URL
depth, but a sub-path such as https://example.org/collab/ does not work. If you use another
domain, pass it to --site-url and change it in the server configs. Then run
python3 tools/prerender.py --static-pages --static-sitemap --site https://<host>, which writes it
into the /pools and /locks pages, web/sitemap.xml and the Sitemap: line of
web/robots.txt (section 3); every prerender run writes the last two again. On a server set up with
server/install.sh, the prerender job does this for the main host by itself.
Clean URLs
Each address is served by one entry file:
| Address | Entry file |
|---|---|
/ | index.html |
/daos | daos.html (filters in the query, such as ?q=&sort=members&mine=1) |
/votes | votes.html |
/tokens, /tokens/<address> | tokens/index.html and tokens/<address>/index.html written by tools/prerender.py when present, otherwise tokens.html |
/dao/<slug> and its sections, such as /dao/<slug>/proposals/3 | dao/<slug>/index.html written by tools/prerender.py when present, otherwise dao.html |
/dao/<address> | dao/<address>/index.html written by tools/prerender.py when present (a page without scripts whose canonical URL and refresh lead to /dao/<slug>), otherwise dao.html, which moves the address to the DAO's slug URL |
/dao/<address>/…, such as /dao/<address>/proposals/3 | dao.html, which moves the address to the DAO's slug URL and keeps the section |
/create | create.html |
/pools, /locks | pools/index.html, locks/index.html: descifi.html with the head of that page (section 3), otherwise descifi.html |
/swap, /swap/<in>/<out>, /pools/<pair>, /pools/add/…, /locks/<owner> | descifi.html |
/docs | docs.html, the documentation's home |
/docs/<slug> | docs/<slug>/index.html, written by tools/build_docs.py |
/docs/<slug>.md | the page's Markdown, served as text/markdown |
/admin, /admin/<tab>[/…] | admin.html (noindex, revalidated on every load; its API needs server/install.sh) |
/settings | settings.html |
/privacy | privacy.html |
The old addresses keep working:
/index.html,/daos.html,/votes.html,/tokens.html,/create.html,/docs.html,/settings.html,/privacy.htmland/admin.htmlanswer301with the clean address. So do/deployand/deploy.html, to/admin/deploy. The query string is kept, and the browser keeps the part after#./dao.htmland/descifi.htmlstill answer. Their old links carry the route after#, which only the browser sees, so the app moves them to the clean address.- A folder's
index.html(/dao/<slug>/index.html,/pools/index.html,/tokens/index.html) answers301with the folder's clean address. - A trailing slash is removed with a
301, but only from a path of plain characters (letters, digits,.,_,~and-between the slashes). The servers read the decoded path, so a line break, backslash or tab from it would reach theLocationheader. Any other path ending in/is not found, and the app drops a trailing slash itself.
Unknown paths answer 404.html with status 404.
On a host without rewrites, 404.html sends a visitor on an app path to /<entry>.html?route=<path>,
and the app restores the clean address.
nginx
deploy/nginx/collab.conf does all of the above, plus HTTPS, the security headers, compression and
caching. It is written for nginx 1.18 and later and was tested with 1.24.
sudo cp deploy/nginx/collab.conf /etc/nginx/sites-available/collab.conf
sudo ln -s /etc/nginx/sites-available/collab.conf /etc/nginx/sites-enabled/collab.conf
sudo nginx -t && sudo systemctl reload nginxEdit server_name, root (default /srv/collab/web), the certificate paths and the redirect
target of the port 80 server first. The config expects a Let's Encrypt certificate
(certbot certonly --webroot or --nginx). The port 80 server only redirects to HTTPS.
Caddy
deploy/caddy/Caddyfile does the same on Caddy 2.6 and later. Caddy obtains and renews the
certificate itself.
sudo cp deploy/caddy/Caddyfile /etc/caddy/Caddyfile
caddy validate --config /etc/caddy/Caddyfile && sudo systemctl reload caddyEdit the site address and root first.
IPFS gateways, Netlify and similar hosts
web/_redirects holds the same rewrites for hosts that read that file:
- IPFS subdomain gateways (
https://<cid>.ipfs.<gateway>/) and DNSLink sites. - Netlify and Cloudflare Pages.
Some limits apply:
- Existing files win. A rule applies only when no file matches the path, and no rule uses
!, which IPFS gateways reject. The old.htmladdresses therefore answer directly, and the app moves them to the clean address. - No path gateways. Path gateways (
https://<gateway>/ipfs/<cid>/) cannot serve the site, because its paths are root-absolute. - Headers on Netlify and Cloudflare Pages.
web/_headerssends the security headers and cache lifetimes of the nginx and Caddy configs, anddeploy/sync_csp.pykeeps its policy in step.- These hosts apply every rule that matches a path, and Cloudflare Pages joins two values of one header with a comma. So each header is set by one matching rule only.
- Paths without a Cache-Control rule, such as the app's own modules, get the host's default, which revalidates on every load.
- Error responses carry the security headers but no Cross-Origin-Resource-Policy, because
_headerscannot name every unknown path.
- Headers on IPFS gateways. Gateways send their own headers. Each page's
<meta>policy still applies, but blocking framing (frame-ancestors) needs a header. - Snapshots. A site published to IPFS is a snapshot: prerendered DAO pages in it do not change,
and newer DAOs use the generic
dao.html.
Security headers
Every response carries these headers, errors and redirects included:
| Header | Value |
|---|---|
Content-Security-Policy | the pages' <meta> policy plus frame-ancestors 'none' |
X-Frame-Options | DENY |
X-Content-Type-Options | nosniff |
Referrer-Policy | strict-origin-when-cross-origin |
Permissions-Policy | camera=(), microphone=(), geolocation=(), payment=() |
Strict-Transport-Security | max-age=63072000; includeSubDomains |
Cross-Origin-Opener-Policy | same-origin |
Cross-Origin-Resource-Policy | same-origin; cross-origin for the preview images /og/dao-*, /assets/img/ and /favicon.svg, so other sites can show them (/og/network.json stays same-origin) |
Both server configs and web/_headers contain the policy written by
tools/write_web_config.py --patch-csp. Whenever you change the configuration, run
python3 deploy/sync_csp.py again, or python3 deploy/sync_csp.py --check to check that they still
agree. On a server set up with server/install.sh, the content policy step does this for you
(INSTALL.md, section 6).
Caching and compression
| Files | Cache-Control |
|---|---|
pages, clean URLs, config.js, /favicon.svg, /manifest.webmanifest, /data/*.json | no-cache (revalidated on every load, so a new deployment or setting takes effect at once) |
the app's own JS, CSS and images under /js/ and /assets/ with the release's ?v= | public, max-age=31536000, immutable (on a server set up by server/install.sh, only the current release's ?v=; any other is no-cache); without ?v=, no-cache |
/assets/fonts/, /js/vendor/, files under /data/ named by their hash | public, max-age=31536000, immutable |
/og/ (preview images and network.json), /sitemap.xml, /robots.txt | public, max-age=3600 |
The release's pages name their scripts, styles and images with ?v= and 12 hex digits of each
file's hash (tools/version_assets.py writes it at build), so a new release is fetched at once and
an unchanged file is never fetched again. nginx and Caddy cache by it; web/_headers cannot see a
query, so Netlify and Cloudflare Pages revalidate those files.
The nginx file that server/install.sh writes keeps only the current release's ?v= for a year:
the installer copies it from web/BUILD.json on every run, a rollback included, and any other
?v= is revalidated. When the release or its assets version changes (an upgrade, a rollback, or the
same version rebuilt with other files), it also removes the old prerendered pages before it switches,
so the new release's own pages serve until the prerender job has written its pages again. The static files here, deploy/nginx/collab.conf and
deploy/caddy/Caddyfile, cannot know the version and keep any ?v= for a year. That is right
while the pages and the files they name come from one release, so on a static host:
- replace the published
web/as a whole, rather than copying the new release over the old one; - then run the prerender job.
Otherwise a prerendered page left from the old release would ask for its old ?v= URLs and get the
new release's files, which browsers and CDNs would then keep for a year. Under /data/, which the
prerender job writes:
- only
.png,.jsonand.txtfiles are served; - a file not written yet answers
204withno-store, and the page then reads the chain instead; - anything else is not found.
Vendored files never change in place. A new version of a library gets a new file name (as
ethers-6.17.0.min.js has), because browsers keep the old one for a year.
Text responses are compressed: gzip on nginx (it also serves a file.gz that sits next to
file), and zstd or gzip on Caddy. Serve .js as JavaScript, .json as application/json,
.wasm as application/wasm, .webmanifest as application/manifest+json and .woff2 as
font/woff2; both configs and web/_headers do.
3. DAO pages, preview images, the sitemap and the member network
tools/prerender.py reads every DAO from the chain and writes these into the published web/
folder, or into --out (on a server set up with server/install.sh, the prerender job's own
folder; see Static pages, data files and prices):
dao/<slug>/index.html. It is a copy ofdao.htmlwith the DAO's own head:- the title, the DAO's name followed by "COLLAB on GenesisL1";
- a description of about 155 characters, taken from its on-chain description after checking that description against its keccak-256 commitment;
- the canonical URL
/dao/<slug>, Open Graph and Twitter tags; - JSON-LD with an Organization and a BreadcrumbList.
Its body also starts with a small static summary in
<section id="prerender" data-prerendered>: the name, the description, members, proposals, the constitution version and links to the DAO's sections. The app removes that summary when it starts. Search engines and link previews see the DAO without running any JavaScript. A DAO the site settings hide getsnoindex, and stays out of the sitemap and the member network.dao/<address>/index.html. A page without scripts for the DAO's treasury address, whose canonical URL and immediate refresh lead to/dao/<slug>.og/dao-<address>.png. A 1200 × 630 preview image with the DAO's name, its treasury address and three facts, beside the DAO's molecule glyph. The glyph is the same one the site draws, becausetools/seo/glyph.pyis an exact port ofweb/js/core/glyph.js. A DAO with a logo (a 128 × 128 PNG read from its profile and checked against the hash the profile stores) shows its logo instead.og/dao-<address>-logo.pngis a 512 × 512 square used as the JSON-LD logo.tokens/index.htmlandtokens/<address>/index.html. The tokens list and a page for every token with a DeSciFi pool.sitemap.xml. The indexable pages, the documentation, every DAO and the token pages, each withlastmod. A DAO's date is its latest profile, constitution or proposal change.robots.txt.web/robots.txtwith itsSitemap:line on the--siteorigin, so a site on another domain points crawlers at its own sitemap.og/network.json. The member network the landing page draws: which DAOs share members, as counts. It holds no member's address, only the factory's and each DAO's treasury address ("The member network" below).pools/index.htmlandlocks/index.html. These are copies ofdescifi.htmlwhose head is the oneweb/js/pages/descifi.jssets for/poolsand/locks:- their title, description, canonical URL,
og:url, Open Graph and Twitter texts, and JSON-LD; - the texts are read from
descifi.json every run, and the body stays as it is.
Without them, a crawler that runs no script would see
/poolsand/locksas copies of/swap. They need no chain, so they ship in the repository and the release:python3 tools/prerender.py --static-pageswrites them;--static-pages --checkexits 1 when one is out of date.
Write them again after
descifi.htmlordescifi.jschanges, and afterwrite_web_config.py --patch-csp.deploy/sync_csp.py --checknotices a stale policy in them.- their title, description, canonical URL,
With --data DIR, it also writes the cache served at /data/: price.json (the L1 price from
CoinGecko), tokens.json, and each DAO's logo and description under logos/ and texts/, named by
their keccak-256 hash. When --out is not --web, it also writes every entry page with the site
settings of config.js written into it. Before a deployment, with a config.js that says
"deployed": false, it reads no chain and writes only what needs none: those entry pages, the
/pools and /locks pages, the static sitemap, robots.txt and the price.
The member network
og/network.json lets the landing page draw which DAOs share members without reading the chain's
logs in every visitor's browser. Its format is in Data formats.
A member is an account with voting units at the file's block: a share holder that is not the
treasury, not frozen and not excluded. That is how each share token counts its memberCount, and
how the landing page counts members when it reads the chain itself.
How the job reads it:
- Only new blocks. The job replays the share tokens'
UnitsChangedlogs, 10,000 blocks and 100 tokens per request (smaller ranges when a node refuses one). A cache keeps the last block read, its hash and every DAO's member addresses, so a run reads only the blocks after it. The first run reads every DAO's history and takes longer. - The cache stays private. It sits beside the lock file (
prerender.lockgivesprerender.members.json), with mode 0600, and never inside the web root: the job refuses a--lockthere. The default lock is incollab-prerender-<uid>in the temp folder, a folder only that user may enter (mode 0700; the job refuses a link, or another user's folder, there). The temp folder may be emptied on reboot, and the next run then reads all logs again. Give the cache a lasting place with--lock(see the cron line below). - Proven before it is written. Each DAO's member set is checked against its share token at
the file's block before anything is written: every account in it must be a member, and the set
must have
memberCountaccounts. A node that keeps no state for that block is asked at its newest block. A set from the cache that fails is read again from the logs once. If it still fails, as with a node that drops logs,og/network.jsonand the cache stay as they are and the job warns; the DAO pages, images and sitemap are written all the same. So are they when the logs cannot be read at all. - A cache that does not fit is ignored. That is a cache for another chain, factory or block hash (a reset chain), or a file another user owns or may write.
- Written when it changes. The file is rewritten when the block or the counts change, and keeps
its
generatedtime otherwise.
The checks need the marketLens address of the deployment, which config.js has. --no-network
skips the file, for an RPC without eth_getLogs: the file already there stays as it is, so delete
it to stop publishing it.
The landing page ignores a file larger than 512 KiB and reads the chain instead; the job warns when
the file is that large. The page also asks for /og/network.json only when it is served from
CFG.siteUrl, the --site-url of write_web_config.py. A mirror or a gateway at another origin
reads the logs of the last 160,000 blocks instead, and its caption says so. So set --site-url to
the address you actually serve from.
nginx, Caddy and web/_headers serve the file as application/json, with the security headers,
Cross-Origin-Resource-Policy: same-origin and Cache-Control: public, max-age=3600.
Requirements
The job needs:
- Python 3.9 or later.
- Pillow. On Debian or Ubuntu,
apt install python3-pil; elsewhere,pip install pillow. - keccak-256. It comes from OpenSSL 3.2 or later, or from pycryptodome (
apt install python3-pycryptodome, orpip install pycryptodome).
tools/seo/requirements.txt lists the pip packages. No other package is needed: the JSON-RPC
client and the small ABI decoder are part of tools/seo/.
The text is set in Inter and JetBrains Mono, read from web/assets/fonts/:
- FreeType reads the WOFF2 files directly when it is built with Brotli, as in Pillow's wheels and on Debian and Ubuntu.
- Otherwise fontTools converts them in memory (
pip install fonttools brotli). - If neither works, DejaVu Sans, or Pillow's built-in font, is used, and every run prints a warning.
Running it
On the server, from the release folder, with web/config.js already in place:
python3 tools/prerender.py --web /srv/collab/web --dry-run # what would change
python3 tools/prerender.py --web /srv/collab/webIt takes the chain id, the DAOLens address, the RPC endpoints and the site URL from
web/config.js, trying the RPC endpoints in order. Every value can also be given explicitly, as the
Deploy tab's Publish by hand shows:
python3 tools/prerender.py --rpc https://rpc.genesisl1.org --deployment deployments/29.json \
--site https://collab.genesisl1.com --web /srv/collab/web --out /srv/collab/webOn a server set up with server/install.sh, systemd runs the job
(Static pages, data files and prices):
- every 10 minutes, and whenever the admin page asks;
- as its own user,
collab-prerender, with--out,--dataand--lockin/var/www/collab/shared; sudo collab prerenderruns it by hand.
Elsewhere, run it every 10 minutes from cron, as a user that may write in the web root, where it
writes its pages and images, sitemap.xml and robots.txt; the web server only needs to read
them. Keep the lock, and with it
the member cache, in a folder that user owns outside the web root
(sudo install -d -m 700 -o <user> /var/lib/collab):
*/10 * * * * cd /srv/collab && python3 tools/prerender.py --web /srv/collab/web --lock /var/lib/collab/prerender.lock --quietAdd --data /srv/collab/web/data to write the /data/ cache too: the price, the token list, logos
and descriptions.
deploy/nginx/collab.conf follows a symbolic link inside the web root only when the link and its
target have the same owner (disable_symlinks if_not_owner), so a link that user plants cannot
publish another owner's file. A link in the root path itself, such as a current link in
/srv/collab/..., is not affected.
The job is safe to run that often:
- All or nothing. All chain data is read before anything is written. An RPC error while reading the DAOs ends the run with exit status 1 and changes nothing. When only the member network or the tokens cannot be read, their files and caches stay as they are, with a warning.
- Atomic writes. Files are written to a temporary file and renamed into place, so the web server never serves half a file.
- Only real changes. A file whose bytes would not change is not touched. An image is drawn again only when what it shows changes.
- No overlap. A run that finds another one still working exits with status 0 and does nothing. A lock file that is a link, or another user's file, is refused with status 2.
- Quiet output.
--quietprints only changes, warnings and errors, so cron mails only when something happened.
These options change what is written:
--pruneremoves the pages and images of DAOs the factory no longer lists, which happens only after a move to another deployment.--outwrites somewhere other than--web.--datawrites the/data/cache into that folder.--locksets the lock file, and the member and token caches beside it.--no-networkleavesog/network.jsonas it is.--tokens-budget SECONDSlimits how long a run reads the tokens' history (240 by default); the next run resumes.
Exit statuses are 0 for success, 1 for a chain or RPC problem (nothing written) and 2 for a configuration problem.
Every string from the chain is untrusted:
- Names and descriptions have control, zero-width and bidirectional characters removed, as
cleanTextdoes in the app, and are escaped for HTML and for JSON-LD. - A description is used only when its content matches its on-chain hash.
- A DAO's logo is opened only when its bytes match the hash its profile stores, only as a PNG and only up to 512 KiB, the most a profile holds. The chain checks only a logo's PNG header, so one the image decoder refuses gives the DAO its molecule instead.
When the page policy changes (after write_web_config.py --patch-csp), the next run copies the new
dao.html into every DAO page and the new descifi.html into pools/ and locks/.
The site's own preview image and the static sitemap
web/assets/img/og.png, the preview image of every other page, is drawn in the same style. Redraw
it after changing the drawing code with python3 tools/prerender.py --default-og, and check it with
--default-og --check. It needs no chain access.
The web/sitemap.xml in the repository and in the release lists only the indexable static pages and
the documentation. The job adds the DAOs and tokens on the server.
python3 tools/prerender.py --static-sitemap rewrites that static version, and web/robots.txt
with it.
The generated web/dao/ and web/og/ folders (with og/network.json) are never part of a
release, and must not be committed. web/pools/ and web/locks/ are committed and ship with the
release.
4. After deploying
Run
python3 deploy/check_site.py https://collab.genesisl1.com(before DNS points at the server, add--resolve collab.genesisl1.com:443:<server address>). It checks:- every clean URL, the redirects and the 404 page;
- that no hidden or development file is served;
- the security headers on pages, redirects and errors, with the CSP header equal to the page
policy plus
frame-ancestors; - caching, compression and content types;
robots.txtandsitemap.xml, and that the noindex pages say so and stay out of the sitemap;- that
/poolsand/locksanswer with heads of their own, while a pool and a lock owner usedescifi.html; - the first DAO pages in the sitemap with their JSON-LD and their 1200 × 630 preview images;
/adminwith its noindex header and, whereserver/install.shruns it, the admin service and the rules of/data/;/og/network.jsonwhen it is there: served as same-origin JSON, in its format, for the chain and factory ofconfig.js, with no address but the factory's and the treasuries'.
It exits with status 1 when a check fails. Add
--dao <slug>to check a particular DAO. On Netlify or Cloudflare Pages add--static-host, which accepts what such hosts do differently:- the old
.htmlfiles answer as they are; - DAO sections use
dao.html; - caching follows the host's default where
_headerssets no rule.
Status, logs and the collab command lists its options.
Paste a DAO link into a chat or a social network: the preview shows the DAO's name, description and image. Services cache previews, and the image URL carries a version (
?v=…) that changes with the image.Run the admin page's Check tab (
/admin/check) on the site's own configuration, or compare every address ofconfig.jswithdeployments/29.json.Create a small test DAO with short voting periods, run one proposal of each kind you plan to use through deposit, vote and execution, and check the treasury and profile pages.
Publish the SHA-256 checksums of the files you serve (
SHA256SUMSin the release), so others can verify they run the same interface or run it themselves from the release archive. The prerender job's output (dao/,og/,tokens/,sitemap.xmlandrobots.txt) is generated on the server and differs from the release by design.
Local development
See README.md, Local development and The web app.
tools/local_chain.py deploys everything to a local anvil chain that also uses chain id 29, with
local copies of the integrations it lists. It writes deployments/local.json and
web/config.local.js and never touches deployments/29.json. tools/serve_local.py serves the same
clean URLs as the server configs. To try the prerender job locally, write its output outside the
repository:
python3 tools/prerender.py --rpc http://127.0.0.1:8545 --deployment deployments/local.json --out /tmp/collab-prerenderBuilding a release archive
python3 tools/merge_css.py # folds any web/assets/css/parts/*.css into app.css
python3 tools/build_release.py --version 2.2.0The build lints the site, runs the tests, and refreshes the ABIs and the contract reference. It
then writes dist/collab-<version>.tar.gz, with SHA256SUMS inside and a .sha256 file next to
it. The archive is reproducible: entries are sorted, timestamps fixed and owners cleared.
Building a release describes every step.
The build then extracts the archive into a temporary folder and checks the checksums, the lint and
that the contracts compile from it. On a machine without network access for downloading the
compiler, pass Foundry flags through FORGE_FLAGS, for example
FORGE_FLAGS="--offline --use /path/to/solc-0.8.28". The build compiles a few files per solc run,
because one run over every test at once can need more than 5 GB of memory with via-IR;
FORGE_BATCH sets how many (4 by default).