Build and release SOP

Mandatory use: Follow this SOP before changing dependencies, lockfiles, build launchers, generated-manifest formats, machine bundles, signing, packaging mechanics, continuous integration, or release automation. Routine application development that only consumes the existing tools follows development-lifecycle.md and does not become release work.

Governing functional baseline

Preserve complete source, package, model, document, message, log, diagnostic, process, and tool content. Never truncate, clip, tail, elide, shorten, or silently discard it. Ordinary build, package, cache, progress, artifact, test, or runtime behavior must not depend on byte counts, byte identities, byte budgets, hashes, digests, checksums, sizes, or verification receipts.

Security, hardening, provenance, integrity, isolation, restrictive admission, signing, trust, freezing, mutation guards, and fail-closed behavior require the user's express authorization for the exact control and scope. A useful secure option defaults to false, and ordinary development and packaging remain fully functional while it is false. Credential and secret protection, non-optional law and platform safety, explicit privacy choices, unavoidable operating-system or external-protocol requirements, and rejection of genuinely malformed selected artifacts remain at their exact owners.

Required outcome

An Arcane build must be reproducible from a fresh checkout when that selected output requires it, follow the active dependency authority and committed lock contract, preserve required source formats on supported operating systems, surface the first failed command, keep development output distinct from a production claim, and avoid multiplying invariant work across apps, models, assets, targets, or platforms.

Current development profile

ARCANE currently has no staging or production environment. Ordinary development does not run tests, checks, linters, validation builds, package regeneration, signing, or promotion work unless the user explicitly selects that operation. A selected release output may run only the verification that output requires.

• Do not require full package regeneration or a clean full-platform build for every feature edit.
• Do not call development or demo output a release candidate.
• Keep production and signing generation dormant unless the user expressly authorizes the exact selected output, mode, and destination.
• Do not infer clean-build, publication, signing, security, or evidence work from a source implementation, commit, push, or handoff.

Procedure

1. Preserve and identify the build boundary

• Record the exact command, platform, Node/npm versions, selected signing mode only when signing is authorized, and the complete first error.
• Preserve the relevant npm or build log before clearing caches or dependencies.
• Identify whether the failure occurs during dependency installation, source generation, compilation, signing, packaging, publication, or final verification.
• Follow docs/debugging.md for any failing or unexpected build.
• Resolve repairable checkout prerequisites centrally before authorized dependency installation, compilation, or packaging. This includes the current Microsoft NT path-depth/tool limitations, upgraded-checkout migration, required tool availability, and selected Git LFS asset hydration.
• Do not let a lower-level compiler or app packager become the first component to discover a repairable prerequisite. Every interactive Microsoft NT first-run or missing-build path must converge on the repository-root developer setup; if setup cannot repair a prerequisite, it reports that failure and preserves the complete error in the setup log.
• Treat a Microsoft NT WebView2 virtual-host change as a persistent browser-origin boundary, not a URL-only edit. The current transition to https://os.arcane intentionally does not migrate https://arcane.local storage: preserve the legacy profile untouched, validate the new origin as a separate scope without clearing existing target-origin state, and do not claim an in-place data-preserving upgrade. Record migration as deferred in the handoff. Any future migration must satisfy the acceptance criteria in application-data-isolation.md before the changed host is distributed as a storage-preserving upgrade.

2. Complete the work-amplification review

Before changing setup, build, package, test, model, app, or runtime work:

1. State every cardinality and loop multiplier across platforms, architectures, applications, models, assets, targets, test cases, retries, and processes.
2. Enumerate expensive downloads, extraction, authorized dependency installation, compilation/linking, package generation, model pulls, service setup, and tests. Estimate representative invocation count, allocation behavior, and elapsed time without byte accounting.
3. Classify each operation as invariant, reusable, batchable, or necessarily per item. Hoist invariant work, reuse results only while their semantic inputs and invalidation rules remain unchanged, and batch compatible work.
4. Measure a representative cold path and compare operation counts and elapsed time with the estimate.
5. Eliminate redundant work before improving progress display. Necessary long work reports phase, current item, completed/total item count, elapsed time, heartbeat, and an honest ETA when available.

OS-owned artifacts and preparation run once per compatible selected OS output, not once per application. Applications own their prebuilt, independently versioned output; OS setup consumes it through the functional package contract. Optional hashing, signing, integrity, provenance, trust, hardening, security tools, smoke tests, and restrictive admission stay inactive without exact authorization.

Document every intentional repetition with the changing semantic input, why batching or reuse is unavailable, the multiplier, and measured cost. Existing loop placement is not a justification.

3. Preserve the responsiveness invariant

Every setup, build, package, test, model, app-install, and runtime front end must acknowledge immediately before potentially blocking work. A graphical launcher creates or activates its visible status surface first. A command-line interface (CLI) prints a start record, current phase, and durable log location when available before its first network, disk, compiler, child-process, or model wait.

• Keep network, disk, compilation, child-process, and model work off the UI-creation and foreground thread. Use event-driven background workers or genuine asynchronous I/O. An async wrapper around synchronous blocking code is still blocking.
• Bound parallel downloads, workers, compilers, child processes, queue depth, memory, and disk pressure. Apply backpressure instead of unbounded fan-out.
• Emit progress with phase, current item, completed/total item count, and elapsed time. Emit heartbeat events during opaque waits.
• Support cancellation and safe retry or resume when the operation supports it. Preserve prior output and keep incomplete output separate when practical.
• Give every task an owner that awaits, joins, or supervises completion. Surface every failure through visible status, a durable log, and nonzero CLI or launcher result. Unobserved fire-and-forget work is prohibited.
• Serialize only an atomic-install, shared-state, or unavoidable external-protocol boundary. Signing is serialized only when exactly authorized. Show visible status before entering the narrow boundary.

Focused launcher and work-graph test source should cover acknowledgement order, foreground responsiveness during a slow representative operation, concurrency limits and backpressure, progress or heartbeat delivery, cancellation, retry/resume, worker-error propagation, and the serialized boundary. Do not execute it unless verification is authorized.

4. Preserve dependency and lockfile behavior

Dependency installation or execution requires the authority granted by the active user-wide dependency rules. An existing manifest or lockfile is not by itself permission to install or execute a package.

When dependency work is authorized:

• use the repository's declared package manager and committed lockfile;
• use npm ci when consuming an existing lock and npm install only when intentionally updating dependency resolution;
• keep every HTTP(S) resolved lockfile source on the explicitly approved public registry, currently registry.npmjs.org over HTTPS;
• never commit workstation, proxy, mirror, credentialed, private, organization-internal, or environment-specific registry URLs;
• do not copy node_modules or another installed tree as a silent substitute for declared resolution;
• use npm ci --offline only when the authorized lock can be satisfied from the existing repository-local cache, and report an incomplete cache honestly;
• do not commit npm caches, node_modules, credentials, tokens, certificates, or private registry configuration; and
• stop on a nonzero dependency result before compilation or packaging.

npm run verify:package-locks is an opt-in diagnostic. Run it only when the user explicitly requests that check or when a selected release output requires it. Do not add byte counts, hashes, digests, receipts, provenance, trust, or integrity gates to ordinary dependency preparation.

5. Preserve canonical files across platforms

• Use .gitattributes for source-format and line-ending behavior that current platform tools actually require.
• Machine-bundle arcane-bundle.json files must remain LF-only through machine_bundles/*/arcane-bundle.json text eol=lf.
• Microsoft NT batch launchers remain CRLF; shell scripts remain LF.
• Treat installed third-party dependencies as opaque package-manager content. Do not rewrite them or apply repository formatting to them.
• When the user explicitly authorizes cross-platform source verification, inspect the Microsoft NT checkout with its actual core.autocrlf behavior, not only a Git blob or Unix checkout.

6. Keep application releases application-owned

• During rapid development, do not regenerate every checked-in public package merely because shared source changed. Generate only the selected app package needed by the request.
• For an explicitly selected application release, keep that application's files, shared-runtime dependencies, product policy, adapters, and reviewed document sources owned by that app.
• Regenerate only the selected checked-in package with npm run app:package -- <app-id>. Do not use --all or run unrelated application suites as part of an Arcane OS output.
• Do not hand-edit generated catalog, release, content, or inventory manifests. Review the selected source and generated changes together, preserve complete content, and exclude credentials, protected data, private source, and unrelated files.
• Run npm run app:check -- <app-id> only when the user explicitly requests that verification or when the selected release output requires it. The same rule governs the app's behavior suite.
• An Arcane OS build consumes the selected app-published dist/<app-id>/ARCANE_APP_RELEASE.json and its complete declared package through the functional package contract. It may reject a genuinely malformed manifest, unreadable member, unsafe archive path, link, device entry, or incompatible public contract, but it must not introduce byte counts, hashes, receipts, provenance, trust, signing, or restrictive admission without exact authorization.
• An external app remains versioned, built, tested, licensed, and published by its own canonical repository. Arcane must not execute app-owned source, workflows, adapters, or tests while consuming the selected external package. The source repository, normalized descriptor, target list, and complete package stay attributable to that publisher; optional authenticity or signing work requires exact authorization.
• Edit machine_bundles/arcane-os-machine-bundle/arcane-external-apps.lock.json deliberately when the functional external-app contract changes. Do not regenerate it from a release, copy an external repository into Arcane, or use a duplicate clone/worktree/archive as active source.
• For local AI, preserve the app's sole authored localAIModelPolicy and complete app-owned Modelfiles in the selected package. The native Core descriptor consumes the policy; browser application code must not duplicate it. Core reports the complete actual local catalog without app-owned byte/hash/provenance filters. Optional verified_only:true behavior is inactive unless the user expressly authorizes that control with secure:true.
• Arcane OS setup must not compile application executables, rebuild app-owned packages, import an app adapter/generator/source verifier, or run an app's business suite. If an app needs an app-specific native host, its own selected output process produces that host.
• An explicitly selected Arcane OS release output uses npm run release:check:arcane-os only when that verification is authorized or required by the output. An application's suite remains separately owned.

7. Preserve failures at every build step

• A launcher must stop after every nonzero subprocess exit code, including negative Microsoft NT process-status values.
• Do not use a positive-only if errorlevel 1 check where a tool can return a negative status. Compare the captured status explicitly against zero.
• Never continue from a failed dependency install into compilation, signing, packaging, or publication.
• Do not suppress, overwrite, or reinterpret a failing exit code merely to complete a build.

8. Keep optional signing and publication separate

Ordinary development produces a functional development output without signing, provenance, attestation, trust, or production publication. Label an unsigned development output honestly.

Signing or publication begins only when the user expressly authorizes the exact selected output, signing mode, identity, and destination. Within that selected operation:

• use only the already authorized platform identity and provider;
• keep development signing distinct from production signing;
• protect credentials, tokens, certificates, and private keys;
• comply with unavoidable external-platform requirements at that platform boundary; and
• report exactly what was produced or published and what remains unverified.

The existing Microsoft NT provider names and platform-specific signing commands remain compatibility surfaces documented in the developer command reference; their presence does not authorize execution or make them ordinary build gates. Archive transport remains separate from executable signing. Neither signing nor archive verification may become a byte/hash/receipt requirement for ordinary source, development build, package, or launch work.

9. Selected-output workflow

When the user explicitly selects a dependency, cross-platform build mechanism, package, or Arcane OS release output:

1. prepare only the selected workspace and target;
2. use only dependencies whose installation and execution are authorized;
3. preserve the first nonzero failure and complete log;
4. compile or package the selected output;
5. run only verification explicitly requested or required by that selected output; and
6. report the output location, development/release classification, and checks not run.

For a selected Microsoft NT machine-bundle build, the documented commands remain:

npm ci --prefix machine_bundles/arcane-os-machine-bundle
npm run build:distribution:windows:unsigned-local-test --prefix machine_bundles/arcane-os-machine-bundle

Dependency installation still requires the active user-wide authority. A signed production command may run only with exact signing and production-output authorization.

Versioning the machine bundle

The active source directory is permanently machine_bundles/arcane-os-machine-bundle. Its name must never contain a release number, and advancing the bundle version must never create, copy, or select a version-suffixed source directory. arcane-bundle.json is the authoritative runtime version; package metadata, generated runtime declarations, and platform manifests must agree with it. Versioned names belong only to generated artifacts such as archives, installers, and evidence tied to an exact build.

Git is the release history; the current tree is not an archive. Advance the canonical manifests, documentation, and selected generated outputs in place, and remove superseded copies in the same change. Do not retain prior version directories, old validation documents, rollback copies, or obsolete archives merely for historical reference. Historical material may remain only for a documented current runtime, legal obligation, or explicitly selected audit scope. A changelog entry and Git history are the default record of replaced release state.

When cleanup is expressly selected as part of an Arcane OS or app release, use npm run cleanup:preview, inspect ignored build/cache output, and remove only exact obsolete paths that no current consumer references. If a dormant checkout can skip an intermediate change and conflict with a later deletion, retain the smallest documented compatibility shim; a release-number advance does not prove migration uptake. Never require users to stash, reset, or manually merge generated output to receive routine cleanup. Run npm run maintenance:gc only when local Git maintenance is separately selected. Never automate git clean -fdx, aggressive object pruning, history rewriting, or remote LFS deletion; those operations can destroy user work or shared recovery state and require separate explicit authorization.

Microsoft NT developer preparation never retires an existing canonical dist/nt release merely because a new build was requested. It preserves the prior working tree until the selected replacement operation succeeds. Genuinely malformed state is rejected, and ambiguous external, installed, reparse, active, or transaction-owned state is preserved for explicit resolution rather than deleted. Publication locks, signing state, and restrictive admission are used only when the user expressly authorizes that exact output and control.

Application versions are independent of the machine-bundle version. An in-repository app derives its version from apps/<id>/arcane-package.json; a source-free external app derives it from the normalized descriptor in arcane-external-apps.lock.json, and its selected ARCANE_APP_RELEASE.json must agree. arcane-apps.json remains the version-free native application and capability catalog. Build tools preserve both bundleVersion and app.version; a bundle-version change must never implicitly bump or relabel an application.

Prepare an upgraded checkout before selected output work

The supported source-migration command is npm run machine:checkout:prepare from the repository root, or npm run prepare:checkout from the permanent machine-bundle directory. It considers only retired version-suffixed machine-bundle siblings whose suffix is a canonical three-component version. Similar backup names and every directory outside that exact pattern are left alone.

Preparation is Git-backed and preserves ambiguous state. The repository root, machine_bundles, canonical source directory, candidate directories, and every descendant must resolve as real in-tree entries. Before removing anything, the migration inspects every candidate and refuses the operation when it finds a tracked path, nonignored untracked work, a symbolic link, junction, reparse point, special entry, or an ignored file outside recognized generated directories. Only empty directories and ignored residue inside known build/cache directories are disposable. A refusal preserves the legacy directories for the developer to inspect, move, commit, or remove manually.

setup-developer.bat and npm run setup:developer own this preparation before authorized dependency work or compilation. When the user explicitly selects npm run check:dev, that command may inspect the same migration state. The direct Microsoft NT build-windows.bat launcher owns preparation before npm ci or compilation. This migration never selects a source by folder version: the canonical source directory remains machine_bundles/arcane-os-machine-bundle, and arcane-bundle.json remains the sole machine-version authority.

Until current generated-tree operations are long-path-safe, Microsoft NT builds use a real short checkout such as C:\ArcaneOS. The documented 46-UTF-16-code- unit root limit reflects a current Windows PowerShell 5.1 copy/cleanup limitation. Setup and direct build launchers must report a deeper-root conflict before asset hydration or authorized dependency installation. This is a platform/tool limitation, not a byte, content-identity, security, or trust gate.

When advancing the machine-bundle version:

• add the new CHANGELOG.md section, then run npm run machine:version -- <major>.<minor>.<patch> so the central arcane-bundle.json authority and its required npm/README mirrors change together;
• run the normal selected machine build so runtime and platform declarations are generated from the central manifest; run npm run machine:version:check only when that check is explicitly requested or required by the selected output;
• derive versioned artifact, archive, and installer names from the authoritative manifest instead of the source-directory name;
• keep all root scripts, workflows, tests, and documentation pointed at the permanent source directory;
• confirm the wildcard .gitattributes rules apply and include the stable lockfile path in CI cache inputs;
• run dependency installation only with active dependency authority, and run the package-lock or Arcane OS release checks only when explicitly requested or required by the selected output;
• keep credentials, secrets, and private key material out of tracked output.

Unified developer setup

• setup-developer.bat and tools/setup-developer.ps1 are the authoritative Microsoft NT developer bootstrap. The interactive batch launcher opens the freshly built Provisioner after success; the PowerShell and npm forms remain build-only for scripted callers.
• machine_bundles/arcane-os-machine-bundle/build-windows.bat is a compatibility entry point that delegates to the repository-root setup. A missing-output start-provisioner.bat invokes that same setup before the original launcher retries the Provisioner. Nested launch ownership prevents duplicate windows. Interactive launchers must not implement an alternate dependency, asset, or build sequence.
• Arcane - Rebuild Windows and Open Provisioner.bat is the prepared-checkout rebuild path. It checks the Microsoft NT checkout limitation and declared package registries before dependency mutation, installs the authorized root and machine-bundle dependency trees once each, runs the canonical selected Microsoft NT development build once, and launches only that new output. Its explicit optional --offline mode applies npm ci --offline to both trees and reports an incomplete cache honestly; an online failure never silently selects offline mode. The launcher never retries a failed rebuild through full setup.
• The bootstrap must report an unsupported Microsoft NT checkout root before network, asset, dependency, or build work. This is the documented Windows PowerShell path/tool limitation described above, not a content-size gate.
• Before asset preparation, authorized dependency installation, or compilation, the bootstrap may inspect functional source and current-state prerequisites: the machine version, declared registries, Windows host/source contracts, required SDK and .NET host paths, WebView2 package metadata and cache path, readable generated-release metadata, and filesystem topology. Reject genuinely malformed selected input, preserve ambiguous state, and do not turn this preparation into byte, hash, identity, receipt, provenance, signing, or restrictive-admission policy.
• OsSetup is the ordinary setup scope. AppDevelopment remains an explicit application-development scope. Asset preparation must preserve complete selected content, expose item progress and elapsed time without byte accounting, and surface every failure. It must not initialize a signer, enforce hardening, or run tests or checks unless the user expressly selects that exact work.
• Install system packages only when dependency authority is active, using the documented public WinGet identifiers. Pin a major toolchain when Arcane's functional build contract requires that version.
• On Microsoft NT, preserve an existing pnpm hardlinked dependency tree under ignored tmp/ before switching that checkout to npm ci; npm may return a negative EPERM status while unlinking pnpm hardlinks.
• Use an ignored repository-local npm cache for unified setup so corrupted or permission-restricted user-global cache state does not control the build.
• Keep focused test source for setup ordering, package identifiers, exit-code propagation, shared-work cardinality, selected per-item work, immediate acknowledgement, bounded concurrency, progress, cancellation, and observed worker errors. Do not execute it automatically; execution requires the user's express verification request or a selected output that requires it.
• Follow docs/developer-reference-sop.md: every new or changed system tool, developer launcher, root npm command, or machine-bundle npm command must update docs/developer-commands.md in the same change.

Repository Microsoft NT launchers must acknowledge work before it begins, preserve a nonzero build or launch status, and never fall through to stale or missing output. The clearly named first-setup, rebuild-and-open, and open-current launchers provide the supported double-click choices; failures remain visible with the applicable setup log or recovery path.

Required handoff

Report:

• the selected application, target, output, and command;
• the dependency authority and declared registry or lockfile behavior used;
• relevant line-ending and upgraded-checkout handling;
• generated outputs and their owning source;
• launcher exit-code behavior and the complete surfaced error when work failed;
• any signing, security, or publication control expressly selected by the user;
• cardinality, loop multiplier, operation count, elapsed time, and repeated work;
• how necessary long-running work acknowledges immediately, exposes progress, owns background work, bounds concurrency, supports cancellation where practical, and surfaces errors;
• generated output location and final exit code; and
• tests, checks, builds, or production-only work run, not run, and any exact blocker.