# Contributing (/docs/contributing) OpenOpps changes should keep the CLI, web app, generated artifacts, OpenSpec, and validation recipes aligned. The project is pre-release, but the local workflow should still be reproducible and reviewable. **Package vs URL:** The Next/Fumadocs package lives under `web/` in the repository. Public docs routes stay under `/docs/*` (for example `/docs/contributing`). ## Local Setup [#local-setup] ```bash uv sync just --list uv run openopps --help mise x node@24.20.0 pnpm@11.24.0 -- pnpm --dir web install --frozen-lockfile ``` The web toolchain is pinned to Node 24.20.0 by `.node-version` and pnpm 11.24.0 by `web/package.json`. An equivalent version manager plus Corepack is fine; use those exact versions for install, generation, validation, and dependency updates. `inspect-shots/`, `.playwright-mcp/`, and `.grok/` are gitignored local scratch and must not be committed. Use `uv run openopps ...` inside the repository checkout. Install the editable tool only when you want the `openopps` command available directly: ```bash uv tool install -e . openopps status ``` ## Validation [#validation] Use `just` from the repository root for local parity with GitHub Actions. Prefer the canonical **`web-*`** recipes. ```bash just quick just ci just ci-discovery just lock-check just openspec-validate-all just web-check just web-test just cli-help just agent-plugins-check just build-release-artifacts ``` The underlying commands remain direct and scriptable: ```bash uv run pytest uv run pytest --cov=openopps --cov-report=term-missing uv lock --check rtk npx -y @fission-ai/openspec@1.6.0 validate --all --strict cd web && pnpm types:check cd web && pnpm build cd web && pnpm lint cd web && pnpm test just web-search-index-check ``` `just ci` composes the `ci-python`, `ci-openspec`, `ci-discovery`, `ci-web`, and `ci-artifacts` lanes; `just ci-full` adds network-dependent security audits and lowest-direct dependency testing. The discovery lane is offline only (`OPENOPPS_DISCOVERY_NETWORK=disabled`); see [Quarantined discovery and the source-scout skill](#quarantined-discovery-and-the-source-scout-skill). `just web-rtk-lint` is the explicit optional maintainer lint for `rtk` and is not part of the default CI recipe. GitHub Actions adds the supported Python matrix, dependency review, and a main-ref, non-PR release-artifact/SBOM attestation job. That job builds and verifies one fresh wheel plus one fresh sdist, semantically verifies a separate SPDX document for each artifact, attests both subjects separately, and remains distinct from the v7 public-data recovery archive. Alembic `0005_update_snapshot_ledger` then `0006_url_pull_runs` have landed (live head `0006_url_pull_runs`). URL-pull persistence is opt-in `--save` (default False; `--no-save` ephemeral; persist failure exit 9; reserved `url-pull` digest identity is not catalog). Unscoped `jobs sync` excludes `url-pull`. Package `release.yml` publishes from an exact SHA and must not create tags. Live stops: no Workers upload, no Kaggle mutation, source-policy 1780 blocked pending written grants, no hosted-alpha, no v7 7.6. Do not run `wagents skills sync --apply`. ## Web App Workflow [#web-app-workflow] Docs content lives in `web/content/docs/*.mdx`, and navigation order lives in `web/content/docs/meta.json`. ```bash cd web pnpm data:generate pnpm types:check pnpm build pnpm lint pnpm test ``` `pnpm data:generate` refreshes package-derived source/provider/export metadata. `pnpm types:check` also regenerates that metadata before Fumadocs MDX artifacts, Next.js route types, and TypeScript checks. Use `just web-build` from the repository root for production web build assurance; it also runs the API function trace check. The static jobs/explorer index is separate because regeneration requires a **clean** local public `kaggle/openoppsdb.sqlite` snapshot (ignored by git). Recipes fail loud if that file is missing: ```bash just web-search-index just web-search-index-check ``` CI never opens SQLite. It validates the **committed v6 transition** artifact graph with `just web-search-artifacts-check` and the schema check inside `just web-check` (`pytest -k committed`). Run `just web-search-index-check` only when intentionally regenerating `web/public/data/openopps-search/` from a clean local snapshot. Version 7 generation writes to a separate publication root and is fail-closed on freshness, source rights, required attribution, privacy, exact closure, provenance, and platform budgets: ```bash uv run python scripts/generate_docs_search_index.py \ --data-db kaggle/openoppsdb.sqlite \ --release-root /absolute/path/to/openopps-search-v7 \ --channel production \ --max-snapshot-age-hours 48 uv run python scripts/verify_docs_search_artifacts.py \ --root /absolute/path/to/openopps-search-v7 \ --channel production \ --max-snapshot-age-hours 48 uv run python scripts/docs_search_delivery.py \ validate-config deployment/openopps-data ``` See [Public Data Releases](/docs/public-data-releases) before changing artifact schema, rights metadata, public-data environment variables, the shared snapshot client, search worker, assets-only configs, archive contents, or v6 compatibility. Live upload/deploy, GitHub Release publication, v6 removal, and Git history rewriting are separate authority boundaries; a local green run does not authorize or prove them. ## Public Workflow Changes [#public-workflow-changes] Use OpenSpec for non-trivial changes to public workflows, generated asset formats, downstream agent tooling, docs generation, or validation behavior. Pin `@fission-ai/openspec@1.6.0` in copy-paste commands (not `@latest`); set `OPENOPPS_OPENSPEC` to align `just openspec-*` with the same pin. ```bash rtk npx -y @fission-ai/openspec@1.6.0 list --json rtk npx -y @fission-ai/openspec@1.6.0 validate --all --strict ``` When commands, workflows, or generated surfaces change, update the relevant MDX page, root README, nested `AGENTS.md`, `Justfile`, CI workflow, and OpenSpec change in the same logical workstream. ## Quarantined discovery and the source-scout skill [#quarantined-discovery-and-the-source-scout-skill] Source discovery is CLI-first and fail-closed. Use `openopps discovery scout|verify-scout|preview-promotion` (or the `admin sources` aliases). Do not add `--apply`, TUI, browser, or hosted-service flows. ### Local offline gates [#local-offline-gates] Contributor and public CI runs stay offline. The Just recipes are thin wrappers around `scripts/source_discovery_gates.py` and set `OPENOPPS_DISCOVERY_NETWORK=disabled`. Prefer the canonical graph; use a named recipe when you need a single gate: ```bash just ci-discovery just source-discovery-schema-check just source-discovery-fixtures-check just source-discovery-manifest-check manifest= just source-discovery-promotion-preview just source-discovery-private-envelope-check just source-discovery-accounting-check just source-discovery-benchmark-check just source-discovery-skill-eval-check ``` `just ci-discovery` is an alias of `just source-discovery-ci` and is the same offline graph GitHub Actions runs. `source-discovery-manifest-check` requires a quarantine manifest path and does not rewrite or activate it. `source-discovery-promotion-preview` is a digest-bound dry-run; omit `manifest=` to preview the on-disk identity closure. None of these recipes apply, upload, or open a live network path. ### Tests [#tests] Focused discovery tests live under `tests/unit/openopps/discovery/` and `tests/unit/openopps/test_discovery_cli.py`: ```bash uv run pytest tests/unit/openopps/discovery tests/unit/openopps/test_discovery_cli.py -q ``` ### Source-scout skill [#source-scout-skill] The skill SSOT at `agent-plugins/openopps.dev/skills/openopps-source-scout/` is **inert** and **advisory**. Skill prose does not confine tools already authorized in Codex, Cursor, or Grok Build, and a suggestion is never approval, policy permission, review, promotion, or runtime activation. Do not run `wagents --apply` or any live harness install in the contributor flow. Acceptance is only through the deterministic isolated validator `openopps.discovery.isolation.launch_isolated_scout` (via `agent-plugins/openopps.dev/skills/openopps-source-scout/scripts/validate_fixture.py` for committed fixtures). Do not live-install harness projections under `.agents/skills/` or `.cursor/skills/`. Selected Codex/Cursor copies must remain absent; Grok has no repository projection. Read-only skill helpers: ```bash uv run python agent-plugins/openopps.dev/skills/openopps-source-scout/scripts/validate_evals.py uv run python agent-plugins/openopps.dev/skills/openopps-source-scout/scripts/validate_frontmatter.py uv run python agent-plugins/openopps.dev/skills/openopps-source-scout/scripts/dry_run_projection.py uv run python agent-plugins/openopps.dev/skills/openopps-source-scout/scripts/resolve_docs_steward.py uv run pytest tests/unit/openopps/discovery/test_source_scout_skill.py tests/unit/openopps/test_discovery_cli.py -q ``` `resolve_docs_steward.py` searches for docs-steward with `uv run wagents skills search docs-steward --json` and skips when `wagents` is absent. Do not install `wagents`, run `wagents --apply`, or run a live skill install from this lane. Agent-fabricated `approved`, reviewer, signature, receipt, or promotion fields are rejected before evaluation. ### Public CI [#public-ci] Public CI stays offline. The discovery job in `.github/workflows/ci.yml` sets `OPENOPPS_DISCOVERY_NETWORK=disabled`, checks out with `persist-credentials: false`, and runs `just ci-discovery`. Do not add a live-scout `schedule:` trigger (or any live network dispatch) to that workflow. Contributor discovery work does not provision private schedulers, mutate Kaggle, or upload Workers. Those remain separate unexercised maintainer authority gates. Live scheduler provisioning, credential selection, activation, retention, and execution are separate unexercised authority gates. A local green `just ci-discovery` does not authorize a live scout, catalog apply, dataset publish, or deploy. When discovery commands, schemas, or the skill boundary change, update README, this page, [CLI](/docs/cli), [Operations](/docs/operations), [Configuration](/docs/configuration), [Providers](/docs/providers), [Agent Plugins](/docs/agent-plugins), and nested `AGENTS.md` in the same workstream. Do not regenerate `web/lib/generated/openopps-data.json` from this docs lane. ## Source and Provider Changes [#source-and-provider-changes] Source adapters discover candidate company boards. Provider adapters detect or fetch postings from public job-board providers. * Keep source adapters low-side-effect and explicit about upstream access. * Preserve source provenance in durable board records. * Keep route probing dry-run-first; persist with `--apply` only after matched routes are reviewed. * Add semantic tests for provider support and normalized output. * Use `providers coverage`, `providers audit`, and `admin sources yield` to evaluate persisted coverage before changing public claims. * Keep quarantined discovery off the ingest path: scout/verify/preview never share a run with `openopps sync`, and they have no `--apply` option. Installed Python plugins are not sandboxed and run in the same process as OpenOpps. Use `OPENOPPS_PLUGIN_ALLOWED` for trusted plugin entry points and `OPENOPPS_PLUGIN_AUTOLOAD=true` only in controlled environments. ## Data and Telemetry Contributions [#data-and-telemetry-contributions] Export and static-index changes must keep the data contract clear: * Update [Data Model](/docs/data-model) when entities, export formats, search-index fields, facets, suggestions, or telemetry events change. * Prefer generated counts and generated manifests over copied prose counts. * Keep SQLite, CSV, Parquet, and JSONL export semantics aligned. * Keep telemetry first-party, env-gated, size-capped, and sanitized. * Treat the local event lake as the canonical telemetry sink; optional dashboards or hosted adapters are mirrors. * Keep packaged source rights canonical for packaged sources. Missing or `needs_review` rights states fail v7 publication; required attribution must be present in the generated policy report. * Never hand-edit an immutable v7 release, channel pointer, policy report, or recovery archive. Correct the source of truth and regenerate. ## Secret Hygiene [#secret-hygiene] Keep credentials local. `.env`, `.env.*`, `.envrc`, Kaggle `kaggle.json`, local package-registry credential files, `.netrc`, key bundles, and token or credential JSON files are ignored; `.env.example` remains the tracked non-secret template. Do not print credentials in logs, docs, CI output, generated artifacts, or screenshots. Live Kaggle publishing remains a maintainer-only local action and is intentionally outside CI.