feat(proof): Proof challenge + dynamic topics + RLM judge digest pin - #209
Conversation
Introduce the fifth live challenge (`proof`, 2000 bps) with operator-published signed topics, a digest-pinned RLM judge (empty digest fails closed), and throwaway trust-root re-sign of the five-row emission split. Co-authored-by: Mathis <echobt@users.noreply.github.com>
Keep Proof HTTP tests compiling under -D warnings, list the proof arena on the site, and match the five-row trust-root emission set. Co-authored-by: Mathis <echobt@users.noreply.github.com>
|
@greptileai review |
primeorder 0.14.0 (via p256/dcap-qvl) pulled wnaf 0.14.0, yanked today in favor of 0.14.1. Smallest lockfile bump to keep cargo-deny green. Co-authored-by: Mathis <echobt@users.noreply.github.com>
|
CI after wnaf bump: deny cleared; failure is unrelated flake |
Greptile SummaryThis PR adds the Proof challenge service, scoring, signed topics, and deployment integration. Validation reproduced three failures that block reliable production operation: submissions can be credited to an arbitrary miner and trigger live scoring without a bound payer, the service never emits signed epoch leaves to a gateway, and topic availability is permanently evaluated at epoch zero. These issues undermine score integrity and prevent Proof results from participating in epoch sealing. Confidence Score: 1/5The change is not safe to merge until submission identity and payer binding, epoch-aware topic evaluation, and signed-leaf delivery are implemented. Direct execution reproduced arbitrary-hotkey attribution, confirmed that the running service has no reachable emission handoff, and demonstrated that the same signed topic fails at epoch zero but succeeds during its configured validity window. Files Needing Attention: crates/proof-http/src/lib.rs needs authenticated miner and payer binding; bins/proof-challenge/src/main.rs needs live epoch wiring and production leaf-emission orchestration.
|
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| let hotkey = parse_hex64(&body.miner_hotkey, "miner_hotkey")?; | ||
| let artifact = parse_hex64(&body.artifact_digest, "artifact_digest")?; | ||
| let _lium_present = headers | ||
| .get("x-lium-api-key") | ||
| .and_then(|v| v.to_str().ok()) | ||
| .is_some_and(|s| !s.is_empty()); |
There was a problem hiding this comment.
POST /v1/submissions accepts a caller-selected miner_hotkey without verifying that the caller owns it or binding it to a Lium payer. A request using an untrusted authorization value, no X-Lium-Api-Key, and a victim hotkey returned 201 Created, invoked live scoring, persisted the victim hotkey, and wrote that miner's topic lattice score. This permits score attribution or overwrites under another miner identity while consuming rental work without a caller-bound payment credential.
Require a verified miner identity or signature over the submission, require it to match miner_hotkey, and bind the Lium credential or account to that verified identity before invoking live evaluation.
Artifacts
Focused unauthenticated hotkey attribution test source
- The executed Rust test builds a live-Lium-configured in-router application, submits an attacker artifact under a victim hotkey without a Lium key, and asserts the resulting attribution and score write; the takeaway is that it directly exercises only the claimed missing binding.
Before and after hotkey validation harness source
- The executed shell harness injects the focused test into clean archives of parent `cebceda` and current `HEAD`, then runs it against both revisions; the takeaway is that it provides a same-scope before/after comparison.
Runtime output for unauthenticated victim-hotkey submissions before and after
- The captured command output records exit code 0 and passing endpoint tests on both revisions: POST returned 201 with no message, GET returned 200 with no message, one live score call occurred, and the victim lattice was written as 1000000; the takeaway is that the failure path is confirmed and unchanged across the comparison.
| async fn serve(bind: SocketAddr, state: AppState) -> Result<(), String> { | ||
| let app = proof_router(state); | ||
| let listener = TcpListener::bind(bind) | ||
| .await | ||
| .map_err(|e| format!("bind {bind}: {e}"))?; | ||
| tracing::info!( | ||
| %bind, | ||
| challenge_id = CHALLENGE_ID, | ||
| scoring_version = SCORING_VERSION, | ||
| "proof-challenge listening" | ||
| ); | ||
| axum::serve(listener, app) | ||
| .with_graceful_shutdown(async { | ||
| let _ = tokio::signal::ctrl_c().await; | ||
| }) | ||
| .await | ||
| .map_err(|e| e.to_string()) | ||
| } |
There was a problem hiding this comment.
The production entrypoint builds the HTTP application and waits on axum::serve, but never invokes proof_challenge::emit_epoch or delivers signed leaves to a gateway. The emission helper has only test call sites. As a result, scored Proof submissions remain in the local store and the Proof challenge cannot provide the signed leaves required for its trust-root share during epoch sealing.
Wire an epoch-aware production emission path that retains the signing key, generates the complete leaf set, submits it to the configured gateway with retry and idempotency, and covers the startup-to-gateway handoff with an integration test.
Artifacts
Focused Proof startup-to-emission harness source
- The authored executable harness starts the compiled Proof service, calls status and submission APIs, probes an emission route, and compares final status; the takeaway is that the executed scope directly covers startup through the absent emission handoff.
Proof service startup status before submission
- The real proof-challenge binary was started without configured inputs and queried before any submission, showing it only serves HTTP state with no emission activity; the takeaway is that startup itself creates no signed Proof leaves.
Proof service status after submission and emission request
- The real service received a submission attempt and a POST to the candidate emission route, returned HTTP 400 then HTTP 404, and retained identical status; the takeaway is that the startup-to-request path has no emission endpoint or state transition.
Proof emission helper call-site search
- The executed repository-wide Rust call-site search lists the Proof helper definition and only its unit-test invocations, with no production wiring from the binary, router, or gateway; the takeaway is that signed Proof emission is unreachable in production.
Targeted Proof package test output
- The targeted Proof library, HTTP, and binary test suites completed with 18 passing tests; the takeaway is that existing tests validate helpers and HTTP scoring but do not validate an end-to-end emission handoff.
| backend, | ||
| live_scorer, | ||
| admin_hashes: Arc::new(load_admin_hashes(cli.admin_tokens_file.as_deref())), | ||
| epoch: 0, |
There was a problem hiding this comment.
The service initializes AppState.epoch to 0 and never refreshes it from the authoritative epoch. Submission and scoring paths use this value for is_open_at, so a topic valid only at a later epoch is permanently treated as closed. A signed topic valid in epochs 7–9 returned 400 topic is not open at epoch 0 but returned an eligible 201 through the identical request path at epoch 7.
Source the current epoch from the authoritative chain and refresh it as epochs advance; retain a regression test that exercises a nonzero topic validity window through the service wiring.
Artifacts
Temporary nonzero epoch window regression test source
- The complete temporary test creates a signed topic valid only in epochs 7–9 and submits through the Axum router at epochs 0 and 7, with the takeaway.
Submission at immutable epoch zero
- Executed `PROOF_EPOCH_PROBE=zero cargo test -p proof-http nonzero_window_rejects_at_epoch_zero_and_accepts_at_current_epoch -- --nocapture` in `/home/user/repo` and captured HTTP 400 `topic is not open`, with the takeaway.
Submission at current valid epoch seven
- Executed `cargo test -p proof-http nonzero_window_rejects_at_epoch_zero_and_accepts_at_current_epoch -- --nocapture` in `/home/user/repo` and captured the matching HTTP 201 eligible submission at epoch 7, with the takeaway.
Exact epoch wiring and evaluation source locations
- Captured numbered source locations for the hard-coded operator epoch, HTTP submission gate, status/scoring selection, store filters, and topic predicate, with the takeaway.
Temporary regression test cleanup verification
- Executed `git diff --exit-code -- crates/proof-http/src/lib.rs` after validation and confirmed exit code 0, showing the temporary test left no source modification, with the takeaway.
2 participants
Summary
Adds the fifth live challenge
proof(2000 bps) with dynamic operator-published topics and a digest-pinned RLM judge. Topics are signed research problems (topic_id), not a frozen catalog in git. The eval image pin (config/proof-pin.toml) ships with an emptyeval_image_digestuntil CI publishes one — live submits fail closed with 503. Do not invent a sha256.Emission retune (sum 10000):
relearn3000,relearn-image1000,relearn-agent1000,bounty3000,proof2000. Trust root re-signed with the existing throwaway owner key ceremony (config/CEREMONY.md). Existing challenge public keys were kept; only shares + the newproofrow + a new throwaway owner pubkey.Paid score is the mean of per-topic lattices over currently
opentopics. Empty open set →NoScore(ChallengeInternal), not a paid 0.What landed
proof-task,proof-score,proof-store,proof-eval,proof-http,proof-harvest,proof-challenge+bins/proof-challengeon:8100config/proof-pin.toml(topic_pubkeymatches the trust-rootproofrow; empty eval digest):8100/ local:28100, Dockerfile target,images.yml,PROOF_FORCE_SIMbanned on droplet overlaysproof-holdout,proof-topic,external-docs-checkpins forproof.mddocs/PROOF.md,docs/external-miner/proof.md, AGENTS / NAMING / COMPLETENESSArenaSlug::Prooflisted with scoringreproducedCargo.lockwnaf0.14.0 → 0.14.1 (yanked today viaprimeorder/p256/dcap-qvl)No Modal. No secrets/mnemonics in git. There is no
bins/ctxin this repo.Greptile
Every PR is reviewed by Greptile before merge. Config:
.greptile/.@greptileai review(echobtalready requested)Test plan
cargo fmt --all -- --checkcargo clippy --workspace --all-targets -- -D warningscargo test --workspacecargo test -p trustroot --test trustroot_verify s9_repo_config_loads_when_present(5 rows, 3000/1000/1000/3000/2000, proof pk3ea26595…)cargo run -p xtask -- loc-capcargo run -p xtask -- consensus-lintcargo run -p xtask -- spec-checkcargo run -p xtask -- design-checkcargo run -p xtask -- external-docs-checkcargo deny check(local 0.20.2: advisories/bans/licenses/sources ok afterwnaf0.14.1)./deploy/scripts/assert-compose-matrix.sh(docker not installed here)Risk
Emission impact: live shares change (relearn 4000→3000, image/agent 1500→1000, proof 2000). Seal D23 will fail until every challenge still covers
E. Trust root: new throwaway owner pubkey; production rotation still follows CEREMONY.md with the offline owner key. Eval image: empty digest is intentional pre-launch 503. No Modal. No secrets/mnemonics in git.PROOF_FORCE_SIMis local/CI only and banned on droplet overlays.Naming
I did not rename
BASE_*environment variables, deployed host paths(
/opt/base,/run/base, …), GHCRbaseintelligence/basepackage names, orbase-*-v1cryptographic domain tags, unless this PR’s purpose is a coordinatedcutover documented in
docs/NAMING.md.