Frozen benchmark · live publication gate
A benchmark that shows what it cannot prove.
Thirty exact Marketplace artifacts are version-pinned and hash-pinned. Results appear only when the matching canonical Deep Scan can be reopened with complete analyzer coverage.
Correct interpretation
Reproducibility before headline numbers.
This page is an exact-artifact publication ledger, not a claim that one number describes all extension risk.
A previous internal run and a newer registry scan cannot replace the canonical report for the frozen version and hash.
Every visible result passes the same checks.
Extension identity, version, and SHA-256 agree.
Every required analyzer completed for this report.
Scan ID, scanner build, and ruleset remain attached.
Artifact evidence
All 30 frozen identities, including withheld results.
An awaiting row preserves corpus identity without inheriting an old outcome. Open any published row to inspect its immutable Deep Scan.
Exact artifactCohortPublicationEvidenceIndexes
amazonwebservices.aws-toolkit-vscode
credential sensitivefresh artifact holdoutallow100% coverage · scan 2c39f30aLOWRules 2026.07.24-policy-v3-calibration.6M0 · R32Diagnostic indexes@4.10.0 · 9197f84348510b68…aquasecurityofficial.trivy-vulnerability-scanner
security tool controlfresh artifact holdoutallow100% coverage · scan 774c4483InformationalRules 2026.07.24-policy-v3-calibration.6M0 · R0Diagnostic indexes@1.8.11 · 421289db1a46d354…Continue.continue
ai agenticfresh artifact holdoutreview100% coverage · scan f8594965LOWRules 2026.07.24-policy-v3-calibration.6M0 · R61Diagnostic indexes@2.1.0 · 28e1c4691f090080…dbaeumer.vscode-eslint
legitimate privilegedprior exposureallow100% coverage · scan a7593dc0InformationalRules 2026.07.24-policy-v3-calibration.6M0 · R0Diagnostic indexes@3.0.33 · a2ed5477988332d5…eamodio.gitlens
legitimate privilegedprior exposureblock100% coverage · scan a01dc0d8HIGHRules 2026.07.24-policy-v3-calibration.6M0 · R99Diagnostic indexes@2026.7.160544 · 20a80baa19fbeb69…esbenp.prettier-vscode
benign controlfresh artifact holdoutallow100% coverage · scan 7b1dc6ddInformationalRules 2026.07.24-policy-v3-calibration.6M0 · R0Diagnostic indexes@12.4.0 · 46d22a567b35ebe5…formulahendry.code-runner
legitimate privilegedfresh artifact holdoutblock100% coverage · scan 721d055eHIGHRules 2026.08.21-obfuscated-bundle.1M0 · R99Diagnostic indexes@0.12.2 · 4c8e4aea7dd07c9c…GitHub.copilot
ai agenticfresh artifact holdoutreview100% coverage · scan f8b69ba0LOWRules 2026.08.21-obfuscated-bundle.1M0 · R61Diagnostic indexes@1.388.0 · ed18caf3e3cd23ea…GitHub.copilot-chat
ai agenticprior exposurereview100% coverage · scan 6921041cHIGHRules 2026.08.21-obfuscated-bundle.1M0 · R80Diagnostic indexes@0.48.1 · 7852df60c171be0b…GitHub.vscode-pull-request-github
credential sensitivefresh artifact holdoutallow100% coverage · scan a3f5f30fLOWRules 2026.07.24-policy-v3-calibration.6M0 · R32Diagnostic indexes@0.159.2026071604 · 5e40bb78c3af7ad2…golang.go
legitimate privilegedfresh artifact holdoutallow100% coverage · scan f59bf505LOWRules 2026.08.21-obfuscated-bundle.1M0 · R32Diagnostic indexes@0.56.0 · 9f5959fb17ba0a8d…humao.rest-client
credential sensitivefresh artifact holdoutallow100% coverage · scan eed08aedLOWRules 2026.07.24-policy-v3-calibration.6M0 · R32Diagnostic indexes@0.25.1 · 0d2cd9f70181d085…mhutchie.git-graph
legitimate privilegedprior exposureallow100% coverage · scan 9ca8bb6cLOWRules 2026.08.21-obfuscated-bundle.1M0 · R32Diagnostic indexes@1.30.0 · b0779a30caf98664…ms-azuretools.vscode-docker
legitimate privilegedprior exposureallow100% coverage · scan df7a6aa7InformationalRules 2026.08.21-obfuscated-bundle.1M0 · R0Diagnostic indexes@2.0.0 · 631cac7a4a7d9c2f…ms-kubernetes-tools.vscode-kubernetes-tools
credential sensitivefresh artifact holdoutblock100% coverage · scan c226181bHIGHRules 2026.07.24-policy-v3-calibration.6M0 · R96Diagnostic indexes@1.4.0 · b7ae0e17eaf14e98…ms-python.python
legitimate privilegedprior exposureblock100% coverage · scan d89d1feaHIGHRules 2026.07.24-policy-v3-calibration.6M0 · R96Diagnostic indexes@2026.5.2026070801 · 7edf45e8e93fd155…ms-vscode-remote.remote-containers
legitimate privilegedprior exposureblock100% coverage · scan 044ce030HIGHRules 2026.08.21-obfuscated-bundle.1M0 · R96Diagnostic indexes@0.467.0 · b3bd40702da5dd7d…ms-vscode-remote.remote-ssh
credential sensitivefresh artifact holdoutallow100% coverage · scan cc9c1178InformationalRules 2026.07.24-policy-v3-calibration.6M0 · R0Diagnostic indexes@0.125.2026062315 · 862aaf35b7154013…ms-vscode.azure-account
credential sensitivefresh artifact holdoutreview100% coverage · scan 8b589d8aHIGHRules 2026.07.24-policy-v3-calibration.6M0 · R80Diagnostic indexes@0.13.0 · c7fb78223cf1b0e1…ms-vscode.cpptools
native packedfresh artifact holdoutreview100% coverage · scan 112d3817LOWRules 2026.07.24-policy-v3-calibration.6M0 · R63Diagnostic indexes@1.33.4 · e05cf0f982318849…PKief.material-icon-theme
benign controlprior exposureallow100% coverage · scan 63cbac79InformationalRules 2026.07.24-policy-v3-calibration.6M0 · R0Diagnostic indexes@5.36.1 · d72c538c8b328fca…redhat.java
legitimate privilegedfresh artifact holdoutallow100% coverage · scan 913c22c8InformationalRules 2026.07.24-policy-v3-calibration.6M0 · R0Diagnostic indexes@1.56.2026071508 · 1f6099292a4e811a…ritwickdey.LiveServer
legitimate privilegedprior exposurereview100% coverage · scan a1f2ff4dLOWRules 2026.08.21-obfuscated-bundle.1M0 · R61Diagnostic indexes@5.7.10 · 0f8ef819d4c2007f…RooVeterinaryInc.roo-cline
ai agenticfresh artifact holdoutreview100% coverage · scan 5981fa90HIGHRules 2026.07.24-policy-v3-calibration.6M0 · R80Diagnostic indexes@3.54.0 · caf96d596d69cb34…rust-lang.rust-analyzer
native packedfresh artifact holdoutreview100% coverage · scan ac6abdc6HIGHRules 2026.07.24-policy-v3-calibration.6M0 · R74Diagnostic indexes@0.4.2976 · 5ff02c55c7d5a732…saoudrizwan.claude-dev
ai agenticfresh artifact holdoutreview100% coverage · scan 8f8c7a66HIGHRules 2026.07.24-policy-v3-calibration.6M0 · R80Diagnostic indexes@4.0.8 · a92973db05d0c293…semgrep.semgrep
security tool controlfresh artifact holdoutreview100% coverage · scan 89b4073eLOWRules 2026.07.24-policy-v3-calibration.6M0 · R51Diagnostic indexes@1.17.0 · a06ce8efc8630256…snyk-security.snyk-vulnerability-scanner
security tool controlfresh artifact holdoutreview100% coverage · scan 3345ffc6HIGHRules 2026.07.24-policy-v3-calibration.6M0 · R75Diagnostic indexes@2.31.0 · 5e6ea92b390f7552…SonarSource.sonarlint-vscode
security tool controlfresh artifact holdoutallow100% coverage · scan 3fe45368InformationalRules 2026.07.24-policy-v3-calibration.6M0 · R0Diagnostic indexes@5.5.0 · d755728fcd9d87b1…usernamehw.errorlens
benign controlfresh artifact holdoutallow100% coverage · scan ba80958eInformationalRules 2026.08.21-obfuscated-bundle.1M0 · R0Diagnostic indexes@3.28.0 · edebbbcbd211d6ec…Non-negotiable
What this benchmark refuses to substitute.
A result stays withheld until the current scanner produces a complete canonical report for the frozen artifact.
- 01No latest-version substitutionRequired
- 02No artifact-hash mismatchRequired
- 03No incomplete analyzer coverageRequired
Evidence boundary
Read the severity guide Re-run first. Publish second.
Use the severity guide to understand decisions and indexes without treating them as calibrated probabilities.