diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index e7dc5add53..0967112898 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -56,24 +56,12 @@ "skills": "./", "description": "Trace and interpret the Pareto frontier across competing objectives using repeated single-objective cuOpt solves (weighted-sum and ε-constraint)." }, - { - "name": "cuopt-routing-formulation", - "source": "./skills/cuopt-routing-formulation", - "skills": "./", - "description": "Vehicle routing (VRP, TSP, PDP) — problem types and data requirements. Domain concepts; no API or interface." - }, { "name": "cuopt-routing-api-python", "source": "./skills/cuopt-routing-api-python", "skills": "./", "description": "Vehicle routing (VRP, TSP, PDP) with cuOpt — Python API only. Use when the user is building or solving routing in Python." }, - { - "name": "cuopt-server-common", - "source": "./skills/cuopt-server-common", - "skills": "./", - "description": "cuOpt REST server — what it does and how requests flow. Domain concepts; no deploy or client code." - }, { "name": "cuopt-server-api-python", "source": "./skills/cuopt-server-api-python", diff --git a/AGENTS.md b/AGENTS.md index bbc3d09dde..749910f507 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -14,8 +14,6 @@ AI agent skills for NVIDIA cuOpt optimization engine. Skills live in **`skills/` ### Common (concepts only; no API code) - `skills/cuopt-numerical-optimization-formulation/` — LP / MILP / QP: concepts + problem parsing + common formulation patterns - `skills/cuopt-multi-objective-exploration/` — Multi-objective: trace + interpret the Pareto frontier across competing objectives (ε-constraint / weighted-sum over repeated cuOpt solves) -- `skills/cuopt-routing-formulation/` — Routing: VRP, TSP, PDP (problem types, data) -- `skills/cuopt-server-common/` — Server: capabilities, workflow ### Installation - `skills/cuopt-install/` — User install for Python, C, and server (pip, conda, Docker, verification). For building cuOpt from source, see `skills/cuopt-developer/`. diff --git a/skills/cuopt-multi-objective-exploration/BENCHMARK.md b/skills/cuopt-multi-objective-exploration/BENCHMARK.md index 9982ced7da..7d20f8a440 100644 --- a/skills/cuopt-multi-objective-exploration/BENCHMARK.md +++ b/skills/cuopt-multi-objective-exploration/BENCHMARK.md @@ -7,7 +7,7 @@ This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the s ## Evaluation Summary - Skill: `cuopt-multi-objective-exploration` -- Evaluation date: 2026-06-26 +- Evaluation date: 2026-06-29 - NVSkills-Eval profile: `external` - Environment: `astra-sandbox` - Dataset: 5 evaluation tasks @@ -55,10 +55,10 @@ Task composition is derived from the evaluation dataset when possible. Entries w | Dimension | Num | `claude-code` | `codex` | |---|---:|---:|---:| | Security | 5 | 100% (+0%) | 100% (+0%) | -| Correctness | 5 | 90% (+37%) | 76% (+24%) | -| Discoverability | 5 | 80% (+60%) | 71% (+41%) | -| Effectiveness | 5 | 91% (+20%) | 68% (+10%) | -| Efficiency | 5 | 80% (+40%) | 71% (+30%) | +| Correctness | 5 | 90% (+0%) | 87% (-8%) | +| Discoverability | 5 | 78% (-2%) | 73% (-22%) | +| Effectiveness | 5 | 85% (-6%) | 85% (+2%) | +| Efficiency | 5 | 75% (-3%) | 71% (-21%) | Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. diff --git a/skills/cuopt-multi-objective-exploration/SKILL.md b/skills/cuopt-multi-objective-exploration/SKILL.md index e8758bda6e..e78392390e 100644 --- a/skills/cuopt-multi-objective-exploration/SKILL.md +++ b/skills/cuopt-multi-objective-exploration/SKILL.md @@ -135,4 +135,4 @@ Practical notes: This skill is solver- and interface-agnostic. The per-solve mechanics — building the objective, adding the ε constraints, passing a warm start, reading status — live in the API skills: - `cuopt-numerical-optimization-api-python` / `-api-c` / `-api-cli` — LP, MILP, QP solves. -- `cuopt-routing-formulation` + `cuopt-routing-api-python` — the same frontier workflow applies to routing tradeoffs (distance vs. vehicles vs. time). +- `cuopt-routing-api-python` — the same frontier workflow applies to routing tradeoffs (distance vs. vehicles vs. time). diff --git a/skills/cuopt-multi-objective-exploration/skill-card.md b/skills/cuopt-multi-objective-exploration/skill-card.md index f3d22a18c8..6b37e0a1d4 100644 --- a/skills/cuopt-multi-objective-exploration/skill-card.md +++ b/skills/cuopt-multi-objective-exploration/skill-card.md @@ -7,16 +7,16 @@ This skill is ready for commercial/non-commercial use.
NVIDIA
### License/Terms of Use:
-Apache 2.0
+Apache-2.0
## Use Case:
-Developers and engineers exploring tradeoffs between competing objectives in optimization problems, tracing Pareto frontiers to make informed multi-objective decisions rather than accepting a single implicit weighting.
+Developers and engineers exploring tradeoffs between competing objectives in optimization problems, using cuOpt to trace the Pareto frontier and interpret exchange rates rather than collapsing to a single weighted answer.
### Deployment Geography for Use:
Global
## Requirements / Dependencies:
-**Requires API Key or External Credential:** [No]
-**Credential Type(s):** [None]
+**Requires API Key or External Credential:** [Not Specified]
+**Credential Type(s):** [None identified]
Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate.
@@ -26,23 +26,23 @@ Mitigation: Review and scan skill before deployment.
## Reference(s):
- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html)
-- [cuOpt Examples](https://github.com/NVIDIA/cuopt-examples)
+- [cuopt-examples](https://github.com/NVIDIA/cuopt-examples)
## Skill Output:
**Output Type(s):** [Analysis, Configuration instructions]
-**Output Format:** [Markdown with inline code blocks]
+**Output Format:** [Markdown with inline formulations and solver-call patterns]
**Output Parameters:** [1D]
**Other Properties Related to Output:** [None]
## Evaluation Agents Used:
-- Claude Code (`claude-code`)
-- Codex (`codex`)
+- claude-code
+- codex
## Evaluation Tasks:
-Evaluated against 5 evaluation tasks (4 positive activation, 1 negative activation) using NVSkills-Eval external profile in astra-sandbox environment.
+Evaluated against 5 internal evaluation tasks (4 positive skill-activation, 1 negative).
## Evaluation Metrics Used:
Reported benchmark dimensions:
@@ -67,10 +67,10 @@ Underlying evaluation signals used in this run:
| Dimension | Num | `claude-code` | `codex` | |---|---:|---:|---:| | Security | 5 | 100% (+0%) | 100% (+0%) | -| Correctness | 5 | 90% (+37%) | 76% (+24%) | -| Discoverability | 5 | 80% (+60%) | 71% (+41%) | -| Effectiveness | 5 | 91% (+20%) | 68% (+10%) | -| Efficiency | 5 | 80% (+40%) | 71% (+30%) | +| Correctness | 5 | 90% (+0%) | 87% (-8%) | +| Discoverability | 5 | 78% (-2%) | 73% (-22%) | +| Effectiveness | 5 | 85% (-6%) | 85% (+2%) | +| Efficiency | 5 | 75% (-3%) | 71% (-21%) | ## Skill Version(s):
26.08.00 (source: frontmatter)
diff --git a/skills/cuopt-multi-objective-exploration/skill.oms.sig b/skills/cuopt-multi-objective-exploration/skill.oms.sig index 7f16ab5873..e6ba2ea66c 100644 --- a/skills/cuopt-multi-objective-exploration/skill.oms.sig +++ b/skills/cuopt-multi-objective-exploration/skill.oms.sig @@ -1 +1 @@ -{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtbXVsdGktb2JqZWN0aXZlLWV4cGxvcmF0aW9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjgwNmFlYWFjOWM3YmMxZTk1ZGJlNDdiOWVhY2FjMDk5NjBiZDczMTA4NTY0ZWE3NDk1OTdlYWI3ZDA2N2VlN2MiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJyZXNvdXJjZXMiOiBbCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJCRU5DSE1BUksubWQiLAogICAgICAgICJkaWdlc3QiOiAiOTcyM2E4ZjZjOWQ1ZTlkZDAxYTU0N2VlOGYyYzE2MzA5ZWU0YzIzMGY0NThiYjBiYWU1ODZjMTllMTJkNWE1MyIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJTS0lMTC5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI1MjQ0ODgwOTFlMTU0NWZmMGQxMDA0M2VmZjhiMDY2OTNkOTJiNTM5MzdkZjYxMjQyZWQzM2Q3NzZhZTM4NGZiIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogImV2YWxzL2V2YWxzLmpzb24iLAogICAgICAgICJkaWdlc3QiOiAiZDk1NjE1ZWUzMjc1MTc3MWYzMDc3ZjE3MzY4M2YwNGZkMTdhZDU3Y2U4MmFlOWIxMjRjYmQ4M2ZhYWJkNjBiYiIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJza2lsbC1jYXJkLm1kIiwKICAgICAgICAiZGlnZXN0IjogImM2OTJlM2Q5ZjAzMDQ1YWM2YTA5NzkwYWM4NjM4NGM4ODYxOTNhZmY2ODQxZDdjMDdhZThjMWZmYjAyYzdhMjkiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9CiAgICBdLAogICAgInNlcmlhbGl6YXRpb24iOiB7CiAgICAgICJhbGxvd19zeW1saW5rcyI6IGZhbHNlLAogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdCIsCiAgICAgICAgIi5naXRpZ25vcmUiLAogICAgICAgICIuZ2l0YXR0cmlidXRlcyIsCiAgICAgICAgIi5naXRodWIiCiAgICAgIF0sCiAgICAgICJoYXNoX3R5cGUiOiAic2hhMjU2IgogICAgfQogIH0KfQ==","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMEbPeltBPGv9lD/CCSbTi12cHXvWbA2qoDfCcVLU420GcQ4kGa4r3aSmODk8E/ylNwIxALYaAi9ppzhEiqwoa7zbN1R0vq9QbUvNCcqFPcx7lLlHblGFS7nHB632atREfijbig==","keyid":""}]}} \ No newline at end of file +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtbXVsdGktb2JqZWN0aXZlLWV4cGxvcmF0aW9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjcxMmY0NDMwMjNkYTc5ODNmNzliOGIwNzlmN2YzODljM2IzNDJmNGY1NGZiNWViZGRhZmM1ODZiMjZlM2ZkYzIiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZSwKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiLAogICAgICAgICIuZ2l0IiwKICAgICAgICAiLmdpdGlnbm9yZSIsCiAgICAgICAgIi5naXRodWIiCiAgICAgIF0KICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICIzNzhjNmMzMDY0YzVkNDE0ZjQ5M2M2NTZjODQ2YWZlYWMyMmMyMWU4NjhmY2MzMzRlMThiNGNjMmFmYjIwMjM4IiwKICAgICAgICAibmFtZSI6ICJCRU5DSE1BUksubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICI5NWFhOTRiNGViNzQwOTE5YTI4MTFmNWY4OWRkZmVhZmVjZTkwNjcxNjJjMzE4NzRlYTBkYTM0ZGM4NWJhZDEyIiwKICAgICAgICAibmFtZSI6ICJTS0lMTC5tZCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImQ5NTYxNWVlMzI3NTE3NzFmMzA3N2YxNzM2ODNmMDRmZDE3YWQ1N2NlODJhZTliMTI0Y2JkODNmYWFiZDYwYmIiLAogICAgICAgICJuYW1lIjogImV2YWxzL2V2YWxzLmpzb24iLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICJiM2Y4NDZhMzExZDc5Zjg4ZTNhYzlmNGJmZmM3YjM4MDI4OGM5Nzk1ZTA3YzNhN2ViZWYxYjY2YWE0MGU3YTEzIiwKICAgICAgICAibmFtZSI6ICJza2lsbC1jYXJkLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfQogICAgXQogIH0KfQ==","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGQCMC4Gw/I2q9Fy5WQ5dDlqOtYhZcp3y08aFKflnEzfYDWsZHEjlAIbutAhexTr80iANQIwIrWIJX73+kbJi/HXWsYHWBKF+3oqu0Yrkqgg2OFrlmdaDGzwZTKSFKqenudCk401","keyid":""}]}} \ No newline at end of file diff --git a/skills/cuopt-routing-api-python/BENCHMARK.md b/skills/cuopt-routing-api-python/BENCHMARK.md index 72f6892ec7..5384e7ef34 100644 --- a/skills/cuopt-routing-api-python/BENCHMARK.md +++ b/skills/cuopt-routing-api-python/BENCHMARK.md @@ -7,13 +7,13 @@ This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the s ## Evaluation Summary - Skill: `cuopt-routing-api-python` -- Evaluation date: 2026-05-29 +- Evaluation date: 2026-06-29 - NVSkills-Eval profile: `external` -- Environment: `local` +- Environment: `astra-sandbox` - Dataset: 1 evaluation tasks -- Attempts per task: 2 +- Attempts per task: 1 - Pass threshold: 50% -- Overall verdict: FAIL +- Overall verdict: PASS ## Agents Used @@ -54,46 +54,27 @@ Task composition is derived from the evaluation dataset when possible. Entries w | Dimension | Num | `claude-code` | `codex` | |---|---:|---:|---:| -| Security | 2 | 100% (+0%) | 100% (+0%) | -| Correctness | 2 | 100% (+0%) | 95% (+3%) | -| Discoverability | 2 | 100% (+0%) | 70% (-5%) | -| Effectiveness | 2 | 83% (+14%) | 83% (+12%) | -| Efficiency | 2 | 93% (-0%) | 56% (-5%) | +| Security | 1 | 100% (+0%) | 100% (+0%) | +| Correctness | 1 | 100% (+70%) | 97% (+42%) | +| Discoverability | 1 | 100% (+100%) | 82% (+57%) | +| Effectiveness | 1 | 82% (+45%) | 74% (+29%) | +| Efficiency | 1 | 95% (+67%) | 72% (+45%) | Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. ## Tier 1: Static Validation Summary -Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 8 total findings. +Tier 1 validation passed with observations. NVSkills-Eval ran 1 checks and found 2 total findings. Top findings: - MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-routing-api-python/SKILL.md`) -- MEDIUM SECURITY/Unknown (SQP-2): Binding the cuOpt server to 0.0.0.0 exposes it on all network interfaces, making it accessible to any host that can reac (`references/server_examples.md:7`) -- LOW QUALITY/quality_discoverability: No '## Purpose' section (`skills/cuopt-routing-api-python/SKILL.md`) -- LOW QUALITY/quality_reliability: No prerequisites/requirements documented (`skills/cuopt-routing-api-python/SKILL.md`) -- LOW QUALITY/quality_reliability: No limitations documented (`skills/cuopt-routing-api-python/SKILL.md`) +- LOW SCHEMA/author_format: Author must be of the form 'Name ' (`skills/cuopt-routing-api-python/SKILL.md`) ## Tier 2: Deduplication Summary -Tier 2 validation reported findings. NVSkills-Eval ran 2 checks and found 4 total findings. - -Top findings: - -- HIGH DUPLICATE/duplicate: Duplicate content found within references/server_examples.md: - "# Poll for solution" in references/server_examples.md (lines 45-51) - vs "# Poll for solution" in references/server_examples.md (lines 156-162) (`references/server_examples.md:45`) -- HIGH DUPLICATE/duplicate: Duplicate content found across SKILL.md and references/examples.md: - "# Capacities" in SKILL.md (lines 30-35) - vs "# Add capacity dimension (name, demand_per_order, capacity_per_vehicle)" in references/examples.md (lines 73-75) - vs "# Add capacity dimension" in references/examples.md (lines 156-158) (`SKILL.md:30`) -- HIGH DUPLICATE/duplicate: Duplicate content found across references/examples.md and references/server_examples.md: - "## Additional References (tested in CI)" in references/examples.md (lines 237-249) - vs "## Additional References (tested in CI)" in references/server_examples.md (lines 193-204) (`references/examples.md:237`) -- HIGH DUPLICATE/duplicate: Duplicate content found across assets/pdp_basic/README.md and assets/pdp_basic/model.py: - "# Pickup-Delivery (PDP)" in assets/pdp_basic/README.md (lines 1-7) - vs "(module docstring)" in assets/pdp_basic/model.py (lines 1-2) (`assets/pdp_basic/README.md:1`) +This tier was not run or did not produce findings in this report. ## Publication Recommendation -The skill should be reviewed before NVSkills-Eval publication. Skill owners should address the findings above and rerun NVSkills-Eval to refresh this benchmark. +The skill is suitable to proceed toward NVSkills-Eval publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change. diff --git a/skills/cuopt-routing-api-python/SKILL.md b/skills/cuopt-routing-api-python/SKILL.md index 421d68bbe7..741badfabf 100644 --- a/skills/cuopt-routing-api-python/SKILL.md +++ b/skills/cuopt-routing-api-python/SKILL.md @@ -17,10 +17,18 @@ metadata: # cuOpt Routing — Python API -Confirm problem type (TSP, VRP, PDP) and data (locations, orders, fleet, constraints) before coding. - This skill is **Python only**. Routing has no C API in cuOpt. +## Required questions + +Ask these if not already clear: + +1. **Problem type** — TSP, VRP, or PDP? +2. **Locations** — How many? Depot(s)? Cost or distance between pairs (matrix or derived)? +3. **Orders / tasks** — Which locations must be visited? Demand or service per stop? +4. **Fleet** — Number of vehicles, capacity per vehicle (and per dimension if multiple), start/end locations? +5. **Constraints** — Time windows (earliest/latest arrival), service times, precedence (order A before B)? + ## Minimal VRP Example ```python diff --git a/skills/cuopt-routing-api-python/skill-card.md b/skills/cuopt-routing-api-python/skill-card.md index 2e5e3a98fd..f5bdc88f34 100644 --- a/skills/cuopt-routing-api-python/skill-card.md +++ b/skills/cuopt-routing-api-python/skill-card.md @@ -9,25 +9,31 @@ NVIDIA
### License/Terms of Use:
Apache-2.0
## Use Case:
-Developers and engineers building or solving vehicle routing problems (VRP, TSP, PDP) using the NVIDIA cuOpt Python API.
+Developers and engineers building or solving vehicle routing optimization problems (VRP, TSP, PDP) using the NVIDIA cuOpt Python API.
### Deployment Geography for Use:
Global
+## Requirements / Dependencies:
+**Requires API Key or External Credential:** [Not Specified]
+**Credential Type(s):** [None identified]
+ +Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate.
+ ## Known Risks and Mitigations:
Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills.
Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [Python API Examples (VRP, PDP, multi-depot)](references/examples.md)
-- [REST Server Examples](references/server_examples.md)
+- [examples.md](references/examples.md)
+- [server_examples.md](references/server_examples.md)
- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html)
-- [cuOpt Examples Repository](https://github.com/NVIDIA/cuopt-examples)
+- [cuopt-examples](https://github.com/NVIDIA/cuopt-examples)
## Skill Output:
-**Output Type(s):** [Code, API Calls]
-**Output Format:** [Python code with cudf/cuOpt API calls]
+**Output Type(s):** [Code, API Calls, Configuration instructions]
+**Output Format:** [Markdown with inline Python code blocks]
**Output Parameters:** [1D]
**Other Properties Related to Output:** [None]
@@ -38,7 +44,7 @@ Mitigation: Review and scan skill before deployment.
## Evaluation Tasks:
-1 evaluation task (positive skill-activation), 2 attempts per task, pass threshold 50%.
+Evaluated against 1 evaluation task in the `external` NVSkills-Eval profile (environment: astra-sandbox).
## Evaluation Metrics Used:
Reported benchmark dimensions:
@@ -62,11 +68,11 @@ Underlying evaluation signals used in this run:
## Evaluation Results:
| Dimension | Num | `claude-code` | `codex` | |---|---:|---:|---:| -| Security | 2 | 100% (+0%) | 100% (+0%) | -| Correctness | 2 | 100% (+0%) | 95% (+3%) | -| Discoverability | 2 | 100% (+0%) | 70% (-5%) | -| Effectiveness | 2 | 83% (+14%) | 83% (+12%) | -| Efficiency | 2 | 93% (-0%) | 56% (-5%) | +| Security | 1 | 100% (+0%) | 100% (+0%) | +| Correctness | 1 | 100% (+70%) | 97% (+42%) | +| Discoverability | 1 | 100% (+100%) | 82% (+57%) | +| Effectiveness | 1 | 82% (+45%) | 74% (+29%) | +| Efficiency | 1 | 95% (+67%) | 72% (+45%) | ## Skill Version(s):
26.08.00 (source: frontmatter)
diff --git a/skills/cuopt-routing-api-python/skill.oms.sig b/skills/cuopt-routing-api-python/skill.oms.sig index 70d7ec278d..27dfd3f0e3 100644 --- a/skills/cuopt-routing-api-python/skill.oms.sig +++ b/skills/cuopt-routing-api-python/skill.oms.sig @@ -1 +1 @@ -{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtcm91dGluZy1hcGktcHl0aG9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjFjNTJlZDRiNGI0NWMyOWQ5YmNlZDE2Yjc5MGQ3YmU3MjQ5MjM1NjQ2NzMwYTE4MjViNzIwZThmNTZkNDNjNzUiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImFsbG93X3N5bWxpbmtzIjogZmFsc2UsCiAgICAgICJpZ25vcmVfcGF0aHMiOiBbCiAgICAgICAgIi5naXRodWIiLAogICAgICAgICIuZ2l0aWdub3JlIiwKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiLAogICAgICAgICIuZ2l0IgogICAgICBdLAogICAgICAiaGFzaF90eXBlIjogInNoYTI1NiIKICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiMzRjZDY0YjIyYmQyMjEyNDc0MDZmZmFkMDhhZjFiYmNkYzE2NWRlODVmYTZkODM3NTVjMWY3OWViMjBjYjBkOSIsCiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiNDhjNDc2MzIwYThmN2YxM2VhNmQzOTA5YzRkODkwZjk5ZDk5MmYyMzJiNDZjMTAwNmM3MDE0YTdiZTI5MzIzMyIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICJjNzAwMmEzMTIxOTgzZjMyOTRlZmJlOGM5NTQxOTQzYmYyNGM4OWEwN2JlYTZhMzIwMDdjNzc0YTJjODA4MDIxIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvUkVBRE1FLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZTBkMWExZmQ3ZDBhZDRlNDU0ZDA4ZjU1ZGU5MWJiZWRlNzhmZjEyMjJkNmE1NDJkNTVhYWFjZjcxYzVhN2U2MiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3BkcF9iYXNpYy9SRUFETUUubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICIxNDRkYTFkZjVkZTI4ZDc4NWE5YjQ2N2IzZDE0NDE3ZTcxNmY1MzJhYzliOTg5MDQ2ZWFmN2U0ZjUyOTlhNWZkIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvcGRwX2Jhc2ljL21vZGVsLnB5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiMGE4NWFlZjFjMWJlNTk5ODlkZTQwYWE2Y2U5ZmU1NGU3MjBlNzA5NWYwODZiYzZmNjg0ZjJiM2M5ZGEzMzg5NCIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3ZycF9iYXNpYy9SRUFETUUubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI1MjQ1Yjc3NDY1YTI2YjY4YWVmYmFhMzI0OWI1MWVmMGRhNDUwNWY0ZTE5NzRjZjZkMGY0NGIxYzc4ZmM4MDcwIiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvdnJwX2Jhc2ljL21vZGVsLnB5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiZGJhMjAxMTE5ZTRmMGM1YjdkN2IxN2RmOWM3MTFkNDQ4M2UwMTg5MzM1MDhlMzYxZWU5MjllNmM0NGU1NmE0YiIsCiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjVlYjM1NzM1NTU5ZDkzNWMyMWUyMGE0OTU5MWQ1NGM5YjZmYTkyNDcyMTdhYTQ3NzZhNWY4OTgzY2JkMjdmODEiLAogICAgICAgICJuYW1lIjogInJlZmVyZW5jZXMvZXhhbXBsZXMubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI1MDhiZjRhZThjYjViYzdlMjQ5YjM3NzI2MGYxNDIxYjcwZDlkMzQ1YmI1YTZkMTZjNmZhMGI1NmUyNTY4MjViIiwKICAgICAgICAibmFtZSI6ICJyZWZlcmVuY2VzL3NlcnZlcl9leGFtcGxlcy5tZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAiZGlnZXN0IjogIjcxOTRjN2FkMmIwMjY2MmU5MTFhNzFmOGIwN2Q2Nzk1NmNiNzdmMDMxZTU3NWEwZTcxZjVjMTZlYjk5ZTMzOWEiLAogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMQC/if57e3mXxcr46MyoN7/Qlrwmk9leJtI83klm2/SuPZXkOfRclZp539nJbCqxcq4CMChXvVkzTj75l5w+zoaUK63MRHUujhIesZqb435AE2hAkKTOIrL4596BL+DxmL+QUA==","keyid":""}]}} \ No newline at end of file +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtcm91dGluZy1hcGktcHl0aG9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjIxNTEyZWMwZjZlZDM0OWE4YjNkMGNiZmY3ZDljOGM5MzA0MThhMzAzMjdjZmM1NDk2ZTE0OTE0NGQ2ZmRiOTEiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJyZXNvdXJjZXMiOiBbCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImE1MzkwNDM2YmNlM2ZkZjMyMTEyZjUxZmIxZjU0Mzk2MjdjNDM2NzU1Zjg3MjRiYzlhYTVhNzhkMjlhYmZmODciLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJCRU5DSE1BUksubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogIjNkODg1YzAzNjAwODQ2NDI2NmFiMzc3NWZkNTczZDBhYjA0ZjdkNTU3NDkwNDI5ODlhYjFkZWQ4MTMzMTU2MGMiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJTS0lMTC5tZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiYzcwMDJhMzEyMTk4M2YzMjk0ZWZiZThjOTU0MTk0M2JmMjRjODlhMDdiZWE2YTMyMDA3Yzc3NGEyYzgwODAyMSIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9SRUFETUUubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImUwZDFhMWZkN2QwYWQ0ZTQ1NGQwOGY1NWRlOTFiYmVkZTc4ZmYxMjIyZDZhNTQyZDU1YWFhY2Y3MWM1YTdlNjIiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvcGRwX2Jhc2ljL1JFQURNRS5tZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiMTQ0ZGExZGY1ZGUyOGQ3ODVhOWI0NjdiM2QxNDQxN2U3MTZmNTMyYWM5Yjk4OTA0NmVhZjdlNGY1Mjk5YTVmZCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9wZHBfYmFzaWMvbW9kZWwucHkiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogIjBhODVhZWYxYzFiZTU5OTg5ZGU0MGFhNmNlOWZlNTRlNzIwZTcwOTVmMDg2YmM2ZjY4NGYyYjNjOWRhMzM4OTQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvdnJwX2Jhc2ljL1JFQURNRS5tZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiNTI0NWI3NzQ2NWEyNmI2OGFlZmJhYTMyNDliNTFlZjBkYTQ1MDVmNGUxOTc0Y2Y2ZDBmNDRiMWM3OGZjODA3MCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy92cnBfYmFzaWMvbW9kZWwucHkiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImRiYTIwMTExOWU0ZjBjNWI3ZDdiMTdkZjljNzExZDQ0ODNlMDE4OTMzNTA4ZTM2MWVlOTI5ZTZjNDRlNTZhNGIiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJldmFscy9ldmFscy5qc29uIgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICI1ZWIzNTczNTU1OWQ5MzVjMjFlMjBhNDk1OTFkNTRjOWI2ZmE5MjQ3MjE3YWE0Nzc2YTVmODk4M2NiZDI3ZjgxIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAicmVmZXJlbmNlcy9leGFtcGxlcy5tZCIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiNTA4YmY0YWU4Y2I1YmM3ZTI0OWIzNzcyNjBmMTQyMWI3MGQ5ZDM0NWJiNWE2ZDE2YzZmYTBiNTZlMjU2ODI1YiIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogInJlZmVyZW5jZXMvc2VydmVyX2V4YW1wbGVzLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICJmYjMxYjBmZWQ4ZDNjOGQ5ZDRiMWU0OTkxMTFkZmJlNzNhMDM3ZmU1YmEzOWY5YjQ0Njc1MTI5NzBhZTZmY2JlIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAic2tpbGwtY2FyZC5tZCIKICAgICAgfQogICAgXSwKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZSwKICAgICAgIm1ldGhvZCI6ICJmaWxlcyIsCiAgICAgICJpZ25vcmVfcGF0aHMiOiBbCiAgICAgICAgIi5naXRodWIiLAogICAgICAgICIuZ2l0YXR0cmlidXRlcyIsCiAgICAgICAgIi5naXRpZ25vcmUiLAogICAgICAgICIuZ2l0IgogICAgICBdLAogICAgICAiaGFzaF90eXBlIjogInNoYTI1NiIKICAgIH0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGQCMEbltLo1vI/GV6W/VzAVAb+tJRcudQZzHMA1PfXTUd+sfc3njINmcL7FN5Z1DcFOpQIwTeEpHNv09maxv2FO4NF9Y4Yu2rCnOW15b7KPjIL3cH8FYgFeLSG2VwfZgFuKa8OE","keyid":""}]}} \ No newline at end of file diff --git a/skills/cuopt-routing-formulation/BENCHMARK.md b/skills/cuopt-routing-formulation/BENCHMARK.md deleted file mode 100644 index f6807194b0..0000000000 --- a/skills/cuopt-routing-formulation/BENCHMARK.md +++ /dev/null @@ -1,87 +0,0 @@ -# Evaluation Report - -Evaluation of the `cuopt-routing-formulation` skill before publication through NVSkills-Eval. - -This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use. - -## Evaluation Summary - -- Skill: `cuopt-routing-formulation` -- Evaluation date: 2026-05-28 -- NVSkills-Eval profile: `external` -- Environment: `local` -- Dataset: 1 evaluation tasks -- Attempts per task: 2 -- Pass threshold: 50% -- Overall verdict: PASS - -## Agents Used - -- `claude-code` -- `codex` - -## Metrics Used - -Reported benchmark dimensions: - -- Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. -- Correctness: checks whether the agent follows the expected workflow and produces the correct final output. -- Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant. -- Effectiveness: checks whether the agent performs measurably better with the skill than without it. -- Efficiency: checks whether the agent uses fewer tokens and avoids redundant work. - -Underlying evaluation signals used in this run: - -- `skill_execution` (Skill Execution): verifies that the agent loaded the expected skill and workflow. -- `skill_efficiency` (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage. -- `accuracy` (Accuracy): grades final-answer correctness against the reference answer. -- `goal_accuracy` (Goal Accuracy): checks whether the overall user task completed successfully. -- `behavior_check` (Behavior Check): verifies expected behavior steps, including safety expectations. -- `token_efficiency` (Token Efficiency): compares token usage with and without the skill. - -## Test Tasks - -The benchmark dataset contained 1 evaluation tasks: - -- Positive tasks: 1 tasks where the skill was expected to activate. -- Negative tasks: 0 tasks where no skill was expected. -- Unlabeled tasks: 0 tasks where positive/negative intent could not be inferred. - -Task composition is derived from the evaluation dataset when possible. Entries with `expected_skill` set are treated as positive skill-activation cases, while entries with `expected_skill: null` are treated as negative activation cases. - -## Results - -| Dimension | Num | `claude-code` | `codex` | -|---|---:|---:|---:| -| Security | 2 | 100% (+0%) | 100% (+0%) | -| Correctness | 2 | 100% (+0%) | 97% (+23%) | -| Discoverability | 2 | 100% (+0%) | 84% (+48%) | -| Effectiveness | 2 | 97% (-2%) | 98% (+0%) | -| Efficiency | 2 | 93% (-0%) | 78% (+34%) | - -Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. - -## Tier 1: Static Validation Summary - -Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 13 total findings. - -Top findings: - -- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-routing-formulation/SKILL.md`) -- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/cuopt-routing-formulation/SKILL.md`) -- LOW QUALITY/quality_correctness: No examples provided (`skills/cuopt-routing-formulation/SKILL.md`) -- LOW QUALITY/quality_discoverability: Description doesn't mention WHEN to use this skill (`skills/cuopt-routing-formulation/SKILL.md`) -- LOW QUALITY/quality_discoverability: Broad description without negative triggers may cause over-triggering (`skills/cuopt-routing-formulation/SKILL.md`) - -## Tier 2: Deduplication Summary - -Tier 2 validation passed. NVSkills-Eval ran 2 checks and found 0 total findings. - -Notable observations: - -- Context Deduplication: Collected 1 file(s) -- Inter-Skill Deduplication: Parsed skill 'cuopt-routing-formulation': 108 char description - -## Publication Recommendation - -The skill is suitable to proceed toward NVSkills-Eval publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change. diff --git a/skills/cuopt-routing-formulation/SKILL.md b/skills/cuopt-routing-formulation/SKILL.md deleted file mode 100644 index dad7ca5282..0000000000 --- a/skills/cuopt-routing-formulation/SKILL.md +++ /dev/null @@ -1,41 +0,0 @@ ---- -name: cuopt-routing-formulation -version: "26.08.00" -description: Vehicle routing (VRP, TSP, PDP) — problem types and data requirements. Domain concepts; no API or interface. -license: Apache-2.0 -metadata: - author: NVIDIA cuOpt Team - tags: - - routing - - vrp - - tsp - - formulation - - concepts ---- - - -# Routing Formulation - -Domain concepts for vehicle routing. No API or interface details here. - -## What is routing - -- **TSP**: Single vehicle, visit all locations once (e.g. shortest tour). -- **VRP**: Multiple vehicles, capacity and/or time limits; assign orders to vehicles and sequence stops. -- **PDP**: Pickup and delivery pairs; pickup must be visited before the corresponding delivery. - -## Required questions (problem and data) - -Ask these if not already clear: - -1. **Problem type** — TSP, VRP, or PDP? -2. **Locations** — How many? Depot(s)? Cost or distance between pairs (matrix or derived)? -3. **Orders / tasks** — Which locations must be visited? Demand or service per stop? -4. **Fleet** — Number of vehicles, capacity per vehicle (and per dimension if multiple), start/end locations? -5. **Constraints** — Time windows (earliest/latest arrival), service times, precedence (order A before B)? - -## Typical data - -- Cost or distance matrix (or travel-time matrix). -- Order locations and, for VRP, demand per order. -- Vehicle capacities and optional time windows for vehicles and orders. diff --git a/skills/cuopt-routing-formulation/evals/evals.json b/skills/cuopt-routing-formulation/evals/evals.json deleted file mode 100644 index 44b823eba8..0000000000 --- a/skills/cuopt-routing-formulation/evals/evals.json +++ /dev/null @@ -1,14 +0,0 @@ -[ - { - "id": "rt-form-eval-001-tsp-vs-vrp-vs-pdp", - "question": "A courier company has 8 trucks and 50 packages to deliver across a city. Some packages must be picked up from one address and dropped off at another. What problem type is this, and what data do I need to collect?", - "expected_skill": "cuopt-routing-formulation", - "expected_script": null, - "ground_truth": "The agent identifies the problem as multi-vehicle PDP (Pickup and Delivery Problem) — not VRP (one-way deliveries from a depot) or TSP (single vehicle). It then walks the user through the data categories needed: locations and a cost/distance matrix, pickup-delivery pairs as the order data, fleet (8 trucks with capacity and depot configuration), and time windows. Does not produce code.", - "expected_behavior": [ - "Identifies the problem type as multi-vehicle PDP, not VRP and not TSP, and explains the pickup-then-deliver pairing as the distinguishing feature", - "Lists the data the user needs to collect across locations / orders (pickup-delivery pairs) / fleet (8 trucks with capacity) / time windows", - "Does not produce code — this skill is concepts only" - ] - } -] diff --git a/skills/cuopt-routing-formulation/skill-card.md b/skills/cuopt-routing-formulation/skill-card.md deleted file mode 100644 index f95730519c..0000000000 --- a/skills/cuopt-routing-formulation/skill-card.md +++ /dev/null @@ -1,75 +0,0 @@ -## Description:
-Vehicle routing (VRP, TSP, PDP) — problem types and data requirements. Domain concepts; no API or interface.
- -This skill is ready for commercial/non-commercial use.
- -## Owner -NVIDIA
- -### License/Terms of Use:
-Apache 2.0
-## Use Case:
-Developers and engineers formulating vehicle routing optimization problems (VRP, TSP, PDP) who need to identify the correct problem type and required input data before using cuOpt APIs.
- -### Deployment Geography for Use:
-Global
- -## Known Risks and Mitigations:
-Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills.
-Mitigation: Review and scan skill before deployment.
- -## Reference(s):
-- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html)
-- [cuopt-examples](https://github.com/NVIDIA/cuopt-examples)
- - -## Skill Output:
-**Output Type(s):** [Analysis]
-**Output Format:** [Markdown]
-**Output Parameters:** [1D]
-**Other Properties Related to Output:** [None]
- -## Evaluation Agents Used:
-- `claude-code`
-- `codex`
- - - -## Evaluation Tasks:
-1 evaluation task (positive skill-activation case) with 2 attempts per task; pass threshold 50%. NVSkills-Eval profile: external.
- -## Evaluation Metrics Used:
-Reported benchmark dimensions:
-- Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
-- Correctness: Checks whether the agent follows the expected workflow and produces the correct final output.
-- Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
-- Effectiveness: Checks whether the agent performs measurably better with the skill than without it.
-- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work.
- -Underlying evaluation signals used in this run:
-- `skill_execution`: Verifies that the agent loaded the expected skill and workflow.
-- `skill_efficiency`: Checks routing quality, decoy avoidance, and redundant tool usage.
-- `accuracy`: Grades final-answer correctness against the reference answer.
-- `goal_accuracy`: Checks whether the overall user task completed successfully.
-- `behavior_check`: Verifies expected behavior steps, including safety expectations.
-- `token_efficiency`: Compares token usage with and without the skill.
- - - -## Evaluation Results:
-| Dimension | Num | `claude-code` | `codex` | -|---|---:|---:|---:| -| Security | 2 | 100% (+0%) | 100% (+0%) | -| Correctness | 2 | 100% (+0%) | 97% (+23%) | -| Discoverability | 2 | 100% (+0%) | 84% (+48%) | -| Effectiveness | 2 | 97% (-2%) | 98% (+0%) | -| Efficiency | 2 | 93% (-0%) | 78% (+34%) | - -## Skill Version(s):
-26.08.00 (source: frontmatter)
- -## Ethical Considerations:
-NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
- -(For Release on NVIDIA Platforms Only)
-Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail).
diff --git a/skills/cuopt-routing-formulation/skill.oms.sig b/skills/cuopt-routing-formulation/skill.oms.sig deleted file mode 100644 index fca3ee584b..0000000000 --- a/skills/cuopt-routing-formulation/skill.oms.sig +++ /dev/null @@ -1 +0,0 @@ -{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtcm91dGluZy1mb3JtdWxhdGlvbiIsCiAgICAgICJkaWdlc3QiOiB7CiAgICAgICAgInNoYTI1NiI6ICJlZWU3OTc2NjQxYmI3MjFjYmE4NTNlOGM4ODEyMTVmMGQ3YTRlMzQ4MGE4NWI3NDYyMGE3OTZlZjUxNzJlNWY0IgogICAgICB9CiAgICB9CiAgXSwKICAicHJlZGljYXRlVHlwZSI6ICJodHRwczovL21vZGVsX3NpZ25pbmcvc2lnbmF0dXJlL3YxLjAiLAogICJwcmVkaWNhdGUiOiB7CiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiYTI2Mzk4MTZjZmU3MzEyYWM1YTNjYTg4YTEyZmVjNWIxOTAxOGZmOTE4YjgzMjQ1MTdhZGUzMWU3NTViZjA4YSIsCiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiOGVmNDM1NWQxM2M1ZGRjY2ViNTM3MjI1OWQ5MGZlZGQ4YTc2ZmIzMDJiN2RiMGMyNWZiNWZjZTFjZWY4ZmZhZSIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgImRpZ2VzdCI6ICI3ZTdjMDMxNTI0MGU0NzIzZTI1ZTM1MjQzYWM4YTE5YjFkZjg4YzEyOThkMWRkNmJlZTY2Mjg2NzY3MjdiMDFhIiwKICAgICAgICAibmFtZSI6ICJldmFscy9ldmFscy5qc29uIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJkaWdlc3QiOiAiNmZhMWVmYjE4Mjg1YzZmYzkyNTJlNGEwMjQ3YmFhMzhjNTVjMWNhOTFiMGIwZmI0YzVhYTdjMWU4ODE0YzdmMSIsCiAgICAgICAgIm5hbWUiOiAic2tpbGwtY2FyZC5tZCIKICAgICAgfQogICAgXSwKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAiaWdub3JlX3BhdGhzIjogWwogICAgICAgICIuZ2l0IiwKICAgICAgICAiLmdpdGlnbm9yZSIsCiAgICAgICAgIi5naXRodWIiLAogICAgICAgICIuZ2l0YXR0cmlidXRlcyIKICAgICAgXSwKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImFsbG93X3N5bWxpbmtzIjogZmFsc2UKICAgIH0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGYCMQCI4t49EXYNez1SAdbQu+J0/GAgNkTruprFGAcZTyJcf+eNFrGjmvXRTPNC16SQZPICMQCI3lcxFduDYAZHsopvlQcXikisyJqUzPb/ZcgHnt6PDOC+cW80vVI/ki5iQ4iA8U4=","keyid":""}]}} \ No newline at end of file diff --git a/skills/cuopt-server-api-python/BENCHMARK.md b/skills/cuopt-server-api-python/BENCHMARK.md index c1bfc0cb63..cc0dda87e0 100644 --- a/skills/cuopt-server-api-python/BENCHMARK.md +++ b/skills/cuopt-server-api-python/BENCHMARK.md @@ -7,11 +7,11 @@ This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the s ## Evaluation Summary - Skill: `cuopt-server-api-python` -- Evaluation date: 2026-05-29 +- Evaluation date: 2026-06-29 - NVSkills-Eval profile: `external` -- Environment: `local` +- Environment: `astra-sandbox` - Dataset: 1 evaluation tasks -- Attempts per task: 2 +- Attempts per task: 1 - Pass threshold: 50% - Overall verdict: PASS @@ -54,34 +54,27 @@ Task composition is derived from the evaluation dataset when possible. Entries w | Dimension | Num | `claude-code` | `codex` | |---|---:|---:|---:| -| Security | 2 | 100% (+0%) | 100% (+0%) | -| Correctness | 2 | 100% (+0%) | 97% (+0%) | -| Discoverability | 2 | 100% (+0%) | 72% (+0%) | -| Effectiveness | 2 | 100% (+0%) | 100% (+0%) | -| Efficiency | 2 | 93% (-0%) | 56% (-1%) | +| Security | 1 | 100% (+0%) | 100% (+0%) | +| Correctness | 1 | 100% (+70%) | 97% (+35%) | +| Discoverability | 1 | 100% (+100%) | 97% (+69%) | +| Effectiveness | 1 | 100% (+56%) | 100% (+40%) | +| Efficiency | 1 | 95% (+68%) | 95% (+67%) | Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. ## Tier 1: Static Validation Summary -Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 15 total findings. +Tier 1 validation passed with observations. NVSkills-Eval ran 1 checks and found 3 total findings. Top findings: -- MEDIUM PII/gps_coordinates: GPS coordinates (location information) (`assets/lp_basic/client.py:40`) -- MEDIUM PII/gps_coordinates: GPS coordinates (location information) (`assets/lp_basic/client.py:47`) -- MEDIUM PII/gps_coordinates: GPS coordinates (location information) (`assets/lp_basic/client.py:51`) -- MEDIUM PII/gps_coordinates: GPS coordinates (location information) (`assets/milp_basic/client.py:38`) -- MEDIUM PII/gps_coordinates: GPS coordinates (location information) (`assets/milp_basic/client.py:44`) +- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-server-api-python/SKILL.md`) +- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/cuopt-server-api-python/SKILL.md`) +- LOW SCHEMA/author_format: Author must be of the form 'Name ' (`skills/cuopt-server-api-python/SKILL.md`) ## Tier 2: Deduplication Summary -Tier 2 validation passed. NVSkills-Eval ran 2 checks and found 0 total findings. - -Notable observations: - -- Context Deduplication: Collected 12 file(s) -- Inter-Skill Deduplication: Parsed skill 'cuopt-server-api-python': 129 char description +This tier was not run or did not produce findings in this report. ## Publication Recommendation diff --git a/skills/cuopt-server-api-python/SKILL.md b/skills/cuopt-server-api-python/SKILL.md index 88c6b2c6e8..ffaee3f792 100644 --- a/skills/cuopt-server-api-python/SKILL.md +++ b/skills/cuopt-server-api-python/SKILL.md @@ -19,6 +19,23 @@ metadata: This skill covers **starting the server** and **client examples** (curl, Python). Server has no separate C API (clients can be any language). +## Problem types supported + +| Problem type | Supported | +|--------------|:---------:| +| Routing | ✓ | +| LP | ✓ | +| MILP | ✓ | +| QP | ✗ | + +## Required questions + +Ask these if not already clear: + +1. **Problem type** — Routing or LP/MILP? (QP not available via REST.) +2. **Deployment** — Local, Docker, Kubernetes, or cloud? +3. **Client** — Which language or tool will call the API (e.g. Python, curl, another service)? + ## Start server ```bash diff --git a/skills/cuopt-server-api-python/skill-card.md b/skills/cuopt-server-api-python/skill-card.md index 5fec6f0803..e8754b2d76 100644 --- a/skills/cuopt-server-api-python/skill-card.md +++ b/skills/cuopt-server-api-python/skill-card.md @@ -9,36 +9,42 @@ NVIDIA
### License/Terms of Use:
Apache-2.0
## Use Case:
-Developers and engineers deploying, configuring, or calling the NVIDIA cuOpt REST server for vehicle routing (VRP, PDP), linear programming (LP), and mixed-integer linear programming (MILP) optimization workloads.
+Developers and engineers deploying the cuOpt optimization server and calling its REST API from Python or curl for vehicle routing, LP, and MILP workloads.
### Deployment Geography for Use:
Global
+## Requirements / Dependencies:
+**Requires API Key or External Credential:** [Not Specified]
+**Credential Type(s):** [None identified]
+ +Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate.
+ ## Known Risks and Mitigations:
Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills.
Mitigation: Review and scan skill before deployment.
## Reference(s):
- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html)
-- [cuOpt Examples](https://github.com/NVIDIA/cuopt-examples)
- [cuOpt Docker Hub](https://hub.docker.com/r/nvidia/cuopt)
-- [Runnable Assets (README)](assets/README.md)
+- [cuOpt Examples](https://github.com/NVIDIA/cuopt-examples)
+- [assets/README.md](assets/README.md)
## Skill Output:
-**Output Type(s):** [API Calls, Code, Shell commands, Configuration instructions]
-**Output Format:** [Markdown with inline Python and bash code blocks]
+**Output Type(s):** [API Calls, Shell commands, Code]
+**Output Format:** [Markdown with inline bash and Python code blocks]
**Output Parameters:** [1D]
**Other Properties Related to Output:** [None]
## Evaluation Agents Used:
-- Claude Code (`claude-code`)
-- Codex (`codex`)
+- `claude-code`
+- `codex`
## Evaluation Tasks:
-Evaluated against 1 internal evaluation task (positive skill-activation) with 2 attempts per task via NVSkills-Eval (external profile, local environment). Pass threshold: 50%.
+Evaluated against 1 evaluation task (1 positive skill-activation case, 0 negative cases) in the NVSkills-Eval external profile, astra-sandbox environment.
## Evaluation Metrics Used:
Reported benchmark dimensions:
@@ -62,11 +68,11 @@ Underlying evaluation signals used in this run:
## Evaluation Results:
| Dimension | Num | `claude-code` | `codex` | |---|---:|---:|---:| -| Security | 2 | 100% (+0%) | 100% (+0%) | -| Correctness | 2 | 100% (+0%) | 97% (+0%) | -| Discoverability | 2 | 100% (+0%) | 72% (+0%) | -| Effectiveness | 2 | 100% (+0%) | 100% (+0%) | -| Efficiency | 2 | 93% (-0%) | 56% (-1%) | +| Security | 1 | 100% (+0%) | 100% (+0%) | +| Correctness | 1 | 100% (+70%) | 97% (+35%) | +| Discoverability | 1 | 100% (+100%) | 97% (+69%) | +| Effectiveness | 1 | 100% (+56%) | 100% (+40%) | +| Efficiency | 1 | 95% (+68%) | 95% (+67%) | ## Skill Version(s):
26.08.00 (source: frontmatter)
diff --git a/skills/cuopt-server-api-python/skill.oms.sig b/skills/cuopt-server-api-python/skill.oms.sig index e176928b61..7a336c4da8 100644 --- a/skills/cuopt-server-api-python/skill.oms.sig +++ b/skills/cuopt-server-api-python/skill.oms.sig @@ -1 +1 @@ -{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtc2VydmVyLWFwaS1weXRob24iLAogICAgICAiZGlnZXN0IjogewogICAgICAgICJzaGEyNTYiOiAiZDYwOTgzMDYyN2M0ZTQ3YTJmMmM0NjM2ZDg5YzIwMWQ3MDczMWFjMzQxZDQ0ZTczODkzM2E1YjVjZjE5MWViOSIKICAgICAgfQogICAgfQogIF0sCiAgInByZWRpY2F0ZVR5cGUiOiAiaHR0cHM6Ly9tb2RlbF9zaWduaW5nL3NpZ25hdHVyZS92MS4wIiwKICAicHJlZGljYXRlIjogewogICAgInNlcmlhbGl6YXRpb24iOiB7CiAgICAgICJhbGxvd19zeW1saW5rcyI6IGZhbHNlLAogICAgICAiaWdub3JlX3BhdGhzIjogWwogICAgICAgICIuZ2l0aHViIiwKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiLAogICAgICAgICIuZ2l0IiwKICAgICAgICAiLmdpdGlnbm9yZSIKICAgICAgXSwKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAibWV0aG9kIjogImZpbGVzIgogICAgfSwKICAgICJyZXNvdXJjZXMiOiBbCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIiwKICAgICAgICAiZGlnZXN0IjogIjNhZmNiZTk1OWYwYTE3MjJkYzA3OGM1NzA0OWJhNDZhMTc4NTExMTcyNjgxMDNmMTVmZjA4ZjUwOTFkZmFhOTMiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiLAogICAgICAgICJkaWdlc3QiOiAiYWQ0NDk1ODMzMWM3MGM3NjEzNzFiMWQ1MTc5NGYxMDcyMDM1OGQ2YmMxNmNjOWU0YjM1ZGJjNzlmYWQ2NWM4OSIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvUkVBRE1FLm1kIiwKICAgICAgICAiZGlnZXN0IjogImE4M2NjZWIxMDFmZWIyODk1M2JlOWZhMDY4OWY3MzE3NDY3NDkxNGU2ZWNhYTJjZWE5M2RmMTAyZTAwMTE0ZmYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL2xwX2Jhc2ljL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICIxYzZlODllZWVlODhkZTdkMjk2OGM2ODRjZjhiNDViYTliZTBjMmU0MjQxZTA5NGYzMGY3MmM2NzAzNGYxZjdiIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9scF9iYXNpYy9jbGllbnQucHkiLAogICAgICAgICJkaWdlc3QiOiAiNmE2ZmY1MmZlYzVjOGZjMmQ1YzUyZjI4YjkwZDYzYjg1NWI0NTk1ZTg0NzFmZTVkMjhhNTA3MTkwMDA3NmVlYiIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvbWlscF9iYXNpYy9SRUFETUUubWQiLAogICAgICAgICJkaWdlc3QiOiAiZmFhNGVlMTBhNjU4NTgzOWFlYjQ0OTJlMDc3MTM2MDM4ZWVlYjBiY2RlYmI5MmExOGIyNGVhYTVkZWQyZTY1OSIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvbWlscF9iYXNpYy9jbGllbnQucHkiLAogICAgICAgICJkaWdlc3QiOiAiZTZkY2VkMWVjNWRjNjcyZDMzYjI4M2UwOTJiMzkwNGE0ODcwYWUxMDVmYjE3ZTM5MDQzMGQ1ODNmNzI0MDlhNyIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvcGRwX2Jhc2ljL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI1YzBiN2UzYzM1ZWIxMTFmOTI0NmQ0OWE4MDIyMTA4YjRkNGU0ZWUyODRiZWYyODNmNWFhMjEyMGZiYzNlZDBkIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9wZHBfYmFzaWMvY2xpZW50LnB5IiwKICAgICAgICAiZGlnZXN0IjogIjk4MWJjYTA5NTFhYzlkODUwMDc0YmNjMzA1YmE2ZjIxYTcwOTQ1ZjA4MGM4MjkwOTQ3Y2U3ODQwMjhiZTE5ZmUiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3ZycF9iYXNpYy9SRUFETUUubWQiLAogICAgICAgICJkaWdlc3QiOiAiNjkyODA3NDQxM2RmZTFlYTQxNWJmMWRlZTc5Y2ViYzE4NjU0NmZlY2E4ZGM0MGZlZTFlNDJhMjk2NTFlNTE2NCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvdnJwX2Jhc2ljL2NsaWVudC5weSIsCiAgICAgICAgImRpZ2VzdCI6ICI2M2QxZWI2ZGYwYzg0MTc3MDkzODA0YTY2MTg5ZmI2YWFlNTBhN2VhNGQ3Y2RiNzQyNWU3YTYxOWNjYzBiMTM2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy92cnBfc2ltcGxlL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI2ZWM0NWJiNWE1ZTBmMWExMjJmMjQxMTc5MTg3YzRlNjA4Y2JjYTg4ZDgwODFkYTQyMjkxNjRkMjgxMzE2YjVjIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy92cnBfc2ltcGxlL2NsaWVudC5weSIsCiAgICAgICAgImRpZ2VzdCI6ICJmN2UxYWU0OTYwN2M5NGUzYzgxOGIxNjkwMDZhMzgzNGM3NjJjMTU4ZmZjMmY0MzYyOTVkMTgwYzgyNTllZTU5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImV2YWxzL2V2YWxzLmpzb24iLAogICAgICAgICJkaWdlc3QiOiAiYjY5NTliYWIxMDNhNWFkM2M2ODY2NTdjZTBkNDVkNzllYWE4OTliNmYzYjk2ZDEwZDg3MjFiZmY4ZWYzNjcxOSIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJza2lsbC1jYXJkLm1kIiwKICAgICAgICAiZGlnZXN0IjogIjUxZDZkYWExYTAyMDRlMzc0Njk2MzI5YjY4ODQxM2VmMWEzNjk4YTIwMTg1OTQzNmEwNGMzZTE0OGExZGE2MmUiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGQCMD9XlXXfUjWnSotdcJo8X67QmnqfH2KPf3zBDiAKb7lVAglL8x8Jcy5BjiGmOwN4TAIwHwNJSUzG0ikdSCDIZ6+gO+fl6TjrOyfXngbDKegwc1cxfdLl6bz/avOpXngP7gii","keyid":""}]}} \ No newline at end of file +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtc2VydmVyLWFwaS1weXRob24iLAogICAgICAiZGlnZXN0IjogewogICAgICAgICJzaGEyNTYiOiAiNDJhZWQzNGVkOWU0Yzg4Y2ZhZmE1YWQzZWM3MzIwZjY5ZDAzMTJkYjVkZjkwMTZlNmVhZTNlNDIzYzI1M2FmNSIKICAgICAgfQogICAgfQogIF0sCiAgInByZWRpY2F0ZVR5cGUiOiAiaHR0cHM6Ly9tb2RlbF9zaWduaW5nL3NpZ25hdHVyZS92MS4wIiwKICAicHJlZGljYXRlIjogewogICAgInJlc291cmNlcyI6IFsKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJCRU5DSE1BUksubWQiLAogICAgICAgICJkaWdlc3QiOiAiYmI3NDQ3NGQwMmExMzAxMTA2YTA2ZWE1ZTBhNTI3MDE0ODQ2YjA1MGQxODQzOTk4NDQ1MGRlZGM3MmRmZTRkNCIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJTS0lMTC5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI5ZjdmMTkxNzA4MDc5YjdlYTgwZjgwYzRlMTZjYjBhMDNkMTc4MmMyYmZhNjM4MDg3NTBlNGNhNTBkMzhmMTE5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9SRUFETUUubWQiLAogICAgICAgICJkaWdlc3QiOiAiYTgzY2NlYjEwMWZlYjI4OTUzYmU5ZmEwNjg5ZjczMTc0Njc0OTE0ZTZlY2FhMmNlYTkzZGYxMDJlMDAxMTRmZiIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvbHBfYmFzaWMvUkVBRE1FLm1kIiwKICAgICAgICAiZGlnZXN0IjogIjFjNmU4OWVlZWU4OGRlN2QyOTY4YzY4NGNmOGI0NWJhOWJlMGMyZTQyNDFlMDk0ZjMwZjcyYzY3MDM0ZjFmN2IiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL2xwX2Jhc2ljL2NsaWVudC5weSIsCiAgICAgICAgImRpZ2VzdCI6ICI2YTZmZjUyZmVjNWM4ZmMyZDVjNTJmMjhiOTBkNjNiODU1YjQ1OTVlODQ3MWZlNWQyOGE1MDcxOTAwMDc2ZWViIgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9taWxwX2Jhc2ljL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICJmYWE0ZWUxMGE2NTg1ODM5YWViNDQ5MmUwNzcxMzYwMzhlZWViMGJjZGViYjkyYTE4YjI0ZWFhNWRlZDJlNjU5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9taWxwX2Jhc2ljL2NsaWVudC5weSIsCiAgICAgICAgImRpZ2VzdCI6ICJlNmRjZWQxZWM1ZGM2NzJkMzNiMjgzZTA5MmIzOTA0YTQ4NzBhZTEwNWZiMTdlMzkwNDMwZDU4M2Y3MjQwOWE3IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy9wZHBfYmFzaWMvUkVBRE1FLm1kIiwKICAgICAgICAiZGlnZXN0IjogIjVjMGI3ZTNjMzVlYjExMWY5MjQ2ZDQ5YTgwMjIxMDhiNGQ0ZTRlZTI4NGJlZjI4M2Y1YWEyMTIwZmJjM2VkMGQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3BkcF9iYXNpYy9jbGllbnQucHkiLAogICAgICAgICJkaWdlc3QiOiAiOTgxYmNhMDk1MWFjOWQ4NTAwNzRiY2MzMDViYTZmMjFhNzA5NDVmMDgwYzgyOTA5NDdjZTc4NDAyOGJlMTlmZSIKICAgICAgfSwKICAgICAgewogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJhc3NldHMvdnJwX2Jhc2ljL1JFQURNRS5tZCIsCiAgICAgICAgImRpZ2VzdCI6ICI2OTI4MDc0NDEzZGZlMWVhNDE1YmYxZGVlNzljZWJjMTg2NTQ2ZmVjYThkYzQwZmVlMWU0MmEyOTY1MWU1MTY0IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogImFzc2V0cy92cnBfYmFzaWMvY2xpZW50LnB5IiwKICAgICAgICAiZGlnZXN0IjogIjYzZDFlYjZkZjBjODQxNzcwOTM4MDRhNjYxODlmYjZhYWU1MGE3ZWE0ZDdjZGI3NDI1ZTdhNjE5Y2NjMGIxMzYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3ZycF9zaW1wbGUvUkVBRE1FLm1kIiwKICAgICAgICAiZGlnZXN0IjogIjZlYzQ1YmI1YTVlMGYxYTEyMmYyNDExNzkxODdjNGU2MDhjYmNhODhkODA4MWRhNDIyOTE2NGQyODEzMTZiNWMiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiYXNzZXRzL3ZycF9zaW1wbGUvY2xpZW50LnB5IiwKICAgICAgICAiZGlnZXN0IjogImY3ZTFhZTQ5NjA3Yzk0ZTNjODE4YjE2OTAwNmEzODM0Yzc2MmMxNThmZmMyZjQzNjI5NWQxODBjODI1OWVlNTkiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIsCiAgICAgICAgImRpZ2VzdCI6ICJiNjk1OWJhYjEwM2E1YWQzYzY4NjY1N2NlMGQ0NWQ3OWVhYTg5OWI2ZjNiOTZkMTBkODcyMWJmZjhlZjM2NzE5IgogICAgICB9LAogICAgICB7CiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiLAogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiLAogICAgICAgICJkaWdlc3QiOiAiZDA2ZTc2ZmE4ZDllMDE2NTMzZDBiNDAwNDRlZTcxMjRlMzY2NGI4MzhhZDRlMTAyMDgzZGQ3ODkxNTRlZGMwZCIKICAgICAgfQogICAgXSwKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAiaGFzaF90eXBlIjogInNoYTI1NiIsCiAgICAgICJhbGxvd19zeW1saW5rcyI6IGZhbHNlLAogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdCIsCiAgICAgICAgIi5naXRhdHRyaWJ1dGVzIiwKICAgICAgICAiLmdpdGlnbm9yZSIsCiAgICAgICAgIi5naXRodWIiCiAgICAgIF0KICAgIH0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGUCMAa1Jdb+1CJ9sMsUOcXOh16duwRQhqlEiZSTVilJ+GovK1FjYC5xbormKRaUi6PZfQIxAKgbOJfCp0SqR7g46TRUldmvHLaNeE4gZp2ZBw8d90XKuhjT/mO7998tKfIkst3/dA==","keyid":""}]}} \ No newline at end of file diff --git a/skills/cuopt-server-common/BENCHMARK.md b/skills/cuopt-server-common/BENCHMARK.md deleted file mode 100644 index 188f44efc8..0000000000 --- a/skills/cuopt-server-common/BENCHMARK.md +++ /dev/null @@ -1,87 +0,0 @@ -# Evaluation Report - -Evaluation of the `cuopt-server-common` skill before publication through NVSkills-Eval. - -This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the skill. The goal is to document whether the skill is safe, discoverable, effective, and useful for agents before it is published for broader workflow use. - -## Evaluation Summary - -- Skill: `cuopt-server-common` -- Evaluation date: 2026-05-28 -- NVSkills-Eval profile: `external` -- Environment: `local` -- Dataset: 1 evaluation tasks -- Attempts per task: 2 -- Pass threshold: 50% -- Overall verdict: PASS - -## Agents Used - -- `claude-code` -- `codex` - -## Metrics Used - -Reported benchmark dimensions: - -- Security: checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access. -- Correctness: checks whether the agent follows the expected workflow and produces the correct final output. -- Discoverability: checks whether the agent loads the skill when relevant and avoids using it when irrelevant. -- Effectiveness: checks whether the agent performs measurably better with the skill than without it. -- Efficiency: checks whether the agent uses fewer tokens and avoids redundant work. - -Underlying evaluation signals used in this run: - -- `skill_execution` (Skill Execution): verifies that the agent loaded the expected skill and workflow. -- `skill_efficiency` (Efficiency): checks routing quality, decoy avoidance, and redundant tool usage. -- `accuracy` (Accuracy): grades final-answer correctness against the reference answer. -- `goal_accuracy` (Goal Accuracy): checks whether the overall user task completed successfully. -- `behavior_check` (Behavior Check): verifies expected behavior steps, including safety expectations. -- `token_efficiency` (Token Efficiency): compares token usage with and without the skill. - -## Test Tasks - -The benchmark dataset contained 1 evaluation tasks: - -- Positive tasks: 1 tasks where the skill was expected to activate. -- Negative tasks: 0 tasks where no skill was expected. -- Unlabeled tasks: 0 tasks where positive/negative intent could not be inferred. - -Task composition is derived from the evaluation dataset when possible. Entries with `expected_skill` set are treated as positive skill-activation cases, while entries with `expected_skill: null` are treated as negative activation cases. - -## Results - -| Dimension | Num | `claude-code` | `codex` | -|---|---:|---:|---:| -| Security | 2 | 100% (+0%) | 50% (+0%) | -| Correctness | 2 | 100% (+8%) | 69% (+5%) | -| Discoverability | 2 | 100% (+33%) | 59% (+0%) | -| Effectiveness | 2 | 98% (+1%) | 50% (+0%) | -| Efficiency | 2 | 93% (+35%) | 43% (-6%) | - -Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. - -## Tier 1: Static Validation Summary - -Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 11 total findings. - -Top findings: - -- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-server-common/SKILL.md`) -- MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/cuopt-server-common/SKILL.md`) -- LOW QUALITY/quality_correctness: No examples provided (`skills/cuopt-server-common/SKILL.md`) -- LOW QUALITY/quality_discoverability: Description doesn't mention WHEN to use this skill (`skills/cuopt-server-common/SKILL.md`) -- LOW QUALITY/quality_discoverability: No '## Purpose' section (`skills/cuopt-server-common/SKILL.md`) - -## Tier 2: Deduplication Summary - -Tier 2 validation passed. NVSkills-Eval ran 2 checks and found 0 total findings. - -Notable observations: - -- Context Deduplication: Collected 1 file(s) -- Inter-Skill Deduplication: Parsed skill 'cuopt-server-common': 98 char description - -## Publication Recommendation - -The skill is suitable to proceed toward NVSkills-Eval publication based on this benchmark. Skill owners should keep this file with the skill and refresh it when the evaluation dataset, skill behavior, or target agents materially change. diff --git a/skills/cuopt-server-common/SKILL.md b/skills/cuopt-server-common/SKILL.md deleted file mode 100644 index b8c643b6fd..0000000000 --- a/skills/cuopt-server-common/SKILL.md +++ /dev/null @@ -1,55 +0,0 @@ ---- -name: cuopt-server-common -version: "26.08.00" -description: cuOpt REST server — what it does and how requests flow. Domain concepts; no deploy or client code. -license: Apache-2.0 -metadata: - author: NVIDIA cuOpt Team - tags: - - cuopt - - server - - rest-api - - concepts ---- - - -# cuOpt Server (common) - -Domain concepts for the cuOpt REST server. No deploy commands or client code here. - -## What the server does - -- Accepts optimization requests (routing, LP, MILP) over HTTP. -- Returns a request ID; solution is obtained by polling with that ID. -- Does **not** support QP via REST. - -## Problem types supported - -| Problem type | Supported | -|--------------|:---------:| -| Routing | ✓ | -| LP | ✓ | -| MILP | ✓ | -| QP | ✗ | - -## Request flow (conceptual) - -1. Client sends problem data in the required schema (matrices, tasks, fleet, solver config). -2. Server returns a `reqId`. -3. Client polls the solution endpoint with `reqId` until the job completes. -4. Response contains status and, on success, solution (routes, objective, primal values, etc.). - -## Required questions (deployment and usage) - -Ask these if not already clear: - -1. **Problem type** — Routing or LP/MILP? (QP not available.) -2. **Deployment** — Local, Docker, Kubernetes, or cloud? -3. **Client** — Which language or tool will call the API (e.g. Python, curl, another service)? - -## Key endpoints (conceptual) - -- Health check. -- Submit request (POST). -- Get solution by request ID (GET). -- OpenAPI spec (e.g. for payload format). diff --git a/skills/cuopt-server-common/evals/evals.json b/skills/cuopt-server-common/evals/evals.json deleted file mode 100644 index bb6bcafcb1..0000000000 --- a/skills/cuopt-server-common/evals/evals.json +++ /dev/null @@ -1,13 +0,0 @@ -[ - { - "id": "srv-common-eval-001-qp-not-supported-over-rest", - "question": "I want to submit a Quadratic Programming (QP) problem to the cuOpt REST server. Can I do that? If yes, walk me through the submission endpoint; if no, explain why and what my options are.", - "expected_skill": "cuopt-server-common", - "expected_script": null, - "ground_truth": "The agent states clearly that QP is NOT supported over the cuOpt REST server. The REST server accepts routing, LP, and MILP problems only. For QP, the user must use a non-REST interface (cuOpt Python API or C API). The agent does not fabricate a QP submission endpoint or pretend QP works via REST.", - "expected_behavior": [ - "States explicitly that QP is NOT supported via the cuOpt REST server (REST accepts routing, LP, MILP only)", - "Directs the user to a non-REST interface (cuOpt Python or C API) for QP and does not invent a QP REST endpoint" - ] - } -] diff --git a/skills/cuopt-server-common/skill-card.md b/skills/cuopt-server-common/skill-card.md deleted file mode 100644 index af9d9475df..0000000000 --- a/skills/cuopt-server-common/skill-card.md +++ /dev/null @@ -1,75 +0,0 @@ -## Description:
-cuOpt REST server — what it does and how requests flow. Domain concepts; no deploy or client code.
- -This skill is ready for commercial/non-commercial use.
- -## Owner -NVIDIA
- -### License/Terms of Use:
-Apache-2.0
-## Use Case:
-Developers and engineers working with NVIDIA cuOpt who need to understand the REST server’s capabilities, supported problem types, and request flow before submitting optimization workloads.
- -### Deployment Geography for Use:
-Global
- -## Known Risks and Mitigations:
-Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills.
-Mitigation: Review and scan skill before deployment.
- -## Reference(s):
-- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html)
-- [cuOpt Examples](https://github.com/NVIDIA/cuopt-examples)
- - -## Skill Output:
-**Output Type(s):** [Analysis, Configuration instructions]
-**Output Format:** [Markdown]
-**Output Parameters:** [1D]
-**Other Properties Related to Output:** [None]
- -## Evaluation Agents Used:
-- Claude Code (`claude-code`)
-- Codex (`codex`)
- - - -## Evaluation Tasks:
-Evaluated against 1 internal evaluation task (positive skill-activation case) with 2 attempts per task at 50% pass threshold.
- -## Evaluation Metrics Used:
-Reported benchmark dimensions:
-- Security: Checks whether skill-assisted execution avoids unsafe behavior such as secret leakage, destructive commands, or unauthorized access.
-- Correctness: Checks whether the agent follows the expected workflow and produces the correct final output.
-- Discoverability: Checks whether the agent loads the skill when relevant and avoids using it when irrelevant.
-- Effectiveness: Checks whether the agent performs measurably better with the skill than without it.
-- Efficiency: Checks whether the agent uses fewer tokens and avoids redundant work.
- -Underlying evaluation signals used in this run:
-- `skill_execution`: Verifies that the agent loaded the expected skill and workflow.
-- `skill_efficiency`: Checks routing quality, decoy avoidance, and redundant tool usage.
-- `accuracy`: Grades final-answer correctness against the reference answer.
-- `goal_accuracy`: Checks whether the overall user task completed successfully.
-- `behavior_check`: Verifies expected behavior steps, including safety expectations.
-- `token_efficiency`: Compares token usage with and without the skill.
- - - -## Evaluation Results:
-| Dimension | Num | `claude-code` | `codex` | -|---|---:|---:|---:| -| Security | 2 | 100% (+0%) | 50% (+0%) | -| Correctness | 2 | 100% (+8%) | 69% (+5%) | -| Discoverability | 2 | 100% (+33%) | 59% (+0%) | -| Effectiveness | 2 | 98% (+1%) | 50% (+0%) | -| Efficiency | 2 | 93% (+35%) | 43% (-6%) | - -## Skill Version(s):
-26.08.00 (source: frontmatter)
- -## Ethical Considerations:
-NVIDIA believes Trustworthy AI is a shared responsibility and we have established policies and practices to enable development for a wide array of AI applications. When downloaded or used in accordance with our terms of service, developers should work with their internal team to ensure this skill meets requirements for the relevant industry and use case and addresses unforeseen product misuse.
- -(For Release on NVIDIA Platforms Only)
-Please report quality, risk, security vulnerabilities or NVIDIA AI Concerns [here](https://app.intigriti.com/programs/nvidia/nvidiavdp/detail).
diff --git a/skills/cuopt-server-common/skill.oms.sig b/skills/cuopt-server-common/skill.oms.sig deleted file mode 100644 index 4a86a5ef5a..0000000000 --- a/skills/cuopt-server-common/skill.oms.sig +++ /dev/null @@ -1 +0,0 @@ -{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtc2VydmVyLWNvbW1vbiIsCiAgICAgICJkaWdlc3QiOiB7CiAgICAgICAgInNoYTI1NiI6ICIzNzg2NjcyYmEzNzFjMmJhMTFmMWMxZTE3MTQyZWQyMmM4MDU3ZTQyZjdjMjhiOGE1YjMzN2ZlYmI1M2I1NTdlIgogICAgICB9CiAgICB9CiAgXSwKICAicHJlZGljYXRlVHlwZSI6ICJodHRwczovL21vZGVsX3NpZ25pbmcvc2lnbmF0dXJlL3YxLjAiLAogICJwcmVkaWNhdGUiOiB7CiAgICAic2VyaWFsaXphdGlvbiI6IHsKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdCIsCiAgICAgICAgIi5naXRodWIiLAogICAgICAgICIuZ2l0YXR0cmlidXRlcyIsCiAgICAgICAgIi5naXRpZ25vcmUiCiAgICAgIF0sCiAgICAgICJoYXNoX3R5cGUiOiAic2hhMjU2IiwKICAgICAgImFsbG93X3N5bWxpbmtzIjogZmFsc2UsCiAgICAgICJtZXRob2QiOiAiZmlsZXMiCiAgICB9LAogICAgInJlc291cmNlcyI6IFsKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiODEwOGFmYTI3ODc5NTBlOWUwNjNmYjUxZTk2NWVmZmRjODdkOTY4NTRiY2VhMmZmYWNlMDBhZjFiNjM1N2NiZiIsCiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiOTM3NmZiMDVjN2M0OTQ0MTU1ZmIxZDUwY2IxZGM1MjI4YzI0NGIwZmU1MzRmNjZkZTY4ZTViMGUwMzM2YjRkNCIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICIyYTZhYjExMDdlNTMzOTI2ODJjYzZhY2QxNWNmMDRkNTcxNDA4OGM4NWQ2NGQyN2NkZTNmMjY3YzU3NzNmMTkyIiwKICAgICAgICAibmFtZSI6ICJldmFscy9ldmFscy5qc29uIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiNzExNmJhNmY0M2NjYzhjMzRkMzY5Y2EwYzI3NGY3NzllODUzZGQ4NjAzNjAyOTY0NGU5NzMzY2FhZTIyMGZhMiIsCiAgICAgICAgIm5hbWUiOiAic2tpbGwtY2FyZC5tZCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0KICAgIF0KICB9Cn0=","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGQCMFyQdQ8YF2IsSbnQXomp4JZQb3ebF9fby5jDsW5EMiOwurIJNlbWgTqEDNbtS+tb3wIwDe4BNNpRibgiWpA8yZOIQiIiaTN2Xpsk5OwwhTwkzlu9uVQoWgih2jD3wWNvvuxG","keyid":""}]}} \ No newline at end of file diff --git a/skills/cuopt-skill-evolution/BENCHMARK.md b/skills/cuopt-skill-evolution/BENCHMARK.md index 37166215bd..179502aa3f 100644 --- a/skills/cuopt-skill-evolution/BENCHMARK.md +++ b/skills/cuopt-skill-evolution/BENCHMARK.md @@ -7,11 +7,11 @@ This benchmark summarizes 3-Tier Evaluation from NVSkills-Eval results for the s ## Evaluation Summary - Skill: `cuopt-skill-evolution` -- Evaluation date: 2026-05-29 +- Evaluation date: 2026-06-29 - NVSkills-Eval profile: `external` -- Environment: `local` +- Environment: `astra-sandbox` - Dataset: 1 evaluation tasks -- Attempts per task: 2 +- Attempts per task: 1 - Pass threshold: 50% - Overall verdict: PASS @@ -54,34 +54,27 @@ Task composition is derived from the evaluation dataset when possible. Entries w | Dimension | Num | `claude-code` | `codex` | |---|---:|---:|---:| -| Security | 2 | 100% (+0%) | 100% (+0%) | -| Correctness | 2 | 90% (-5%) | 97% (+0%) | -| Discoverability | 2 | 100% (+12%) | 84% (+12%) | -| Effectiveness | 2 | 60% (-1%) | 66% (+2%) | -| Efficiency | 2 | 93% (+19%) | 76% (+19%) | +| Security | 1 | 100% (+0%) | 100% (+0%) | +| Correctness | 1 | 25% (+0%) | 0% (+0%) | +| Discoverability | 1 | 50% (+0%) | 0% (+0%) | +| Effectiveness | 1 | 14% (+0%) | 14% (+0%) | +| Efficiency | 1 | 60% (-0%) | 28% (-0%) | Score values show skill-assisted performance. Values in parentheses show uplift versus the no-skill baseline when baseline data is available. ## Tier 1: Static Validation Summary -Tier 1 validation passed with observations. NVSkills-Eval ran 9 checks and found 9 total findings. +Tier 1 validation passed with observations. NVSkills-Eval ran 1 checks and found 3 total findings. Top findings: -- MEDIUM QUALITY/quality_discoverability: Description contains vague words (`skills/cuopt-skill-evolution/SKILL.md`) - MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Instructions' (`skills/cuopt-skill-evolution/SKILL.md`) - MEDIUM SCHEMA/body_recommended_section: Missing recommended section: '## Examples' (`skills/cuopt-skill-evolution/SKILL.md`) -- LOW QUALITY/quality_discoverability: Description doesn't mention WHEN to use this skill (`skills/cuopt-skill-evolution/SKILL.md`) -- LOW QUALITY/quality_discoverability: No '## Purpose' section (`skills/cuopt-skill-evolution/SKILL.md`) +- LOW SCHEMA/author_format: Author must be of the form 'Name ' (`skills/cuopt-skill-evolution/SKILL.md`) ## Tier 2: Deduplication Summary -Tier 2 validation passed. NVSkills-Eval ran 2 checks and found 0 total findings. - -Notable observations: - -- Context Deduplication: Collected 1 file(s) -- Inter-Skill Deduplication: Parsed skill 'cuopt-skill-evolution': 140 char description +This tier was not run or did not produce findings in this report. ## Publication Recommendation diff --git a/skills/cuopt-skill-evolution/SKILL.md b/skills/cuopt-skill-evolution/SKILL.md index 7dcb3002ba..13b3025c86 100644 --- a/skills/cuopt-skill-evolution/SKILL.md +++ b/skills/cuopt-skill-evolution/SKILL.md @@ -97,7 +97,7 @@ If a draft proposal feels heavy-handed or rigid, rewrite it as if explaining the Always place the learning in the **single skill where it has the widest effect**. Do NOT duplicate the same content across multiple skills. Choose the target using this priority: -1. **Common / concept skill** (e.g. `cuopt-numerical-optimization-formulation`, `cuopt-routing-formulation`, `cuopt-user-rules`) — if the learning applies regardless of language or interface, put it here. All downstream API skills already read the common skill. +1. **Common / concept skill** (e.g. `cuopt-numerical-optimization-formulation`, `cuopt-user-rules`) — if the learning applies regardless of language or interface, put it here. All downstream API skills already read the common skill. 2. **API skill** (e.g. `cuopt-numerical-optimization-api-python`, `cuopt-routing-api-python`) — if the learning is specific to one API or language. 3. **New skill** — only if the learning doesn't fit any existing skill. diff --git a/skills/cuopt-skill-evolution/skill-card.md b/skills/cuopt-skill-evolution/skill-card.md index c14c953444..31eeaff335 100644 --- a/skills/cuopt-skill-evolution/skill-card.md +++ b/skills/cuopt-skill-evolution/skill-card.md @@ -1,5 +1,5 @@ ## Description:
-After solving a non-trivial problem, detect generalizable learnings and propose skill updates.
+After solving a non-trivial problem, detect generalizable learnings and propose skill updates. Always active — applies to every interaction.
This skill is ready for commercial/non-commercial use.
@@ -9,34 +9,40 @@ NVIDIA
### License/Terms of Use:
Apache 2.0
## Use Case:
-Developers and engineers using cuOpt AI agent skills who need to capture generalizable learnings from corrections, failures, and undocumented behaviors, and propose structured skill updates.
+Developers and engineers working with NVIDIA cuOpt optimization skills use this to detect generalizable learnings from problem-solving interactions and propose skill updates that improve future agent behavior.
### Deployment Geography for Use:
Global
+## Requirements / Dependencies:
+**Requires API Key or External Credential:** [Not Specified]
+**Credential Type(s):** [None identified]
+ +Do not include secrets in prompts/logs/output; use least-privilege credentials; rotate keys as appropriate.
+ ## Known Risks and Mitigations:
Risk: Review before execution as proposals could introduce incorrect or misleading guidance into skills.
Mitigation: Review and scan skill before deployment.
## Reference(s):
-- [SKILL.md (Skill Evolution workflow)](skills/cuopt-skill-evolution/SKILL.md)
-- [cuOpt User Guide](https://docs.nvidia.com/cuopt/user-guide/latest/introduction.html)
+- [SKILL.md](SKILL.md)
+- [BENCHMARK.md](BENCHMARK.md)
## Skill Output:
**Output Type(s):** [Analysis, Code]
-**Output Format:** [Markdown with inline code diffs]
+**Output Format:** [Markdown with inline code blocks]
**Output Parameters:** [1D]
-**Other Properties Related to Output:** [Proposals require explicit user approval before application]
+**Other Properties Related to Output:** [Proposals require user approval before application]
## Evaluation Agents Used:
-- claude-code
-- codex
+- `claude-code`
+- `codex`
## Evaluation Tasks:
-Evaluated against 1 evaluation task (1 positive skill-activation case) with 2 attempts per task, pass threshold 50%.
+Evaluated against 1 evaluation task in the astra-sandbox environment using the NVSkills-Eval external profile with a 50% pass threshold.
## Evaluation Metrics Used:
Reported benchmark dimensions:
@@ -60,11 +66,11 @@ Underlying evaluation signals used in this run:
## Evaluation Results:
| Dimension | Num | `claude-code` | `codex` | |---|---:|---:|---:| -| Security | 2 | 100% (+0%) | 100% (+0%) | -| Correctness | 2 | 90% (-5%) | 97% (+0%) | -| Discoverability | 2 | 100% (+12%) | 84% (+12%) | -| Effectiveness | 2 | 60% (-1%) | 66% (+2%) | -| Efficiency | 2 | 93% (+19%) | 76% (+19%) | +| Security | 1 | 100% (+0%) | 100% (+0%) | +| Correctness | 1 | 25% (+0%) | 0% (+0%) | +| Discoverability | 1 | 50% (+0%) | 0% (+0%) | +| Effectiveness | 1 | 14% (+0%) | 14% (+0%) | +| Efficiency | 1 | 60% (-0%) | 28% (-0%) | ## Skill Version(s):
26.08.00 (source: frontmatter)
diff --git a/skills/cuopt-skill-evolution/skill.oms.sig b/skills/cuopt-skill-evolution/skill.oms.sig index 2208036693..9e7d3bf8e3 100644 --- a/skills/cuopt-skill-evolution/skill.oms.sig +++ b/skills/cuopt-skill-evolution/skill.oms.sig @@ -1 +1 @@ -{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtc2tpbGwtZXZvbHV0aW9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjI2OTQxNzQwYzNlMzE3MDExMjI0OGVkNWI0ZmRhOWQ4ZjYwZWU2NTcwNGFkOGMxMDY1YmE5MmM5MGI4Nzc4N2MiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJyZXNvdXJjZXMiOiBbCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogIjE1YjY1ODIxNGQzM2U2OTM4OTdiMGMxNGNjYzBmMTRmNDRlOWUwMjNmOTM2YTdlZDhjNWE4NzMwNmQ4YjNlOWQiLAogICAgICAgICJuYW1lIjogIkJFTkNITUFSSy5tZCIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogIjY5Y2ZmYzRmNGE0MjIwOGRmYzA0MDYxOGZiOTY5NDVkMmE4MWM0OWIwNGRlZTEwOTZiYzkzODc4MGIxNDJhN2YiLAogICAgICAgICJuYW1lIjogIlNLSUxMLm1kIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIKICAgICAgfSwKICAgICAgewogICAgICAgICJkaWdlc3QiOiAiNmM4ZmU4YTQxNGFiODdmOTQwZDVlZjJhYjUyMWI5Yjc3ODg3MGE3MGFiZjA4Zjc0NDlkMWM4MzQ2ZTRiNDYzZSIsCiAgICAgICAgIm5hbWUiOiAiZXZhbHMvZXZhbHMuanNvbiIsCiAgICAgICAgImFsZ29yaXRobSI6ICJzaGEyNTYiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogImUzM2UzNDQwN2U2NThiNjgxOGIyNGVlMzM0MDdhYzBmM2ZhNDE1ZTFhNmVlZDUyNWM2MWYyM2I2MDk2NmQ3MDMiLAogICAgICAgICJuYW1lIjogInNraWxsLWNhcmQubWQiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IgogICAgICB9CiAgICBdLAogICAgInNlcmlhbGl6YXRpb24iOiB7CiAgICAgICJoYXNoX3R5cGUiOiAic2hhMjU2IiwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdGh1YiIsCiAgICAgICAgIi5naXRhdHRyaWJ1dGVzIiwKICAgICAgICAiLmdpdCIsCiAgICAgICAgIi5naXRpZ25vcmUiCiAgICAgIF0sCiAgICAgICJtZXRob2QiOiAiZmlsZXMiLAogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZQogICAgfQogIH0KfQ==","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGQCMEvXNF3AaHDYUPtx+5WiMjg4bM7xCCyZIyGxK2wc0eBukQlkqSMzNxqGuEEpMdOHIQIwHwDtKvBMckgRr5CEcQtEPHfCxsNPg8shokhSh/U4EqWGn60/eoFDJmv0NBTTftoh","keyid":""}]}} \ No newline at end of file +{"mediaType":"application/vnd.dev.sigstore.bundle.v0.3+json","verificationMaterial":{"x509CertificateChain":{"certificates":[{"rawBytes":"MIICgzCCAgmgAwIBAgIUKIyS7SxNteQIiWzK1dWj85E6520wCgYIKoZIzj0EAwMwVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwHhcNMjYwNDAxMDAwMDAwWhcNMjgwNDIyMTUzMzA5WjBUMQswCQYDVQQGEwJVUzEbMBkGA1UECgwSTlZJRElBIENvcnBvcmF0aW9uMSgwJgYDVQQDDB9OVklESUEgQWdlbnQgU2tpbGxzIFNpZ25pbmcgMDAxMHYwEAYHKoZIzj0CAQYFK4EEACIDYgAEYoRM9bQl/dGlwSRNi6bTpIJUXH8Nv9GciP6LSflJYYMLCc296kpyuTSsk5ddbAWiDcFX3C/ydX3jwc+qCLYP6uHy9XphyLjOQ27Yb2J6rBLVtRBS1mgGco/Gr7fL6ODco4GaMIGXMB0GA1UdDgQWBBRQ/5ZW3nJ6lmo9SVk7I15o7UGmpTAfBgNVHSMEGDAWgBRPGpILxMBBleJSsBGjrMKsby1CgjAMBgNVHRMBAf8EAjAAMA4GA1UdDwEB/wQEAwIHgDA3BggrBgEFBQcBAQQrMCkwJwYIKwYBBQUHMAGGG2h0dHA6Ly9vY3NwLm5kaXMubnZpZGlhLmNvbTAKBggqhkjOPQQDAwNoADBlAjAUygu/GiOCIXrgGr4SmLgeEVDcEitfFUv7ALbvLVGVyMysB3mxmO/uInZfXzWcJZsCMQDxuoxj4ZmO30jhkPIcCxGFCOvnUsnfU3TfGcouYm4M6iRpbKvtVnHPiy4bi6pcKf0="},{"rawBytes":"MIICiDCCAg6gAwIBAgIUZsIuSv9NkpJCNqtYEfCouVv5BzowCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowVTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjEpMCcGA1UEAwwgTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBJQ0EgMDEwdjAQBgcqhkjOPQIBBgUrgQQAIgNiAASI72cR3ctKGg4VWnB3bNja6g1Z2PnOmFEopkPof+QeIcPk9rT+g9MjJnq51EQXL93a7C2GJ9J985G4o2V85VD7wJ1RaXhluHW2rf3y8bQGeAYaKMr5s/hUgn+M3/9WlWejgaAwgZ0wHQYDVR0OBBYEFE8akgvEwEGV4lKwEaOswqxvLUKCMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMBIGA1UdEwEB/wQIMAYBAf8CAQAwDgYDVR0PAQH/BAQDAgEGMDcGCCsGAQUFBwEBBCswKTAnBggrBgEFBQcwAYYbaHR0cDovL29jc3AubmRpcy5udmlkaWEuY29tMAoGCCqGSM49BAMDA2gAMGUCMQCeIMMfAbyzPDacw2MxG+Yt1cikrJX/DVxiGfXuHmkkXn6VgSzE79+lkqDErpVO2gYCMCNEColOyvUvkzZGUEI1hQ3PfMgi3FIo9tHoBKMw4/wGBLFpu/0ubtmbBXM6/UMOEw=="},{"rawBytes":"MIICRTCCAcygAwIBAgIUeJdY3rV86EdvFmG7L8LJBsyQFYkwCgYIKoZIzj0EAwMwUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTAgFw0yNjA0MDEwMDAwMDBaGA85OTk5MTIzMTIzNTk1OVowUTELMAkGA1UEBhMCVVMxGzAZBgNVBAoMEk5WSURJQSBDb3Jwb3JhdGlvbjElMCMGA1UEAwwcTlZJRElBIEFnZW50IENhcGFiaWxpdGllcyBDQTB2MBAGByqGSM49AgEGBSuBBAAiA2IABAYpiXCDjJ9NT2eSDhyHJVSw1Tbze18cGG2F/578oWvHxg23eQAhNRYdq88i1iOshZSO6C29doKui5Xpmo/7Ctw9Sx4PP2RzOmIuOLCuTdNtKcTRwi4GEsd5BAFvWj42M6NjMGEwHQYDVR0OBBYEFItnoAjjfuCEUvzyvWyI2vOGvwPjMB8GA1UdIwQYMBaAFItnoAjjfuCEUvzyvWyI2vOGvwPjMA8GA1UdEwEB/wQFMAMBAf8wDgYDVR0PAQH/BAQDAgEGMAoGCCqGSM49BAMDA2cAMGQCMCwtAjWLaNwgGWNCgdyNoTyvNhqWRECRJV2r3+7w8g0PL6NHLOsbkgE09BH95h8XlgIwTaQmbbUh2ChAJ5TA1wRiVDnCcvbzHlZl2jM2FcwQQZlk19LOAbyGMRixbu2Ww/rj"}]},"tlogEntries":[]},"dsseEnvelope":{"payload":"ewogICJfdHlwZSI6ICJodHRwczovL2luLXRvdG8uaW8vU3RhdGVtZW50L3YxIiwKICAic3ViamVjdCI6IFsKICAgIHsKICAgICAgIm5hbWUiOiAiY3VvcHQtc2tpbGwtZXZvbHV0aW9uIiwKICAgICAgImRpZ2VzdCI6IHsKICAgICAgICAic2hhMjU2IjogIjIwZjgyM2FmZTA0OTBjMWQ3NzlmNDM4MjU5MjM3MGE4OGY0NzI0YWEzYjY2OGEwMmFhZmIwODg3NTA4NjU2OGYiCiAgICAgIH0KICAgIH0KICBdLAogICJwcmVkaWNhdGVUeXBlIjogImh0dHBzOi8vbW9kZWxfc2lnbmluZy9zaWduYXR1cmUvdjEuMCIsCiAgInByZWRpY2F0ZSI6IHsKICAgICJzZXJpYWxpemF0aW9uIjogewogICAgICAibWV0aG9kIjogImZpbGVzIiwKICAgICAgImhhc2hfdHlwZSI6ICJzaGEyNTYiLAogICAgICAiYWxsb3dfc3ltbGlua3MiOiBmYWxzZSwKICAgICAgImlnbm9yZV9wYXRocyI6IFsKICAgICAgICAiLmdpdGh1YiIsCiAgICAgICAgIi5naXQiLAogICAgICAgICIuZ2l0aWdub3JlIiwKICAgICAgICAiLmdpdGF0dHJpYnV0ZXMiCiAgICAgIF0KICAgIH0sCiAgICAicmVzb3VyY2VzIjogWwogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICIwY2Y3Y2RiMThhODg5MWNhMTAyYTM1NDRmYzQzMzJkNmRjNTBjZTg0YTgzOTU5YmVkYjA4ZDVjODk5NTEzMjZmIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiQkVOQ0hNQVJLLm1kIgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICI0MGQ5MjM3ZWI2ZmRiMTczMWVkOWJlYTA5ZmU2OWJlNzQxMWVmMTA0ZTkyZTY4MmU2OGZiMWJkMjQyNjNiOTE0IiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAiU0tJTEwubWQiCiAgICAgIH0sCiAgICAgIHsKICAgICAgICAiZGlnZXN0IjogIjZjOGZlOGE0MTRhYjg3Zjk0MGQ1ZWYyYWI1MjFiOWI3Nzg4NzBhNzBhYmYwOGY3NDQ5ZDFjODM0NmU0YjQ2M2UiLAogICAgICAgICJhbGdvcml0aG0iOiAic2hhMjU2IiwKICAgICAgICAibmFtZSI6ICJldmFscy9ldmFscy5qc29uIgogICAgICB9LAogICAgICB7CiAgICAgICAgImRpZ2VzdCI6ICJhODIzMWY2NzcxZjczZGI3OWJjYmE5MDQ5NmM1ODdkNzZlNmVjMzIzNGQyN2I2MDdmZWUyZjJlNWFlOWMxN2ZiIiwKICAgICAgICAiYWxnb3JpdGhtIjogInNoYTI1NiIsCiAgICAgICAgIm5hbWUiOiAic2tpbGwtY2FyZC5tZCIKICAgICAgfQogICAgXQogIH0KfQ==","payloadType":"application/vnd.in-toto+json","signatures":[{"sig":"MGYCMQDnNMx93v5S3KYsEVjzmQRsAwHZfrMb1kNsfu9MWtRnQAQfCKgugmknv9vJqMTir94CMQDksmtiajiivXO9rPNrryYAZcQwvhvtA8TFiA3xpbGHm91Iiewan8/AmivrC9XVKyo=","keyid":""}]}} \ No newline at end of file