Skip to content

Add ogc.osc.prov-processing-step: PROV profile for a single geospatial processing step - #18

Open
nsnarayanam wants to merge 19 commits into
ogcincubator:masterfrom
nsnarayanam:clean-pr-1789123394.9575
Open

nsnarayanam wants to merge 19 commits into
ogcincubator:masterfrom
nsnarayanam:clean-pr-1789123394.9575

Conversation

@nsnarayanam

Copy link
Copy Markdown

D104 (Aganitha), Activities 1 and 2. Supersedes #17. Five files: four new sources under _sources/prov-processing-step/ and one import added to bblocks-config.yaml for bblock-prov-schema, which the block profiles.

All four points from your review:

Fork master has a complete build, reviewable in the viewer: https://nsnarayanam.github.io/bblocks-openscience/bblock/ogc.osc.prov-processing-step

The About link on the fork now points there rather than upstream.

Branch produced by create-clean-pr.sh.

Build output links below.

The block profiles ogc.ogc-utils.prov, constrained to a single geospatial processing step, adding a required processType URI linking the step to a registered process type. Nine examples, profiled against a drought intelligence workflow we run over Telangana, forming one connected provenance graph with ten resolving links.

Build output for review:

Validation report, 9/9 passing
https://nsnarayanam.github.io/bblocks-openscience/build/tests/osc/prov-processing-step/_report.json

The two ML examples, the ones I would most like a view on. Training generates a fitted model entity and inference consumes that same entity, so the chain closes rather than terminating:
https://nsnarayanam.github.io/bblocks-openscience/build/tests/osc/prov-processing-step/example_7_1.ttl
https://nsnarayanam.github.io/bblocks-openscience/build/tests/osc/prov-processing-step/example_8_1.ttl

Annotated schema and JSON-LD context:
https://nsnarayanam.github.io/bblocks-openscience/build/annotated/osc/prov-processing-step/schema.yaml
https://nsnarayanam.github.io/bblocks-openscience/build/annotated/osc/prov-processing-step/context.jsonld

One finding worth reporting. Syncing with master brought in the cross-domain-model PROV shapes, which my examples then failed: the qualifiedUsage blank nodes carried prov:entity and prov:hadRole but did not declare themselves prov:Usage, so sh:class rejected them. Correct PROV in substance, under-specified in the graph. Fixed by typing them explicitly and
all nine pass again. An upstream shape catching something my own validation would not have is the pattern working as intended.

On your question: no, I had not been using the OGC LLM skills. I have now worked through the authoring set, and three things came out of it.

metadata.md: I had a redundant dependsOn where the schema already references the block via bblocks://, no isProfileOf despite this being a profile, and two fields (maturity, scope) that are not in the schema at all. Corrected.

examples.md: my examples used a description field where the schema defines content, so nine paragraphs of prose were being silently dropped from the generated docs. Corrected, and they now appear.

contributing.md: I had been suppressing build/ by hand rather than letting the script do it.

So the skills caught real defects, not just process ones, which supports your point to Alejandro. One thing that might be worth a line in contributing.md: create-clean-pr.sh strips build/ changes from your commits, but if your fork's
build/ has drifted from upstream — as mine had, after resolving merge conflicts by hand — those differences survive into the clean branch. I had to realign build/ against fork-parent/master explicitly.

Also noting the openscience ontology typo fix, processes-ontology to processing-ontology, is not in this branch. I will raise it separately since it is unrelated to this block.

@rob-metalinkage

…es via qualifiedUsage — examples now validate 2/2
# Conflicts:
#	build/annotated/_cache/2ca316da1ef467fe1f43774f3888bce31290b36670e724d47ed42b7b0b32f1b9
#	build/annotated/_cache/769bb14c8b0f7a9d2ed48bab98bb42793e39c648c553b13d694c2d16a570cbc2
#	build/annotated/_cache/9b9f081461b2b99026848a00d04b973d8747f35e6d5bc47cf89049ed3423f61e
#	build/annotated/_cache/b1f672ac1f3ffb4ca4de45f5eaa0e99a40ba897ab98c6b70411e940eaaa6c257
#	build/annotated/osc/api-profiles/processes/sample-implementation/schemas/buffer-geometry/inputDescription/resolvedProperties.json
#	build/annotated/osc/api-profiles/processes/sample-implementation/schemas/buffer-geometry/processDescription/resolvedProperties.json
#	build/annotated/osc/api-profiles/processes/sample-implementation/schemas/inputDescriptions/resolvedProperties.json
#	build/annotated/osc/geodcat-stac-earthcode/common/resolvedProperties.json
#	build/annotated/osc/geodcat-stac-earthcode/experiments/resolvedProperties.json
#	build/annotated/osc/geodcat-stac-earthcode/products/resolvedProperties.json
#	build/annotated/osc/geodcat-stac-earthcode/workflows/resolvedProperties.json
#	build/bblocks.jsonld
#	build/bblocks.ttl
#	build/generateddocs/json-full/osc/api-profiles/processes/ipt/api/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/ipt/execute/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/ipt/results/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/ospd/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/sample-implementation/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/sample-implementation/schemas/buffer-geometry/inputDescription/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/sample-implementation/schemas/buffer-geometry/inputSchema/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/sample-implementation/schemas/buffer-geometry/outputDescription/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/sample-implementation/schemas/buffer-geometry/outputSchema/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/sample-implementation/schemas/buffer-geometry/processDescription/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/sample-implementation/schemas/inputDescriptions/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/sample-implementation/schemas/outputDescriptions/index.json
#	build/generateddocs/json-full/osc/api-profiles/processes/workflow/index.json
#	build/generateddocs/json-full/osc/application-package/index.json
#	build/generateddocs/json-full/osc/geodcat-stac-earthcode/common/index.json
#	build/generateddocs/json-full/osc/geodcat-stac-earthcode/experiments/index.json
#	build/generateddocs/json-full/osc/geodcat-stac-earthcode/products/index.json
#	build/generateddocs/json-full/osc/geodcat-stac-earthcode/workflows/index.json
#	build/generateddocs/json-full/osc/ontology/cwlprov/index.json
#	build/generateddocs/json-full/osc/ontology/openscience/index.json
#	build/generateddocs/markdown/osc/geodcat-stac-earthcode/common/index.md
#	build/generateddocs/markdown/osc/geodcat-stac-earthcode/experiments/index.md
#	build/generateddocs/markdown/osc/geodcat-stac-earthcode/products/index.md
#	build/generateddocs/markdown/osc/geodcat-stac-earthcode/workflows/index.md
#	build/register-summary.md
#	build/register.json
#	build/tests/osc/api-profiles/processes/ipt/api/_report.json
#	build/tests/osc/api-profiles/processes/ipt/execute/_report.json
#	build/tests/osc/api-profiles/processes/ipt/results/_report.json
#	build/tests/osc/api-profiles/processes/ospd/_report.json
#	build/tests/osc/api-profiles/processes/sample-implementation/_report.json
#	build/tests/osc/api-profiles/processes/workflow/_report.json
#	build/tests/osc/application-package/_report.json
#	build/tests/osc/application-package/example_1_1.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/common/_report.json
#	build/tests/osc/geodcat-stac-earthcode/common/example_1_1.ttl
#	build/tests/osc/geodcat-stac-earthcode/common/example_1_1.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/experiments/_report.json
#	build/tests/osc/geodcat-stac-earthcode/experiments/example_1_1.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/experiments/example_2_1.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/experiments/example_3_1.ttl
#	build/tests/osc/geodcat-stac-earthcode/experiments/example_3_1.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/experiments/example_4_1.ttl
#	build/tests/osc/geodcat-stac-earthcode/experiments/example_4_1.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/products/_report.json
#	build/tests/osc/geodcat-stac-earthcode/products/empty-themes-fail.ttl
#	build/tests/osc/geodcat-stac-earthcode/products/empty-themes-fail.validation_failed.txt
#	build/tests/osc/geodcat-stac-earthcode/products/example_1_1.ttl
#	build/tests/osc/geodcat-stac-earthcode/products/example_1_1.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/products/example_2_1.ttl
#	build/tests/osc/geodcat-stac-earthcode/products/example_2_1.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/products/invalid-cf-parameters-fail.ttl
#	build/tests/osc/geodcat-stac-earthcode/products/invalid-cf-parameters-fail.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/products/invalid-osc-status-fail.ttl
#	build/tests/osc/geodcat-stac-earthcode/products/invalid-osc-status-fail.validation_failed.txt
#	build/tests/osc/geodcat-stac-earthcode/products/missing-project-fail.ttl
#	build/tests/osc/geodcat-stac-earthcode/products/missing-project-fail.validation_failed.txt
#	build/tests/osc/geodcat-stac-earthcode/products/missing-type-fail.ttl
#	build/tests/osc/geodcat-stac-earthcode/products/missing-type-fail.validation_failed.txt
#	build/tests/osc/geodcat-stac-earthcode/products/valid-cf-parameters.ttl
#	build/tests/osc/geodcat-stac-earthcode/products/valid-cf-parameters.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/workflows/_report.json
#	build/tests/osc/geodcat-stac-earthcode/workflows/example_1_1.ttl
#	build/tests/osc/geodcat-stac-earthcode/workflows/example_1_1.validation_passed.txt
#	build/tests/osc/geodcat-stac-earthcode/workflows/example_2_1.ttl
#	build/tests/osc/geodcat-stac-earthcode/workflows/example_2_1.validation_passed.txt
#	build/tests/osc/ontology/cwlprov/_report.json
#	build/tests/osc/ontology/openscience/_report.json
#	build/tests/report.html
#	build/tests/report.json

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant