From f6b3678687aa2ab95402811b7e85838eb4369fd5 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Thu, 6 Aug 2026 12:09:41 +0200 Subject: [PATCH 1/5] [maven-release-plugin] prepare for next development iteration --- pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index d858c74cd..530bd9fe4 100644 --- a/pom.xml +++ b/pom.xml @@ -3,7 +3,7 @@ com.atomgraph linkeddatahub - 5.7.1 + 5.7.2-SNAPSHOT ${packaging.type} AtomGraph LinkedDataHub @@ -46,7 +46,7 @@ https://github.com/AtomGraph/LinkedDataHub scm:git:git://github.com/AtomGraph/LinkedDataHub.git scm:git:git@github.com:AtomGraph/LinkedDataHub.git - linkeddatahub-5.7.1 + linkeddatahub-5.5.4 From 026a5175381ee12d0ae29a17366a7beb475cc4be Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Fri, 7 Aug 2026 18:45:26 +0200 Subject: [PATCH 2/5] Serve raw ontology graphs without RDFS inference (#337) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * Serve raw ontology graphs without RDFS inference Resolve the application ontology's owl:imports closure natively via ontapi (OntModelFactory.createModel over a ScopedGraphRepository view) instead of manually flattening the closure and materializing an RDFS-inferred model. No inference is applied anymore — every consumer (constructor/constraint inheritance, client-side (rdfs:subClassOf)* queries) traverses hierarchies explicitly. rdfs:Class terms are promoted to owl:Class in a separate union member so no document graph is polluted; closure union graphs are cached in a bounded map on Application, keyed by ontology URI. The shared repository now only ever holds raw per-document graphs, which ProxyRequestFilter serves directly for closure documents — asserted triples only, identical to a direct document GET. The DESCRIBE fallback over the in-memory closure (terms minted in external namespaces) is inference-free as well. This fixes inferred rdf:type rdfs:Resource leaking into proxied namespace documents, where the extra type produced multi-token @typeof and silently degraded View blocks to a generic property list. Co-Authored-By: Claude Fable 5 * Fix ScopedGraphRepository.contains() dropping resolvable imports ontapi consults GraphRepository.contains() before get() when resolving an ontology's imports closure. PrefixGraphRepository.contains() reports cache state (loaded graphs), not resolvability, so every import that resolves through a bundled location mapping (dh, sp, spin, foaf, sioc, sd), SPARQL-first loading or HTTP was answered with false on first resolution — and ontapi silently substituted an empty ontology graph for it (its ignoreUnresolvedImports fallback). The closure kept its shape but lost the content of every such import: SPIN constraints vanished, so validation enforced nothing (422 tests wrote through, eventually applying invalid dataspace settings and cascading into NPEs), and vocabulary term lookups came up empty. This is what failed the HTTP test suite in CI. contains() now attempts resolution through the backing repository (which loads and caches the graph) after the cache checks, reporting absent only for genuinely unresolvable ids. Adds a production-shaped regression test: ns# ontology importing the SPARQL-seeded ldh# vocabulary with its transitive imports resolved through the real bundled location mappings. Co-Authored-By: Claude Fable 5 * Alias only repository-held closure ids under their document URIs ontapi keys imports under their declared ontology IRIs, which need not be repository entries: a content-addressed upload is cached under its uploads/ URI while declaring a foreign ontology IRI. The doc-URI aliasing loop called repository.get() on such declared IRIs, which fell through to an HTTP dereference of the foreign IRI (e.g. https://example.org/test) during ontology load — failing the load and the ontology-import-upload-no-deadlock HTTP test. Guard the loop with isCached() so only graphs the shared repository actually holds get aliased. Adds a mismatched-IRI import case to the closure regression test. Co-Authored-By: Claude Fable 5 * Fix GET-proxied-ontology-ns.sh assertions for closure DESCRIBE semantics The admin ontologies/namespace/ document stores the ontology but the closure keys it under the ontology URI, so a proxied GET of the document URI is answered by the closure DESCRIBE fallback — the document's own #-fragment term descriptions — not the raw graph branch. Assert on a class minted in the document's hash namespace (mirroring the original #related_View regression) instead of the made-up-namespace classes, which only appear under their own namespace document URI. Co-Authored-By: Claude Fable 5 * Remove Linked Data proxy DESCRIBE-over-closure fallback The proxy is document-keyed transport; term lookups over the ontology closure belong on /ns (SPARQL), not on a DESCRIBE synthesized from the proxy target URI. Drop the fallback and its ParameterizedSparqlString/ QueryExecution imports; the isCached closure-cache branch (raw per-doc graphs, no inference) still serves closure documents. Realign tests: delete GET-proxied-ontology-ns.sh (it only exercised the removed DESCRIBE path and dereferenced ontology terms via the admin doc URI, which was never a supported path). Add GET-ns-no-query.sh (raw /ns ontology graph, no rdfs:Resource leak) and GET-proxied-mapped-vocab.sh (dct:title, foaf:Person, skos:Concept served from the static prefix mapping; skos also covers fragment-strip + xml:base resolution). Co-Authored-By: Claude Opus 4.8 (1M context) * Make the Linked Data proxy dumb: drop ontology-closure serving Remove the getOntology()/isCached branch that answered ?uri= requests from the app ontology owl:imports closure cache, plus the now-unused ontology injection, getOntology(), and PrefixGraphRepository/Application/EndUserApplication/ OntModel imports. The proxy is now dumb transport: bundled-vocab file cache (isMapped) + SSRF-checked external fetch. Ontology terms are served by /ns. Co-Authored-By: Claude Opus 4.8 (1M context) * Fix GET-proxied-mapped-vocab SIGPIPE on large vocab responses The assertions piped the whole vocabulary graph through `echo | grep -q`. `grep -q` exits on first match and closes the pipe while `echo` is still writing; with `set -o pipefail` the SIGPIPE'd `echo` (write error: broken pipe, exit 141) fails the pipeline whenever the response exceeds the ~64 KiB pipe buffer — so a *successful* match killed the test. Only this test trips it, being the only one that returns entire vocabulary documents (40-113 KiB). Read from a here-string instead (temp file, no pipe to break), and match the full language-tagged label literal the bundled documents actually carry. Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Fable 5 --- CHANGELOG.md | 11 ++ http-tests/proxy/GET-proxied-mapped-vocab.sh | 65 +++++++++ http-tests/proxy/GET-proxied-ontology-ns.sh | 66 --------- .../sparql-protocol/query/GET-ns-no-query.sh | 42 ++++++ .../atomgraph/linkeddatahub/Application.java | 20 ++- .../linkeddatahub/resource/Namespace.java | 6 +- .../resource/admin/ClearOntology.java | 6 +- .../server/filter/request/OntologyFilter.java | 89 ++++++------ .../filter/request/ProxyRequestFilter.java | 37 +---- .../server/io/ValidatingModelProvider.java | 3 +- .../server/util/ScopedGraphRepository.java | 133 ++++++++++++++++++ .../request/OntologyClosureCIReproTest.java | 131 +++++++++++++++++ .../OntologyImportsCharacterizationTest.java | 66 ++++++--- .../util/SPINConstraintValidationTest.java | 7 +- 14 files changed, 503 insertions(+), 179 deletions(-) create mode 100755 http-tests/proxy/GET-proxied-mapped-vocab.sh delete mode 100755 http-tests/proxy/GET-proxied-ontology-ns.sh create mode 100755 http-tests/sparql-protocol/query/GET-ns-no-query.sh create mode 100644 src/main/java/com/atomgraph/linkeddatahub/server/util/ScopedGraphRepository.java create mode 100644 src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyClosureCIReproTest.java diff --git a/CHANGELOG.md b/CHANGELOG.md index c5ed2dae2..109d3ac07 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,14 @@ +## [Unreleased] +### Changed +- Application ontologies resolved as a native ontapi `owl:imports` union graph (cached per ontology URI) instead of a manually flattened, RDFS-materialized model — no RDFS inference +- `Namespace` no-query GET serves the raw ontology graph from the shared repository instead of rebuilding a repository per request + +### Fixed +- Raw ontology graphs no longer leak inferred `rdf:type rdfs:Resource`, which produced multi-token `@typeof` that broke View block rendering + +### Removed +- The Linked Data proxy no longer serves ontology terms; it is now dumb transport (bundled-vocab file cache + SSRF-checked external fetch), with ontology terms served by `/ns` + ## [5.7.1] - 2026-08-06 ### Changed - RDFa editor: annotation overlay rebuilt on demand (`rdfa-editor/overlay.xsl`) diff --git a/http-tests/proxy/GET-proxied-mapped-vocab.sh b/http-tests/proxy/GET-proxied-mapped-vocab.sh new file mode 100755 index 000000000..f2250e07a --- /dev/null +++ b/http-tests/proxy/GET-proxied-mapped-vocab.sh @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" + +# add agent to the readers group to be able to read documents + +add-agent-to-group.sh \ + -f "$OWNER_CERT_FILE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/readers/" + +# well-known vocab terms that are statically prefix-mapped to bundled documents +# (src/main/resources/prefix-mapping.ttl), so the proxy serves them straight from +# that cache (isMapped branch) instead of dereferencing the network. +# +# The whole vocabulary graph is returned (tens of KiB), so assertions read from a +# here-string rather than `echo "$response" | grep -q`: `grep -q` closes the pipe on +# first match, and with `set -o pipefail` the SIGPIPE'd `echo` (write error: broken +# pipe) fails the whole pipeline whenever the response exceeds the ~64 KiB pipe buffer. +# Labels are language-tagged in the bundled documents, so the expected literal is +# matched in full including its tag. + +# dct:title - slash-based namespace (http://purl.org/dc/terms/); the proxy request +# URI equals the term URI itself + +dct_response=$(curl -k -f -s \ + -G \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/n-triples" \ + --data-urlencode "uri=http://purl.org/dc/terms/title" \ + "$END_USER_BASE_URL") + +grep -qF ' "Title"@en-US' <<< "$dct_response" + +# foaf:Person - also slash-based (http://xmlns.com/foaf/0.1/); label is a plain literal + +foaf_response=$(curl -k -f -s \ + -G \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/n-triples" \ + --data-urlencode "uri=http://xmlns.com/foaf/0.1/Person" \ + "$END_USER_BASE_URL") + +grep -qF ' "Person"' <<< "$foaf_response" + +# skos:Concept - hash-based namespace (http://www.w3.org/2004/02/skos/core#); the +# request carries a #fragment that ProxyRequestFilter strips before matching the +# mapped prefix, and the bundled document declares terms as relative (#Concept) +# under its own xml:base, so this also confirms that base resolves back to the +# full hash URI rather than leaking a bare fragment or the classpath location + +skos_response=$(curl -k -f -s \ + -G \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/n-triples" \ + --data-urlencode "uri=http://www.w3.org/2004/02/skos/core#Concept" \ + "$END_USER_BASE_URL") + +grep -qF ' "Concept"@en' <<< "$skos_response" diff --git a/http-tests/proxy/GET-proxied-ontology-ns.sh b/http-tests/proxy/GET-proxied-ontology-ns.sh deleted file mode 100755 index c97f48505..000000000 --- a/http-tests/proxy/GET-proxied-ontology-ns.sh +++ /dev/null @@ -1,66 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" -purge_cache "$FRONTEND_VARNISH_SERVICE" - -# add agent to the readers group to be able to read documents - -add-agent-to-group.sh \ - -f "$OWNER_CERT_FILE" \ - -p "$OWNER_CERT_PWD" \ - --agent "$AGENT_URI" \ - "${ADMIN_BASE_URL}acl/groups/readers/" - -# use a made-up hash-based namespace: not mapped as a static file, not a registered app -namespace_uri="http://made-up-test-ns.example/ns" -class1="${namespace_uri}#ClassOne" -class2="${namespace_uri}#ClassTwo" -ontology_doc="${ADMIN_BASE_URL}ontologies/namespace/" -namespace="${END_USER_BASE_URL}ns#" - -# add two classes with URIs in the made-up namespace to the app's ontology - -add-class.sh \ - -f "$OWNER_CERT_FILE" \ - -p "$OWNER_CERT_PWD" \ - -b "$ADMIN_BASE_URL" \ - --uri "$class1" \ - --label "Class One" \ - "$ontology_doc" - -add-class.sh \ - -f "$OWNER_CERT_FILE" \ - -p "$OWNER_CERT_PWD" \ - -b "$ADMIN_BASE_URL" \ - --uri "$class2" \ - --label "Class Two" \ - "$ontology_doc" - -# clear the in-memory ontology so the new classes are present on next request - -clear-ontology.sh \ - -f "$OWNER_CERT_FILE" \ - -p "$OWNER_CERT_PWD" \ - -b "$ADMIN_BASE_URL" \ - --ontology "$namespace" - -# request the namespace document URI (without fragment) via ?uri= proxy. -# the namespace document is not DataManager-mapped and not a registered app, -# so ProxyRequestFilter falls through to the OntModel DESCRIBE path, which -# returns descriptions of all #-fragment terms in that namespace. - -response=$(curl -k -f -s \ - -G \ - -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ - -H "Accept: application/n-triples" \ - --data-urlencode "uri=${namespace_uri}" \ - "$END_USER_BASE_URL") - -# verify both class descriptions are present in the response - -echo "$response" | grep -q "$class1" -echo "$response" | grep -q "$class2" diff --git a/http-tests/sparql-protocol/query/GET-ns-no-query.sh b/http-tests/sparql-protocol/query/GET-ns-no-query.sh new file mode 100755 index 000000000..c2e99f759 --- /dev/null +++ b/http-tests/sparql-protocol/query/GET-ns-no-query.sh @@ -0,0 +1,42 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" + +# add a class to the app's namespace ontology + +namespace_doc="${END_USER_BASE_URL}ns" +namespace="${namespace_doc}#" +ontology_doc="${ADMIN_BASE_URL}ontologies/namespace/" +class="${namespace}ClassThree" + +add-class.sh \ + -f "$OWNER_CERT_FILE" \ + -p "$OWNER_CERT_PWD" \ + -b "$ADMIN_BASE_URL" \ + --uri "$class" \ + --label "Class Three" \ + "$ontology_doc" + +# clear ontology from memory so the new class is loaded on next request + +clear-ontology.sh \ + -f "$OWNER_CERT_FILE" \ + -p "$OWNER_CERT_PWD" \ + -b "$ADMIN_BASE_URL" \ + --ontology "$namespace" + +# GET with no ?query= should return the raw namespace ontology graph (asserted +# triples only, no RDFS materialization) rather than run a SPARQL query + +response=$(curl -k -f -s \ + -E "$OWNER_CERT_FILE":"$OWNER_CERT_PWD" \ + -H "Accept: application/n-triples" \ + "$namespace_doc") + +echo "$response" | grep -q "$class" +! echo "$response" | grep -q "http://www.w3.org/2000/01/rdf-schema#Resource" diff --git a/src/main/java/com/atomgraph/linkeddatahub/Application.java b/src/main/java/com/atomgraph/linkeddatahub/Application.java index 9147b37fc..b83be580b 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/Application.java +++ b/src/main/java/com/atomgraph/linkeddatahub/Application.java @@ -161,6 +161,7 @@ import javax.net.ssl.TrustManagerFactory; import jakarta.servlet.ServletContext; import javax.xml.transform.Source; +import org.apache.jena.ontapi.UnionGraph; import org.apache.jena.ontapi.model.OntModel; import org.apache.jena.query.Dataset; import org.apache.jena.query.Query; @@ -199,6 +200,7 @@ import javax.xml.parsers.ParserConfigurationException; import javax.xml.transform.TransformerException; import javax.xml.transform.stream.StreamSource; +import net.jodah.expiringmap.ExpirationPolicy; import net.jodah.expiringmap.ExpiringMap; import net.sf.saxon.om.TreeInfo; import net.sf.saxon.s9api.Processor; @@ -288,6 +290,7 @@ public class Application extends ResourceConfig private final KeyStore keyStore, trustStore; private final URI secretaryWebIDURI; private final List supportedLanguages; + private final ExpiringMap ontologyGraphs = ExpiringMap.builder().maxSize(1000).expirationPolicy(ExpirationPolicy.ACCESSED).expiration(1, TimeUnit.HOURS).build(); // assembled ontology imports-closure union graphs, keyed by ontology URI; evicted entries are transparently rebuilt by OntologyFilter on the next cache miss private final ExpiringMap webIDmodelCache = ExpiringMap.builder().expiration(Long.parseLong(System.getProperty("com.atomgraph.linkeddatahub.webIDCacheExpiration", "86400")), TimeUnit.SECONDS).build(); // TTL (seconds) configurable via WEBID_CACHE_EXPIRATION; a lower value bounds how long a revoked WebID stays cached private final ExpiringMap oidcModelCache = ExpiringMap.builder().variableExpiration().build(); private final ExpiringMap jwksCache = ExpiringMap.builder().expiration(Long.parseLong(System.getProperty("com.atomgraph.linkeddatahub.jwksCacheExpiration", "86400")), TimeUnit.SECONDS).build(); // Cache JWKS responses; TTL (seconds) configurable via JWKS_CACHE_EXPIRATION @@ -1804,10 +1807,23 @@ public OntologyRepository createRepository(EndUserApplication app) return appRepository; } - + + /** + * Returns the cache of assembled ontology imports-closure union graphs, keyed by ontology URI + * (origin-scoped per dataspace, so a single map cannot collide across applications). + * The union graph is ontapi's view over the raw per-document graphs cached in the (per-app or + * system) repository; it is not a document graph itself and is never served on the wire. + * + * @return ontology URI to union graph map + */ + public Map getOntologyGraphs() + { + return ontologyGraphs; + } + /** * Returns a registry of readable and writeable media types. - * + * * @return registry object */ public MediaTypes getMediaTypes() diff --git a/src/main/java/com/atomgraph/linkeddatahub/resource/Namespace.java b/src/main/java/com/atomgraph/linkeddatahub/resource/Namespace.java index 5ce68929e..50cd157ca 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/resource/Namespace.java +++ b/src/main/java/com/atomgraph/linkeddatahub/resource/Namespace.java @@ -140,9 +140,9 @@ public Response get(@QueryParam(QUERY) Query query, // the application ontology MUST use a URI! This is the URI this ontology endpoint is deployed on by the Dispatcher class String ontologyURI = getApplication().getOntology().getURI(); if (log.isDebugEnabled()) log.debug("Returning raw namespace ontology: {}", ontologyURI); - // not returning the injected in-memory ontology because it has inferences applied to it; - // a fresh, mapping-seeded repository serves the raw SPARQL-loaded ontology - OntologyRepository repository = getSystem().createRepository(getApplication().as(EndUserApplication.class)); + // not returning the injected in-memory ontology because it is the full imports closure (a union view); + // the shared repository serves the standalone raw ontology graph + OntologyRepository repository = getSystem().getRepository(getApplication().as(EndUserApplication.class)); return getResponseBuilder(org.apache.jena.rdf.model.ModelFactory.createModelForGraph(repository.get(ontologyURI))).build(); } else throw new BadRequestException("SPARQL query string not provided"); diff --git a/src/main/java/com/atomgraph/linkeddatahub/resource/admin/ClearOntology.java b/src/main/java/com/atomgraph/linkeddatahub/resource/admin/ClearOntology.java index 9ce40f1a6..190fbd346 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/resource/admin/ClearOntology.java +++ b/src/main/java/com/atomgraph/linkeddatahub/resource/admin/ClearOntology.java @@ -78,12 +78,14 @@ public Response post(@FormParam("uri") String ontologyURI, @HeaderParam("Referer EndUserApplication endUserApp = getApplication().as(AdminApplication.class).getEndUserApplication(); // we're assuming the current app is admin OntologyRepository repository = getSystem().getRepository(endUserApp); - if (repository.isCached(ontologyURI)) + if (repository.isCached(ontologyURI) || getSystem().getOntologyGraphs().containsKey(ontologyURI)) { if (log.isDebugEnabled()) log.debug("Clearing ontology with URI '{}' from memory", ontologyURI); repository.remove(ontologyURI); + getSystem().getOntologyGraphs().remove(ontologyURI); URI ontologyDocURI = UriBuilder.fromUri(ontologyURI).fragment(null).build(); // skip fragment from the ontology URI to get its graph URI + repository.remove(ontologyDocURI.toString()); // the raw graph is also aliased under the fragment-stripped document URI // frontend proxy still uses URL-pattern BAN for direct document GETs (until Stage 3 brings xkey tagging to varnish-frontend). // xkey purge covers proxied SPARQL CONSTRUCT/SELECT responses tagged by their backend (varnish-admin / varnish-end-user). URI frontendProxy = getSystem().getFrontendProxy(); @@ -110,7 +112,7 @@ public Response post(@FormParam("uri") String ontologyURI, @HeaderParam("Referer } // !!! we need to reload the ontology model before returning a response, to make sure the next request already gets the new version !!! - OntologyFilter.loadOntology(repository, ontologyURI); + getSystem().getOntologyGraphs().put(ontologyURI, OntologyFilter.loadOntology(repository, ontologyURI)); } if (referer != null) return Response.seeOther(referer).build(); diff --git a/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilter.java b/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilter.java index e9bd7af71..319cf68e3 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilter.java +++ b/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilter.java @@ -19,14 +19,13 @@ import com.atomgraph.linkeddatahub.apps.model.Application; import com.atomgraph.linkeddatahub.apps.model.EndUserApplication; import com.atomgraph.client.util.jena.PrefixGraphRepository; +import com.atomgraph.linkeddatahub.server.util.ScopedGraphRepository; import com.atomgraph.linkeddatahub.vocabulary.LAPP; import com.atomgraph.server.exception.OntologyException; import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; -import java.util.HashSet; import java.util.Optional; -import java.util.Set; import jakarta.annotation.Priority; import jakarta.inject.Inject; import jakarta.ws.rs.container.ContainerRequestContext; @@ -34,12 +33,13 @@ import jakarta.ws.rs.container.PreMatching; import org.apache.jena.ontapi.OntModelFactory; import org.apache.jena.ontapi.OntSpecification; +import org.apache.jena.ontapi.UnionGraph; import org.apache.jena.ontapi.model.OntModel; import org.apache.jena.rdf.model.Model; import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.vocabulary.OWL; import org.apache.jena.vocabulary.RDF; import org.apache.jena.vocabulary.RDFS; +import org.apache.jena.vocabulary.OWL; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -131,8 +131,9 @@ public OntModel getOntology(Application app) } /** - * Loads the ontology model for the specified ontology URI, building its owl:imports closure with - * RDFS inference and materializing the inferences into the repository cache. + * Returns the ontology model for the specified ontology URI, assembling its owl:imports closure + * on a cache miss. The returned model is a fresh per-request wrapper over the shared closure + * union graph. * * @param app application resource * @param uri ontology URI @@ -146,64 +147,58 @@ public OntModel getOntology(Application app, String uri) final PrefixGraphRepository repository = app.canAs(EndUserApplication.class) ? getSystem().getRepository(app.as(EndUserApplication.class)) : getSystem().getRepository(); - // only build the materialized model if the ontology is not already cached; the double check under the - // repository lock ensures a single thread materializes it (loadOntology is a compound load + inference + - // put, not atomic), so concurrent cold requests don't duplicate the work or race each other's writes - if (!repository.isCached(uri)) + // only assemble the closure if it is not already cached; the double check under the repository + // lock ensures a single thread assembles it (loadOntology is a compound load + union build, not + // atomic), so concurrent cold requests don't duplicate the work or race each other's writes + UnionGraph union = getSystem().getOntologyGraphs().get(uri); + if (union == null) { synchronized (repository) { - if (!repository.isCached(uri)) loadOntology(repository, uri); + union = getSystem().getOntologyGraphs().get(uri); + if (union == null) + { + union = loadOntology(repository, uri); + getSystem().getOntologyGraphs().put(uri, union); + } } } - return OntModelFactory.createModel(repository.get(uri), OntSpecification.OWL2_FULL_MEM); + return OntModelFactory.createModel(union, OntSpecification.OWL2_FULL_MEM); } /** - * Builds and caches the materialized ontology model. Assembles the owl:imports closure into a single - * graph (so ontapi never manages a union-graph hierarchy over the shared repository), applies RDFS - * inference over the flattened closure, and materializes the inferences into the repository cache so - * the rules engine is not invoked on every request. + * Assembles the ontology's owl:imports closure as a union graph. ontapi resolves the closure through + * a scoped repository view: raw per-document graphs are read through (and cached in) the shared + * repository, while ontapi's union-graph bookkeeping stays local to the view — the shared repository + * keeps serving raw document graphs, and duplicate ontology IDs across applications cannot collide. + * No inference is applied: all consumers traverse class/property hierarchies explicitly. * * @param repository graph repository * @param uri ontology URI + * @return closure union graph */ - public static void loadOntology(PrefixGraphRepository repository, String uri) + public static UnionGraph loadOntology(PrefixGraphRepository repository, String uri) { if (log.isDebugEnabled()) log.debug("Started loading ontology with URI '{}'", uri); - Model union = ModelFactory.createDefaultModel(); - Set closure = new HashSet<>(); - loadClosure(repository, uri, union, closure); - OntModel inferred = OntModelFactory.createModel(union.getGraph(), OntSpecification.OWL2_FULL_MEM_RDFS_INF); - OntModel materialized = OntModelFactory.createModel(OntSpecification.OWL2_FULL_MEM); - materialized.add(inferred); - // promote rdfs:Class to owl:Class so OWL2 profiles recognise third-party vocab terms (e.g. sp:Describe in sp.ttl) - inferred.listSubjectsWithProperty(RDF.type, RDFS.Class).forEach(r -> materialized.add(r, RDF.type, OWL.Class)); - repository.put(uri, materialized.getGraph()); - // cache imported graphs under their fragment-stripped document URIs too - closure.stream().filter(closureURI -> !closureURI.equals(uri)).forEach(importURI -> addDocumentModel(repository, importURI)); + ScopedGraphRepository scoped = new ScopedGraphRepository(repository); + OntModel ontology = OntModelFactory.createModel(repository.get(uri), OntSpecification.OWL2_FULL_MEM, scoped); + UnionGraph union = (UnionGraph)ontology.getGraph(); + // promote rdfs:Class to owl:Class so the OWL2 profile recognises third-party vocab terms (e.g. sp:Describe + // in sp.ttl) as named classes. The promotions live in their own union member so no document graph is + // polluted; carrying no owl:Ontology header, the member is ignored by ontapi's union-graph listener + Model promotions = ModelFactory.createDefaultModel(); + ontology.listSubjectsWithProperty(RDF.type, RDFS.Class).forEach(r -> promotions.add(r, RDF.type, OWL.Class)); + if (!promotions.isEmpty()) union.addSubGraph(promotions.getGraph()); + // cache closure graphs under their fragment-stripped document URIs too. ontapi keys imports under + // their declared ontology IRIs, which need not be repository entries (a content-addressed upload is + // cached under its uploads/ URI while declaring a foreign ontology IRI) — only alias ids the shared + // repository actually holds, lest the lookup dereference a foreign IRI over HTTP + scoped.ids().filter(closureURI -> closureURI.startsWith("http://") || closureURI.startsWith("https://")). + filter(repository::isCached). + forEach(closureURI -> addDocumentModel(repository, closureURI)); if (log.isDebugEnabled()) log.debug("Finished loading ontology with URI '{}'", uri); - } - - /** - * Recursively loads the transitive owl:imports closure of an ontology into a single union model, - * fetching each graph via the repository (SPARQL-first / bundled mappings). - * - * @param repository graph repository - * @param uri ontology URI - * @param union accumulator model - * @param seen accumulator of visited URIs (prevents cycles) - */ - public static void loadClosure(PrefixGraphRepository repository, String uri, Model union, Set seen) - { - if (!seen.add(uri)) return; - Model model = ModelFactory.createModelForGraph(repository.get(uri)); - union.add(model); - model.listObjectsOfProperty(OWL.imports).toList().forEach(imp -> - { - if (imp.isURIResource()) loadClosure(repository, imp.asResource().getURI(), union, seen); - }); + return union; } /** diff --git a/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/ProxyRequestFilter.java b/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/ProxyRequestFilter.java index 9166b28ea..da1050384 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/ProxyRequestFilter.java +++ b/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/ProxyRequestFilter.java @@ -22,7 +22,6 @@ import com.atomgraph.core.exception.BadGatewayException; import com.atomgraph.core.util.ModelUtils; import com.atomgraph.linkeddatahub.apps.model.Dataset; -import org.apache.jena.ontapi.model.OntModel; import com.atomgraph.linkeddatahub.client.GraphStoreClient; import com.atomgraph.linkeddatahub.client.filter.auth.IDTokenDelegationFilter; import com.atomgraph.linkeddatahub.client.filter.auth.WebIDDelegationFilter; @@ -38,8 +37,6 @@ import java.util.List; import java.util.Optional; import java.util.Set; -import org.apache.jena.query.ParameterizedSparqlString; -import org.apache.jena.query.QueryExecution; import jakarta.annotation.Priority; import jakarta.inject.Inject; import jakarta.ws.rs.NotAllowedException; @@ -133,7 +130,6 @@ public class ProxyRequestFilter implements ContainerRequestFilter "Age"); @Inject com.atomgraph.linkeddatahub.Application system; - @Inject jakarta.inject.Provider> ontology; @Inject MediaTypes mediaTypes; @Context Request request; @@ -186,27 +182,6 @@ public void filter(ContainerRequestContext requestContext) throws IOException return; } - // serve terms from the app's in-memory namespace ontology (full imports closure) via DESCRIBE. - // covers both slash-based term URIs (e.g. schema:category) and hash-based namespaces - // (e.g. sioc:UserAccount → ac:document-uri strips to sioc:ns, so we also describe all - // ?term where STR(?term) starts with "#") - if (isSafeMethod && getOntology().isPresent()) - { - ParameterizedSparqlString pss = new ParameterizedSparqlString( - "DESCRIBE ?doc ?term WHERE { ?term ?p ?o FILTER(STRSTARTS(STR(?term), CONCAT(STR(?doc), \"#\"))) }"); - pss.setIri("doc", targetURI.toString()); - try (QueryExecution qe = QueryExecution.create(pss.asQuery(), getOntology().get())) - { - Model description = qe.execDescribe(); - if (!description.isEmpty()) - { - if (log.isDebugEnabled()) log.debug("Serving URI from namespace ontology: {}", targetURI); - requestContext.abortWith(getResponse(description, Response.Status.OK)); - return; - } - } - } - boolean isRegisteredApp = getSystem().matchApp(targetURI) != null; if (!isRegisteredApp && !getSystem().isEnableLinkedDataProxy()) throw new NotAllowedException("Linked Data proxy not enabled"); @@ -419,7 +394,7 @@ private Response overlayHeaders(Response response, Response clientResponse, bool /** * Builds a response for the given RDF model with type-appropriate content negotiation. - * Used for locally-served responses (DataManager cache, namespace ontology DESCRIBE) and for + * Used for locally-served responses (DataManager cache) and for * the proxy's Model branch. * * @param model RDF model @@ -481,16 +456,6 @@ public com.atomgraph.linkeddatahub.Application getSystem() return system; } - /** - * Returns the current application's namespace ontology, if available. - * - * @return optional ontology - */ - public Optional getOntology() - { - return ontology.get(); - } - /** * Returns the media types registry used for content negotiation and outbound {@code Accept} headers. * diff --git a/src/main/java/com/atomgraph/linkeddatahub/server/io/ValidatingModelProvider.java b/src/main/java/com/atomgraph/linkeddatahub/server/io/ValidatingModelProvider.java index 018bca6c2..757c86bc1 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/server/io/ValidatingModelProvider.java +++ b/src/main/java/com/atomgraph/linkeddatahub/server/io/ValidatingModelProvider.java @@ -239,8 +239,9 @@ public Resource processRead(Resource resource) // this logic really belongs in a if (getApplication().isPresent() && getApplication().get().canAs(AdminApplication.class) && resource.hasProperty(RDF.type, OWL.Ontology)) { - // clear cached OntModel if ontology is updated. TO-DO: send event instead + // clear cached raw graph and closure union graph if ontology is updated. TO-DO: send event instead getSystem().getRepository().remove(resource.getURI()); + getSystem().getOntologyGraphs().remove(resource.getURI()); } if (getApplication().isPresent() && resource.hasProperty(RDF.type, ACL.Authorization)) diff --git a/src/main/java/com/atomgraph/linkeddatahub/server/util/ScopedGraphRepository.java b/src/main/java/com/atomgraph/linkeddatahub/server/util/ScopedGraphRepository.java new file mode 100644 index 000000000..4fec524ed --- /dev/null +++ b/src/main/java/com/atomgraph/linkeddatahub/server/util/ScopedGraphRepository.java @@ -0,0 +1,133 @@ +/** + * Copyright 2026 Martynas Jusevičius + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package com.atomgraph.linkeddatahub.server.util; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Stream; +import org.apache.jena.graph.Graph; +import org.apache.jena.ontapi.GraphRepository; + +/** + * A graph repository view that reads through a shared backing repository but keeps writes local. + *

+ * Handed to {@code OntModelFactory.createModel(Graph, OntSpecification, GraphRepository)} so that + * ontapi's union-graph bookkeeping — a {@code UnionGraph} wrapper per ontology in the imports + * closure, with listeners — lands in this instance's private store instead of the shared repository. + * The shared repository must keep answering {@code get(uri)} with the raw per-document graph (that is + * what proxied and direct document GETs serve), and duplicate ontology IDs across applications must + * not collide in one store. Reads fall through to the backing repository, triggering its + * SPARQL-first/mapped/HTTP loading and raw caching as usual, so resolving an imports closure through + * this view populates the shared raw cache as a side effect. + *

+ * After model construction, {@link #ids()} equals the set of resolved closure ontology IDs. + * + * @author Martynas Jusevičius {@literal } + */ +public class ScopedGraphRepository implements GraphRepository +{ + + private final GraphRepository backing; + private final Map local = new HashMap<>(); + + /** + * Constructs the view over a shared backing repository. + * + * @param backing shared graph repository + */ + public ScopedGraphRepository(GraphRepository backing) + { + this.backing = backing; + } + + @Override + public Graph get(String id) + { + Graph graph = local.get(id); + if (graph != null) return graph; + + return getBacking().get(id); + } + + @Override + public Stream ids() + { + return List.copyOf(local.keySet()).stream(); + } + + @Override + public Graph put(String id, Graph graph) + { + return local.put(id, graph); + } + + @Override + public Graph remove(String id) + { + return local.remove(id); + } + + @Override + public void clear() + { + local.clear(); + } + + @Override + public boolean contains(String id) + { + if (local.containsKey(id) || getBacking().contains(id)) return true; + + // the backing repository's contains() only reports already-cached graphs, but ontapi consults + // contains() before get() when resolving imports — a false negative for a resolvable id (bundled + // mapping, SPARQL-first, HTTP) makes ontapi silently substitute an empty ontology graph for the + // import. Attempt resolution instead: the backing repository loads and caches the graph, and only + // a genuinely unresolvable id reports absent + try + { + return getBacking().get(id) != null; + } + catch (RuntimeException ex) + { + return false; + } + } + + @Override + public long count() + { + return local.size(); + } + + @Override + public Stream graphs() + { + return List.copyOf(local.values()).stream(); + } + + /** + * Returns the shared backing repository. + * + * @return graph repository + */ + public GraphRepository getBacking() + { + return backing; + } + +} diff --git a/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyClosureCIReproTest.java b/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyClosureCIReproTest.java new file mode 100644 index 000000000..6cf9bfd2a --- /dev/null +++ b/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyClosureCIReproTest.java @@ -0,0 +1,131 @@ +/** + * Copyright 2026 Martynas Jusevičius + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + * + */ +package com.atomgraph.linkeddatahub.server.filter.request; + +import com.atomgraph.client.util.jena.PrefixGraphRepository; +import org.apache.jena.ontapi.UnionGraph; +import org.apache.jena.rdf.model.Model; +import org.apache.jena.rdf.model.ModelFactory; +import org.apache.jena.rdf.model.Resource; +import org.apache.jena.rdf.model.ResourceFactory; +import org.apache.jena.riot.RDFParser; +import org.apache.jena.vocabulary.OWL; +import org.apache.jena.vocabulary.RDF; +import org.apache.jena.vocabulary.RDFS; +import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.*; + +/** + * Production-shaped regression guard for the imports closure: the end-user ns# ontology as the SPARQL + * CONSTRUCT returns it (header + imports + unrelated document resources), importing the SPARQL-loaded + * ldh# vocabulary, whose transitive imports resolve through the real bundled location mappings + * (dh, spin, sp, foaf, sioc, sd) plus store-seeded stubs for the non-mapped ones (ac, nfo, owl). + *

+ * Pins the fix for ontapi consulting {@code contains()} before {@code get()} during import resolution: + * a cache-state (rather than resolvability) answer made ontapi silently substitute empty ontology + * graphs for every bundled-mapped import, stripping SPIN constraints and vocabularies from the closure. + * + * @author Martynas Jusevičius {@literal } + */ +public class OntologyClosureCIReproTest +{ + + private static final String NS = "https://localhost:4443/ns#"; + private static final String LDH = "https://w3id.org/atomgraph/linkeddatahub#"; + + @Test + public void productionShapedClosureContainsAllImports() + { + PrefixGraphRepository repository = new PrefixGraphRepository(null); + + // real bundled mappings + Model mappingModel = ModelFactory.createDefaultModel(); + RDFParser.create().source("location-mapping.ttl").streamManager(repository.getStreamManager()).build().parse(mappingModel); + repository.processConfig(mappingModel); + + // mimic SPARQL-first load result: the ldh# vocabulary as a store graph + Model ldh = ModelFactory.createDefaultModel(); + RDFParser.create().source("com/atomgraph/linkeddatahub/ldh.ttl").base(LDH).streamManager(repository.getStreamManager()).build().parse(ldh); + repository.put(LDH, ldh.getGraph()); + + // stub graphs for ldh#'s non-mapped imports (SPARQL/HTTP-loaded in production) + for (String stub : new String[] { + "https://w3id.org/atomgraph/client#", + "http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#", + "http://www.w3.org/2002/07/owl#" }) + { + Model m = ModelFactory.createDefaultModel(); + m.add(m.createResource(stub), RDF.type, OWL.Ontology); + repository.put(stub, m.getGraph()); + } + + // ns# base graph as the ontology CONSTRUCT returns it: ontology header + imports + document resource + Model ns = ModelFactory.createDefaultModel(); + Resource nsOnt = ns.createResource(NS); + ns.add(nsOnt, RDF.type, OWL.Ontology); + ns.add(nsOnt, OWL.imports, ns.createResource(LDH)); + Resource doc = ns.createResource("https://admin.localhost:4443/ontologies/namespace/"); + ns.add(doc, RDF.type, ns.createResource("https://www.w3.org/ns/ldt/document-hierarchy#Item")); + ns.add(doc, ResourceFactory.createProperty("http://purl.org/dc/terms/title"), "Namespace"); + repository.put(NS, ns.getGraph()); + + UnionGraph union = OntologyFilter.loadOntology(repository, NS); + Model closure = ModelFactory.createModelForGraph(union); + + // direct import: ldh.ttl content + assertTrue(closure.contains(closure.createResource(LDH + "View"), RDF.type, RDFS.Class), "ldh# (direct import) must be in the closure"); + // transitive via ldh#: dh.ttl (bundled mapping) + assertTrue(closure.contains(closure.createResource("https://www.w3.org/ns/ldt/document-hierarchy#Item"), RDF.type, OWL.Class), "dh# (transitive, bundled) must be in the closure"); + // transitive via ldh#: spin.ttl imported as http://spinrdf.org/spin (no hash) + assertTrue(closure.contains(closure.createResource("http://spinrdf.org/spin#constraint"), RDF.type, RDF.Property), "spin (transitive, bundled, hashless import URI) must be in the closure"); + // transitive via dh#: sp.ttl imported as http://spinrdf.org/sp# + assertTrue(closure.contains(closure.createResource("http://spinrdf.org/sp#text"), RDF.type, RDF.Property), "sp# (transitive via dh#, bundled) must be in the closure"); + // transitive via dh#: foaf (bundled) + assertFalse(closure.listStatements(closure.createResource("http://xmlns.com/foaf/0.1/Agent"), null, (org.apache.jena.rdf.model.RDFNode)null).toList().isEmpty(), "foaf (transitive, bundled) must be in the closure"); + } + + @Test + public void importedDocumentWithMismatchedOntologyIRIIsInTheClosure() + { + // content-addressed uploads: the document URI (uploads/) necessarily differs from the + // ontology IRI the uploaded file declares — both must still land in the closure + String uploadURI = "https://localhost:4443/uploads/da39a3ee5e6b4b0d3255bfef95601890afd80709"; + String declaredURI = "https://example.org/test#"; + + PrefixGraphRepository repository = new PrefixGraphRepository(null); + + Model uploaded = ModelFactory.createDefaultModel(); + Resource declaredOnt = uploaded.createResource(declaredURI); + uploaded.add(declaredOnt, RDF.type, OWL.Ontology); + Resource testClass = uploaded.createResource(declaredURI + "TestClass"); + uploaded.add(testClass, RDF.type, OWL.Class); + uploaded.add(testClass, RDFS.label, "Test Class"); + repository.put(uploadURI, uploaded.getGraph()); + + Model ns = ModelFactory.createDefaultModel(); + Resource nsOnt = ns.createResource(NS); + ns.add(nsOnt, RDF.type, OWL.Ontology); + ns.add(nsOnt, OWL.imports, ns.createResource(uploadURI)); + repository.put(NS, ns.getGraph()); + + UnionGraph union = OntologyFilter.loadOntology(repository, NS); + Model closure = ModelFactory.createModelForGraph(union); + + assertTrue(closure.contains(testClass, RDFS.label, closure.createLiteral("Test Class")), "content of an import whose declared ontology IRI differs from its document URI must be in the closure"); + } + +} diff --git a/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyImportsCharacterizationTest.java b/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyImportsCharacterizationTest.java index 312a61415..bafae14a9 100644 --- a/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyImportsCharacterizationTest.java +++ b/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyImportsCharacterizationTest.java @@ -19,6 +19,7 @@ import com.atomgraph.client.util.jena.PrefixGraphRepository; import org.apache.jena.ontapi.OntModelFactory; import org.apache.jena.ontapi.OntSpecification; +import org.apache.jena.ontapi.UnionGraph; import org.apache.jena.ontapi.model.OntModel; import org.apache.jena.rdf.model.Model; import org.apache.jena.rdf.model.ModelFactory; @@ -31,9 +32,10 @@ import static org.junit.jupiter.api.Assertions.*; /** - * Pins {@link OntologyFilter#loadOntology}: it flattens the owl:imports closure into one graph, - * applies RDFS inference, and materializes the inferences into the repository cache — without ontapi - * managing a union-graph hierarchy over the shared repository (which collides on duplicate ontology IDs). + * Pins {@link OntologyFilter#loadOntology}: it assembles the owl:imports closure as a union graph + * (resolved natively by ontapi over a scoped repository view), applies no inference, and + * leaves the shared repository holding raw per-document graphs — which is what proxied and direct + * document GETs serve. * * @author Martynas Jusevičius {@literal } */ @@ -45,7 +47,7 @@ public class OntologyImportsCharacterizationTest private static final String NS = "http://example.org/ns#"; @Test - public void testLoadOntologyFlattensClosureWithMaterializedRDFSInference() + public void testLoadOntologyResolvesClosureWithoutInference() { PrefixGraphRepository repository = new PrefixGraphRepository(null); @@ -70,22 +72,52 @@ public void testLoadOntologyFlattensClosureWithMaterializedRDFSInference() base.add(baseOnt, OWL.imports, base.createResource(IMPORT_URI)); repository.put(BASE_URI, base.getGraph()); - OntologyFilter.loadOntology(repository, BASE_URI); + UnionGraph union = OntologyFilter.loadOntology(repository, BASE_URI); + Model closure = ModelFactory.createModelForGraph(union); - Model result = ModelFactory.createModelForGraph(repository.get(BASE_URI)); - // (a) imported terms flattened into the cached graph - assertTrue(result.contains(b, RDFS.subClassOf, a), "imported terms should be flattened in"); - // (b) RDFS inference materialized as a concrete triple: x a A - assertTrue(result.contains(x, RDF.type, a), "RDFS-inferred 'x a A' should be materialized in the cached graph"); - // (c) the import is also cached under its (fragment-stripped) document URI - assertTrue(repository.isCached(IMPORT_URI), "import should remain cached"); - // (d) REGRESSION GUARD: both owl:Class and rdfs:Class-only terms must be recognized as OntClasses by the returned - // model, so GET /ns?forClass= resolves the class and runs its SPIN constructor. - // OntologyFilter promotes all rdfs:Class subjects to owl:Class so OWL2 profiles (which do not recognize bare - // rdfs:Class) can find third-party vocab terms like sp:Describe. - OntModel ontology = OntModelFactory.createModel(repository.get(BASE_URI), OntSpecification.OWL2_FULL_MEM); + // (a) imported terms are visible through the closure union + assertTrue(closure.contains(b, RDFS.subClassOf, a), "imported terms should be visible through the closure union"); + // (b) no inference: neither type propagation nor vacuous rdfs:Resource typing appears + assertFalse(closure.contains(x, RDF.type, a), "no RDFS type propagation expected in the closure"); + assertFalse(closure.contains(x, RDF.type, RDFS.Resource), "no vacuous rdfs:Resource typing expected in the closure"); + // (c) the shared repository still holds the RAW document graphs — this is what document GETs serve + assertTrue(ModelFactory.createModelForGraph(repository.get(BASE_URI)).isIsomorphicWith(base), "repository must keep serving the raw base ontology graph"); + assertTrue(ModelFactory.createModelForGraph(repository.get(IMPORT_URI)).isIsomorphicWith(imported), "repository must keep serving the raw imported ontology graph"); + // (d) REGRESSION GUARD: both owl:Class and rdfs:Class-only terms must be recognized as OntClasses by the model + // wrapped over the union, so GET /ns?forClass= resolves the class and runs its SPIN constructor. + // OntologyFilter promotes rdfs:Class subjects to owl:Class in a separate union member so the OWL2 profile + // (which does not recognize bare rdfs:Class) can find third-party vocab terms like sp:Describe. + OntModel ontology = OntModelFactory.createModel(union, OntSpecification.OWL2_FULL_MEM); assertNotNull(ontology.getOntClass(NS + "A"), "owl:Class term must be recognized as an OntClass under OWL2_FULL_MEM"); assertNotNull(ontology.getOntClass(NS + "B"), "rdfs:Class-only term must be recognized as an OntClass after promotion"); + // the promotion must not leak into the raw document graphs + assertFalse(ModelFactory.createModelForGraph(repository.get(IMPORT_URI)).contains(b, RDF.type, OWL.Class), "owl:Class promotion must not be written into the raw document graph"); + } + + @Test + public void testLoadOntologyToleratesImportCycles() + { + PrefixGraphRepository repository = new PrefixGraphRepository(null); + + String firstURI = "http://example.org/first"; + String secondURI = "http://example.org/second"; + Resource term = ResourceFactory.createResource(NS + "Term"); + + Model first = ModelFactory.createDefaultModel(); + Resource firstOnt = first.createResource(firstURI); + first.add(firstOnt, RDF.type, OWL.Ontology); + first.add(firstOnt, OWL.imports, first.createResource(secondURI)); + repository.put(firstURI, first.getGraph()); + + Model second = ModelFactory.createDefaultModel(); + Resource secondOnt = second.createResource(secondURI); + second.add(secondOnt, RDF.type, OWL.Ontology); + second.add(secondOnt, OWL.imports, second.createResource(firstURI)); + second.add(term, RDF.type, OWL.Class); + repository.put(secondURI, second.getGraph()); + + UnionGraph union = OntologyFilter.loadOntology(repository, firstURI); + assertTrue(ModelFactory.createModelForGraph(union).contains(term, RDF.type, OWL.Class), "cyclic imports must resolve without recursing infinitely"); } } diff --git a/src/test/java/com/atomgraph/linkeddatahub/server/util/SPINConstraintValidationTest.java b/src/test/java/com/atomgraph/linkeddatahub/server/util/SPINConstraintValidationTest.java index 6d2062f9c..c2a69790a 100644 --- a/src/test/java/com/atomgraph/linkeddatahub/server/util/SPINConstraintValidationTest.java +++ b/src/test/java/com/atomgraph/linkeddatahub/server/util/SPINConstraintValidationTest.java @@ -63,11 +63,8 @@ private OntModel loadOntology() "com/atomgraph/linkeddatahub/ldh.ttl" }) RDFDataMgr.read(closure, classpath); - // mirror OntologyFilter.loadOntology: RDFS-infer then materialize into a plain OWL2_FULL_MEM graph - OntModel inferred = OntModelFactory.createModel(closure.getGraph(), OntSpecification.OWL2_FULL_MEM_RDFS_INF); - OntModel materialized = OntModelFactory.createModel(OntSpecification.OWL2_FULL_MEM); - materialized.add(inferred); - return materialized; + // mirror OntologyFilter.loadOntology: a plain OWL2_FULL_MEM model over the assembled closure, no inference + return OntModelFactory.createModel(closure.getGraph(), OntSpecification.OWL2_FULL_MEM); } /** A dh:Item with NO dct:title — violates the MissingTitle constraint. */ From 84c27aff941815b853554e6a709f8fbd88afcb42 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sat, 8 Aug 2026 20:40:23 +0200 Subject: [PATCH 3/5] Resolve view term labels from /ns instead of the Linked Data proxy (#340) The four view controls (order-by dropdown, facet headers, parallax properties, rdf:type facet values) resolved predicate/class labels by proxying each term's vocabulary document (GET ?uri=), then applying ac:label to the response. With the proxy no longer serving ontology terms, route these through the app's /ns SPARQL endpoint instead. - view-results chain: add a property-metadata load step (mirroring the existing object-metadata step) that DESCRIBEs the result predicates over /ns before ldh:render-view, threading $property-metadata through to the order-by and facet-header rendering. - order-by + facet headers: render labels synchronously from $property-metadata via ac:label (local-name fallback for terms absent from the closure); drop the per-predicate proxy promises and the ldh:order-by-response / ldh:facet-filter-response handlers. - parallax + rdf:type facet values: swap each per-term proxy GET for a POST /ns DESCRIBE $Type VALUES { }; the response is still application/rdf+xml so the existing handlers are unchanged. Verified against the unesco-thesaurus demo: on /concepts/concept1002/ the redundant vocab-document proxy fetches drop (skos/core 9->5, rdf 2->1, prov 2->1), the SKOS view predicates now resolve via /ns, and the 429/502 the redundant load triggered are gone. Co-authored-by: Claude Opus 4.8 (1M context) --- .../xsl/bootstrap/2.3.2/client/block/view.xsl | 186 ++++++------------ 1 file changed, 62 insertions(+), 124 deletions(-) diff --git a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/bootstrap/2.3.2/client/block/view.xsl b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/bootstrap/2.3.2/client/block/view.xsl index c309ecd42..bc95f3ea6 100644 --- a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/bootstrap/2.3.2/client/block/view.xsl +++ b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/bootstrap/2.3.2/client/block/view.xsl @@ -158,6 +158,10 @@ exclude-result-prefixes="#all" ixsl:then(ldh:http-request-threaded(?, 'metadata-request', 'metadata-response')) => ixsl:then(ldh:handle-response(?, 'metadata-response')) => ixsl:then(ldh:set-object-metadata#1) => + ixsl:then(ldh:load-property-metadata(?, 'view-results-response')) => + ixsl:then(ldh:http-request-threaded(?, 'property-metadata-request', 'property-metadata-response')) => + ixsl:then(ldh:handle-response(?, 'property-metadata-response')) => + ixsl:then(ldh:set-property-metadata#1) => ixsl:then(ldh:render-view#1) "/> @@ -781,6 +785,7 @@ exclude-result-prefixes="#all" + @@ -822,18 +827,33 @@ exclude-result-prefixes="#all" - + - - - + + @@ -865,31 +885,6 @@ exclude-result-prefixes="#all"

- - - - - - - - - - - - - @@ -939,6 +934,7 @@ exclude-result-prefixes="#all" + @@ -965,21 +961,28 @@ exclude-result-prefixes="#all" - - - - + + + + + + + + + + + + + + + + + + + + + + @@ -1845,6 +1848,7 @@ exclude-result-prefixes="#all" + @@ -1920,6 +1924,7 @@ exclude-result-prefixes="#all" + @@ -1929,6 +1934,7 @@ exclude-result-prefixes="#all" + @@ -1954,45 +1960,6 @@ exclude-result-prefixes="#all" - - - - - - - - - - ldh:facet-filter-response - - - - - - - - - - - - - - - - - - - - - - - - - - - - - @@ -2010,8 +1977,9 @@ exclude-result-prefixes="#all" - - + + + + ), map{ 'duplicates': 'use-last' })"/> - - + + + - - - - - - - - - - - - - - - - - - - - - - - - \ No newline at end of file From f7adf3d2d18d450ad89261e46abb518dcf3b3342 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sat, 8 Aug 2026 21:44:18 +0200 Subject: [PATCH 4/5] Replace flaky DBpedia proxy test with a local cross-origin query POST-proxied-external-query.sh proxied a live query to dbpedia.org/sparql, whose public endpoint intermittently returns 502 in CI. Rename it to POST-proxied-cross-origin-query.sh and point it at the admin app's SPARQL endpoint (a different origin than the end-user app), so it still exercises the ProxyRequestFilter remote-fetch + SPARQL-results re-serialization path but runs deterministically against a local target. Use the owner cert since the admin endpoint is ACL-protected, and assert the response is actual SPARQL results rather than only a 200 status. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../proxy/POST-proxied-cross-origin-query.sh | 37 +++++++++++++++++ .../proxy/POST-proxied-external-query.sh | 41 ------------------- 2 files changed, 37 insertions(+), 41 deletions(-) create mode 100644 http-tests/proxy/POST-proxied-cross-origin-query.sh delete mode 100644 http-tests/proxy/POST-proxied-external-query.sh diff --git a/http-tests/proxy/POST-proxied-cross-origin-query.sh b/http-tests/proxy/POST-proxied-cross-origin-query.sh new file mode 100644 index 000000000..989d5591c --- /dev/null +++ b/http-tests/proxy/POST-proxied-cross-origin-query.sh @@ -0,0 +1,37 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" + +# Execute a SPARQL query against a cross-origin endpoint (the admin app's SPARQL +# endpoint) using the end-user app as a proxy. Because admin.localhost is a different +# origin than the end-user app, the request goes through ProxyRequestFilter, which +# fetches the remote endpoint and re-serializes the SPARQL results back to the caller. +# The owner is used because the admin SPARQL endpoint is ACL-protected. + +response_body=$(curl -k -s \ + -X POST \ + -E "$OWNER_CERT_FILE":"$OWNER_CERT_PWD" \ + -H 'Content-Type: application/sparql-query' \ + -H 'Accept: application/sparql-results+xml' \ + --url-query "uri=${ADMIN_BASE_URL}sparql" \ + --data 'SELECT (COUNT(*) AS ?count) WHERE { ?s ?p ?o }' \ + "$END_USER_BASE_URL") + +http_code=$(curl -k -s -o /dev/null -w "%{http_code}" \ + -X POST \ + -E "$OWNER_CERT_FILE":"$OWNER_CERT_PWD" \ + -H 'Content-Type: application/sparql-query' \ + -H 'Accept: application/sparql-results+xml' \ + --url-query "uri=${ADMIN_BASE_URL}sparql" \ + --data 'SELECT (COUNT(*) AS ?count) WHERE { ?s ?p ?o }' \ + "$END_USER_BASE_URL") + +# verify successful status and that the proxy re-serialized actual SPARQL results +if [ "$http_code" -ne 200 ] || [[ "$response_body" != *"http://www.w3.org/2005/sparql-results#"* ]]; then + exit 1 +fi diff --git a/http-tests/proxy/POST-proxied-external-query.sh b/http-tests/proxy/POST-proxied-external-query.sh deleted file mode 100644 index 18e33ba81..000000000 --- a/http-tests/proxy/POST-proxied-external-query.sh +++ /dev/null @@ -1,41 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" -purge_cache "$FRONTEND_VARNISH_SERVICE" - -# add agent to the writers group - POST requests count as write operations - -add-agent-to-group.sh \ - -f "$OWNER_CERT_FILE" \ - -p "$OWNER_CERT_PWD" \ - --agent "$AGENT_URI" \ - "${ADMIN_BASE_URL}acl/groups/writers/" - -# execute SPARQL query using LDH as a proxy to query DBpedia - -response_body=$(curl -k -s \ - -X POST \ - -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ - -H 'Content-Type: application/sparql-query' \ - -H 'Accept: application/sparql-results+xml' \ - --url-query "uri=https://dbpedia.org/sparql" \ - --data 'SELECT ?title WHERE { ?title } LIMIT 1' \ - "$END_USER_BASE_URL") - -http_code=$(curl -k -s -o /dev/null -w "%{http_code}" \ - -X POST \ - -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ - -H 'Content-Type: application/sparql-query' \ - -H 'Accept: application/sparql-results+xml' \ - --url-query "uri=https://dbpedia.org/sparql" \ - --data 'SELECT ?title WHERE { ?title } LIMIT 1' \ - "$END_USER_BASE_URL") - -# verify response has non-empty body and successful status -if [ "$http_code" -ne 200 ] || [ -z "$response_body" ]; then - exit 1 -fi \ No newline at end of file From 9da6ff0d05e43e78a439299d77018b0368028827 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Sun, 9 Aug 2026 21:12:42 +0200 Subject: [PATCH 5/5] Client-orchestrated graph writes; inference-free ontology serving (#343) * Replace /add and /generate endpoints with client-orchestrated graph writes Move the "Add data" and "Generate containers" orchestration to the client over the uniform Graph Store Protocol interface, removing the server-side fetch/SSRF surface (pen-test LNK-002) and collapsing two special-purpose endpoints into standard per-document writes. - Add data: the submit handler splits by the presence of a spin:query input. The add/clone variant fetches dct:source through the same-origin ?uri= proxy as RDF/XML (CORS + Jena format conversion) and POST-appends it to the sd:name target (ldh:add-data-source-response / ldh:add-data-form-error). The import-ontology variant is unchanged and still RDF/POSTs to /transform. - Generate containers: builds one container document per checked class and PUTs them via parallel ixsl:all fan-out (ldh:generate-containers-fanout -> ldh:generate-containers-join), seeded by ixsl:resolve so the requests run in an active promise context. The view block is now correctly wrapped as ldh:Object -> rdf:value -> ldh:View (the endpoint's bare ldh:View bypassed ldh:InvalidContentBlockType validation). - Delete Add.java, Generate.java and their Dispatcher locators. /transform is retained until a client-side SPARQL engine lands. - http-tests: drop the obsolete /add and /generate tests (the system/ suite is unregistered); add add/GET-proxied-source-POST-append.sh and add/PUT-generate-container.sh. CHANGELOG updated. Co-Authored-By: Claude Opus 4.8 (1M context) * Fix generate-containers dialog: parent init, disabled Generate, async schema load - Initialise the parent typeahead with the current container (ldh:LoadTypeaheads from ldh:base-uri, matching btn-save-as), which was lost when the form stopped carrying a source param. - Disable the Generate button until the schema is loaded; enable it in ldh:endpoint-classes-response once the class list is populated. - Make Load schema fully async: the service-endpoint resolution used a blocking document() fetch (the "TO-DO: asynchronous request"). Replaced with an ixsl:resolve -> ldh:load-schema-endpoint -> ldh:load-schema-results promise chain, so the request no longer blocks and the progress cursor shows. Co-Authored-By: Claude Opus 4.8 (1M context) * Add RDF data: seed target graph from the local dataspace, reject remote targets The "Add RDF data" dialog seeded its target graph typeahead from ldh:base-uri(.), which resolves to the proxied remote resource when viewing one (ac:document-uri(ac:uri())). So adding data while viewing a proxied document defaulted the write target to that remote and proxied the append to it (403). The source is still ldh:base-uri (the remote being imported). - btn-save-as now seeds the graph typeahead from ac:absolute-path(ldh:request-uri()) (the local browser location), which is always a local dataspace document. - Guard the append: a cross-origin target renders an inline error instead of proxying the write. ldh:add-data-form-error now tolerates a missing response so pre-fetch validation can reuse it. Co-Authored-By: Claude Opus 4.8 (1M context) * Condense Unreleased CHANGELOG entries to one line each Co-Authored-By: Claude Opus 4.8 (1M context) --------- Co-authored-by: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 10 +- .../add/GET-proxied-source-POST-append.sh | 54 +++ http-tests/add/POST-add.sh | 41 -- http-tests/add/PUT-generate-container.sh | 82 ++++ http-tests/system/end-user/POST-add-401.sh | 23 - http-tests/system/end-user/POST-add-403.sh | 24 - .../system/end-user/POST-add-readers-403.sh | 31 -- http-tests/system/end-user/POST-add.sh | 44 -- .../system/end-user/POST-generate-401.sh | 23 - .../system/end-user/POST-generate-403.sh | 24 - .../end-user/POST-generate-readers-403.sh | 31 -- http-tests/system/end-user/POST-generate.sh | 50 -- .../atomgraph/linkeddatahub/resource/Add.java | 185 -------- .../linkeddatahub/resource/Generate.java | 313 ------------- .../server/model/impl/Dispatcher.java | 24 - .../xsl/bootstrap/2.3.2/client/modal.xsl | 431 ++++++++++++++---- .../atomgraph/linkeddatahub/xsl/client.xsl | 3 +- 17 files changed, 486 insertions(+), 907 deletions(-) create mode 100755 http-tests/add/GET-proxied-source-POST-append.sh delete mode 100755 http-tests/add/POST-add.sh create mode 100755 http-tests/add/PUT-generate-container.sh delete mode 100755 http-tests/system/end-user/POST-add-401.sh delete mode 100755 http-tests/system/end-user/POST-add-403.sh delete mode 100755 http-tests/system/end-user/POST-add-readers-403.sh delete mode 100755 http-tests/system/end-user/POST-add.sh delete mode 100755 http-tests/system/end-user/POST-generate-401.sh delete mode 100755 http-tests/system/end-user/POST-generate-403.sh delete mode 100755 http-tests/system/end-user/POST-generate-readers-403.sh delete mode 100755 http-tests/system/end-user/POST-generate.sh delete mode 100644 src/main/java/com/atomgraph/linkeddatahub/resource/Add.java delete mode 100644 src/main/java/com/atomgraph/linkeddatahub/resource/Generate.java diff --git a/CHANGELOG.md b/CHANGELOG.md index 109d3ac07..66d9c2c1a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,13 +1,15 @@ ## [Unreleased] ### Changed -- Application ontologies resolved as a native ontapi `owl:imports` union graph (cached per ontology URI) instead of a manually flattened, RDFS-materialized model — no RDFS inference -- `Namespace` no-query GET serves the raw ontology graph from the shared repository instead of rebuilding a repository per request +- Application ontologies resolved as a native ontapi `owl:imports` union graph (cached per ontology URI), no RDFS inference — replaces the manually flattened, RDFS-materialized model +- `Namespace` no-query GET serves the raw ontology graph from the shared repository instead of rebuilding one per request +- **BREAKING**: "Add data" and "Generate containers" orchestrated client-side over the Graph Store Protocol (POST-append via `?uri=` proxy; per-class container PUT fan-out embedding the view as `ldh:Object` → `rdf:value` → `ldh:View`), replacing the `/add` and `/generate` endpoints ### Fixed -- Raw ontology graphs no longer leak inferred `rdf:type rdfs:Resource`, which produced multi-token `@typeof` that broke View block rendering +- Raw ontology graphs no longer leak inferred `rdf:type rdfs:Resource` that broke View block rendering via multi-token `@typeof` ### Removed -- The Linked Data proxy no longer serves ontology terms; it is now dumb transport (bundled-vocab file cache + SSRF-checked external fetch), with ontology terms served by `/ns` +- Linked Data proxy no longer serves ontology terms (now dumb transport: bundled-vocab file cache + SSRF-checked external fetch); ontology terms served by `/ns` +- **BREAKING**: `/add` and `/generate` server-side endpoints (`Add`/`Generate` JAX-RS resources), superseded by the client-orchestrated writes; removes their server-side fetch/SSRF surface (LNK-002); `/transform` retained until a client-side SPARQL engine lands ## [5.7.1] - 2026-08-06 ### Changed diff --git a/http-tests/add/GET-proxied-source-POST-append.sh b/http-tests/add/GET-proxied-source-POST-append.sh new file mode 100755 index 000000000..5db813ac2 --- /dev/null +++ b/http-tests/add/GET-proxied-source-POST-append.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" + +# Exercises the client-orchestrated "Add data" flow that replaced the server-side /add endpoint: +# the browser GETs the external source through the same-origin ?uri= proxy as RDF/XML, then +# POSTs (appends) it to the target document. Two requests, no /add endpoint. + +# add agent to the readers group (to read through the proxy) and the writers group (to append) + +add-agent-to-group.sh \ + -f "$OWNER_CERT_FILE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/readers/" + +add-agent-to-group.sh \ + -f "$OWNER_CERT_FILE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/writers/" + +# create the target container + +container=$(create-container.sh \ + -f "$AGENT_CERT_FILE" \ + -p "$AGENT_CERT_PWD" \ + -b "$END_USER_BASE_URL" \ + --title "Test" \ + --slug "test" \ + --parent "$END_USER_BASE_URL") + +# step 1: fetch the external source through the LDH proxy, converted to RDF/XML + +source_rdfxml=$(curl -k -f -s -G \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/rdf+xml" \ + --data-urlencode "uri=https://orcid.org/0000-0003-1750-9906" \ + "$END_USER_BASE_URL") + +# step 2: append the fetched triples to the target container document (GSP append -> 204) + +echo "$source_rdfxml" | curl -k -w "%{http_code}\n" -o /dev/null -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -X POST \ + -H "Content-Type: application/rdf+xml" \ + --data-binary @- \ + "$container" \ +| grep -q "$STATUS_NO_CONTENT" diff --git a/http-tests/add/POST-add.sh b/http-tests/add/POST-add.sh deleted file mode 100755 index dcca9c6d8..000000000 --- a/http-tests/add/POST-add.sh +++ /dev/null @@ -1,41 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" - -# add agent to the writers group - -add-agent-to-group.sh \ - -f "$OWNER_CERT_FILE" \ - -p "$OWNER_CERT_PWD" \ - --agent "$AGENT_URI" \ - "${ADMIN_BASE_URL}acl/groups/writers/" - -# create container - -slug="test" - -container=$(create-container.sh \ - -f "$AGENT_CERT_FILE" \ - -p "$AGENT_CERT_PWD" \ - -b "$END_USER_BASE_URL" \ - --title "Test" \ - --slug "$slug" \ - --parent "$END_USER_BASE_URL") - -# import data into the container - -curl -w "%{http_code}\n" -o /dev/null -k -s \ - -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ - -H "Content-Type: application/x-www-form-urlencoded" \ - --data-urlencode "rdf=" \ - --data-urlencode "sb=clone" \ - --data-urlencode "pu=http://purl.org/dc/terms/source" \ - --data-urlencode "ou=https://orcid.org/0000-0003-1750-9906" \ - --data-urlencode "pu=http://www.w3.org/ns/sparql-service-description#name" \ - --data-urlencode "ou=${container}" \ - "${END_USER_BASE_URL}add" \ -| grep -q "$STATUS_NO_CONTENT" diff --git a/http-tests/add/PUT-generate-container.sh b/http-tests/add/PUT-generate-container.sh new file mode 100755 index 000000000..9500e29a1 --- /dev/null +++ b/http-tests/add/PUT-generate-container.sh @@ -0,0 +1,82 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" + +# Exercises the client-orchestrated "Generate containers" flow that replaced the server-side +# /generate endpoint. The client builds one container document per checked class -- a dh:Container +# whose content block is an ldh:Object wrapping an ldh:View over a $type-parameterized SELECT -- and +# PUTs it. This test PUTs one such container (shaped exactly like ldh:generate-container-doc output) +# and verifies: creation succeeds (the Object-wrapped block passes ldh:InvalidContentBlockType / +# MissingValue / MissingQuery validation), the server stamps metadata, and the block persists. + +# add agent to the writers group + +add-agent-to-group.sh \ + -f "$OWNER_CERT_FILE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/writers/" + +parent="$END_USER_BASE_URL" +uuid=$(uuidgen | tr '[:upper:]' '[:lower:]') +container="${parent}${uuid}/" +class="https://www.w3.org/ns/ldt/document-hierarchy#Container" + +# PUT the generated container document (blank nodes are skolemized server-side) + +http_code=$(curl -k -s -o /dev/null -w "%{http_code}" \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -X PUT \ + -H "Content-Type: application/rdf+xml" \ + --data-binary @- \ + "$container" < + + + + Containers + ${uuid} + + + + + + + + + + Select Container + SELECT DISTINCT ?s WHERE { ?s a <${class}> ; ?p ?o } + + + + + + + + +EOF +) + +[ "$http_code" = "$STATUS_CREATED" ] + +# fetch the created container and verify the shape + server-stamped metadata + +ntriples=$(curl -k -f -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/n-triples" \ + "$container") + +# parent link, generated title, and server-stamped creation date +echo "$ntriples" | grep -q " <${parent}>" +echo "$ntriples" | grep -q " \"Containers\"" +echo "$ntriples" | grep -q "" + +# content block persisted as an ldh:Object, and the SELECT carries the substituted class IRI +echo "$ntriples" | grep -q "" +echo "$ntriples" | grep "" | grep -q "${class}" diff --git a/http-tests/system/end-user/POST-add-401.sh b/http-tests/system/end-user/POST-add-401.sh deleted file mode 100755 index bb927b19a..000000000 --- a/http-tests/system/end-user/POST-add-401.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" -purge_cache "$FRONTEND_VARNISH_SERVICE" - -# POST /add without a certificate should return 401 -# Only owners and writers have acl:Append access to /add via write-append authorization - -curl -k -w "%{http_code}\n" -o /dev/null -s \ - -X POST \ - -H "Content-Type: application/x-www-form-urlencoded" \ - --data-urlencode "rdf=" \ - --data-urlencode "sb=clone" \ - --data-urlencode "pu=http://purl.org/dc/terms/source" \ - --data-urlencode "ou=https://orcid.org/0000-0003-1750-9906" \ - --data-urlencode "pu=http://www.w3.org/ns/sparql-service-description#name" \ - --data-urlencode "ou=${END_USER_BASE_URL}" \ - "${END_USER_BASE_URL}add" \ -| grep -q "$STATUS_UNAUTHORIZED" diff --git a/http-tests/system/end-user/POST-add-403.sh b/http-tests/system/end-user/POST-add-403.sh deleted file mode 100755 index 68d43ea7e..000000000 --- a/http-tests/system/end-user/POST-add-403.sh +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" -purge_cache "$FRONTEND_VARNISH_SERVICE" - -# POST /add with a signed-up agent not in any group should return 403 -# The write-append authorization grants acl:Append to owners and writers groups only - -curl -k -w "%{http_code}\n" -o /dev/null -s \ - -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ - -X POST \ - -H "Content-Type: application/x-www-form-urlencoded" \ - --data-urlencode "rdf=" \ - --data-urlencode "sb=clone" \ - --data-urlencode "pu=http://purl.org/dc/terms/source" \ - --data-urlencode "ou=https://orcid.org/0000-0003-1750-9906" \ - --data-urlencode "pu=http://www.w3.org/ns/sparql-service-description#name" \ - --data-urlencode "ou=${END_USER_BASE_URL}" \ - "${END_USER_BASE_URL}add" \ -| grep -q "$STATUS_FORBIDDEN" diff --git a/http-tests/system/end-user/POST-add-readers-403.sh b/http-tests/system/end-user/POST-add-readers-403.sh deleted file mode 100755 index d512b3ee4..000000000 --- a/http-tests/system/end-user/POST-add-readers-403.sh +++ /dev/null @@ -1,31 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" -purge_cache "$FRONTEND_VARNISH_SERVICE" - -# POST /add with a reader should return 403 -# The write-append authorization grants acl:Append to owners and writers groups only; -# readers only have acl:Read on dh:Item/Container and /sparql, not on /add - -add-agent-to-group.sh \ - -f "$OWNER_CERT_FILE" \ - -p "$OWNER_CERT_PWD" \ - --agent "$AGENT_URI" \ - "${ADMIN_BASE_URL}acl/groups/readers/" - -curl -k -w "%{http_code}\n" -o /dev/null -s \ - -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ - -X POST \ - -H "Content-Type: application/x-www-form-urlencoded" \ - --data-urlencode "rdf=" \ - --data-urlencode "sb=clone" \ - --data-urlencode "pu=http://purl.org/dc/terms/source" \ - --data-urlencode "ou=https://orcid.org/0000-0003-1750-9906" \ - --data-urlencode "pu=http://www.w3.org/ns/sparql-service-description#name" \ - --data-urlencode "ou=${END_USER_BASE_URL}" \ - "${END_USER_BASE_URL}add" \ -| grep -q "$STATUS_FORBIDDEN" diff --git a/http-tests/system/end-user/POST-add.sh b/http-tests/system/end-user/POST-add.sh deleted file mode 100755 index 40f715c76..000000000 --- a/http-tests/system/end-user/POST-add.sh +++ /dev/null @@ -1,44 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" -purge_cache "$FRONTEND_VARNISH_SERVICE" - -# add agent to the writers group - -add-agent-to-group.sh \ - -f "$OWNER_CERT_FILE" \ - -p "$OWNER_CERT_PWD" \ - --agent "$AGENT_URI" \ - "${ADMIN_BASE_URL}acl/groups/writers/" - -# create container to hold the cloned data - -slug=$(uuidgen | tr '[:upper:]' '[:lower:]') - -container=$(create-container.sh \ - -f "$AGENT_CERT_FILE" \ - -p "$AGENT_CERT_PWD" \ - -b "$END_USER_BASE_URL" \ - --title "Test container" \ - --slug "$slug" \ - --parent "$END_USER_BASE_URL") - -# POST /add with a writer should succeed -# Clone data from a remote RDF source into the container - -curl -k -w "%{http_code}\n" -o /dev/null -s \ - -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ - -X POST \ - -H "Content-Type: application/x-www-form-urlencoded" \ - --data-urlencode "rdf=" \ - --data-urlencode "sb=clone" \ - --data-urlencode "pu=http://purl.org/dc/terms/source" \ - --data-urlencode "ou=https://orcid.org/0000-0003-1750-9906" \ - --data-urlencode "pu=http://www.w3.org/ns/sparql-service-description#name" \ - --data-urlencode "ou=${container}" \ - "${END_USER_BASE_URL}add" \ -| grep -q "$STATUS_NO_CONTENT" diff --git a/http-tests/system/end-user/POST-generate-401.sh b/http-tests/system/end-user/POST-generate-401.sh deleted file mode 100755 index a99df0375..000000000 --- a/http-tests/system/end-user/POST-generate-401.sh +++ /dev/null @@ -1,23 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" -purge_cache "$FRONTEND_VARNISH_SERVICE" - -# POST /generate without a certificate should return 401 -# Only owners and writers have acl:Append access to /generate via write-append authorization - -( -curl -k -w "%{http_code}\n" -o /dev/null -s \ - -X POST \ - -H "Content-Type: text/turtle" \ - --data-binary @- \ - "${END_USER_BASE_URL}generate" < . -[] sioc:has_parent <${END_USER_BASE_URL}> . -EOF -) \ -| grep -q "$STATUS_UNAUTHORIZED" diff --git a/http-tests/system/end-user/POST-generate-403.sh b/http-tests/system/end-user/POST-generate-403.sh deleted file mode 100755 index 70838dafb..000000000 --- a/http-tests/system/end-user/POST-generate-403.sh +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" -purge_cache "$FRONTEND_VARNISH_SERVICE" - -# POST /generate with a signed-up agent not in any group should return 403 -# The write-append authorization grants acl:Append to owners and writers groups only - -( -curl -k -w "%{http_code}\n" -o /dev/null -s \ - -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ - -X POST \ - -H "Content-Type: text/turtle" \ - --data-binary @- \ - "${END_USER_BASE_URL}generate" < . -[] sioc:has_parent <${END_USER_BASE_URL}> . -EOF -) \ -| grep -q "$STATUS_FORBIDDEN" diff --git a/http-tests/system/end-user/POST-generate-readers-403.sh b/http-tests/system/end-user/POST-generate-readers-403.sh deleted file mode 100755 index 751aeb866..000000000 --- a/http-tests/system/end-user/POST-generate-readers-403.sh +++ /dev/null @@ -1,31 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" -purge_cache "$FRONTEND_VARNISH_SERVICE" - -# POST /generate with a reader should return 403 -# The write-append authorization grants acl:Append to owners and writers groups only; -# readers only have acl:Read on dh:Item/Container and /sparql, not on /generate - -add-agent-to-group.sh \ - -f "$OWNER_CERT_FILE" \ - -p "$OWNER_CERT_PWD" \ - --agent "$AGENT_URI" \ - "${ADMIN_BASE_URL}acl/groups/readers/" - -( -curl -k -w "%{http_code}\n" -o /dev/null -s \ - -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ - -X POST \ - -H "Content-Type: text/turtle" \ - --data-binary @- \ - "${END_USER_BASE_URL}generate" < . -[] sioc:has_parent <${END_USER_BASE_URL}> . -EOF -) \ -| grep -q "$STATUS_FORBIDDEN" diff --git a/http-tests/system/end-user/POST-generate.sh b/http-tests/system/end-user/POST-generate.sh deleted file mode 100755 index 715b671db..000000000 --- a/http-tests/system/end-user/POST-generate.sh +++ /dev/null @@ -1,50 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" -initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" -purge_cache "$END_USER_VARNISH_SERVICE" -purge_cache "$ADMIN_VARNISH_SERVICE" -purge_cache "$FRONTEND_VARNISH_SERVICE" - -# add agent to the writers group - -add-agent-to-group.sh \ - -f "$OWNER_CERT_FILE" \ - -p "$OWNER_CERT_PWD" \ - --agent "$AGENT_URI" \ - "${ADMIN_BASE_URL}acl/groups/writers/" - -# create a parent container to generate into - -slug=$(uuidgen | tr '[:upper:]' '[:lower:]') - -parent=$(create-container.sh \ - -f "$AGENT_CERT_FILE" \ - -p "$AGENT_CERT_PWD" \ - -b "$END_USER_BASE_URL" \ - --title "Generate parent" \ - --slug "$slug" \ - --parent "$END_USER_BASE_URL") - -# POST /generate with a writer: generate a container for dh:Container class using ldh:SelectChildren query - -( -curl -k -w "%{http_code}\n" -o /dev/null -s \ - -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ - -X POST \ - -H "Content-Type: text/turtle" \ - --data-binary @- \ - "${END_USER_BASE_URL}generate" < . -@prefix void: . -@prefix spin: . -@prefix dh: . -@prefix ldh: . - -[] sioc:has_parent <${parent}> ; - void:class dh:Container ; - spin:query ldh:SelectChildren . -EOF -) \ -| grep -q "$STATUS_OK" diff --git a/src/main/java/com/atomgraph/linkeddatahub/resource/Add.java b/src/main/java/com/atomgraph/linkeddatahub/resource/Add.java deleted file mode 100644 index 9bc6b93ad..000000000 --- a/src/main/java/com/atomgraph/linkeddatahub/resource/Add.java +++ /dev/null @@ -1,185 +0,0 @@ -/** - * Copyright 2021 Martynas Jusevičius - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - */ -package com.atomgraph.linkeddatahub.resource; - -import com.atomgraph.core.MediaTypes; -import com.atomgraph.core.vocabulary.SD; -import com.atomgraph.linkeddatahub.client.GraphStoreClient; -import com.atomgraph.linkeddatahub.server.security.AgentContext; -import java.io.InputStream; -import java.io.OutputStream; -import java.net.URI; -import java.util.Optional; -import jakarta.inject.Inject; -import jakarta.ws.rs.BadRequestException; -import jakarta.ws.rs.POST; -import jakarta.ws.rs.client.Entity; -import jakarta.ws.rs.core.Context; -import jakarta.ws.rs.core.Request; -import jakarta.ws.rs.core.Response; -import jakarta.ws.rs.core.StreamingOutput; -import jakarta.ws.rs.core.UriInfo; -import jakarta.ws.rs.ext.Providers; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ResIterator; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.DCTerms; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -/** - * JAX-RS endpoint for adding RDF data. - * - * @author {@literal Martynas Jusevičius } - */ -public class Add -{ - - private static final Logger log = LoggerFactory.getLogger(Add.class); - - private final UriInfo uriInfo; - private final MediaTypes mediaTypes; - private final Optional agentContext; - private final com.atomgraph.linkeddatahub.Application system; - - /** - * Constructs endpoint for synchronous RDF data imports. - * - * @param request current request - * @param uriInfo current URI info - * @param mediaTypes supported media types - * @param providers JAX-RS providers - * @param system system application - * @param agentContext authenticated agent's context - */ - @Inject - public Add(@Context Request request, @Context UriInfo uriInfo, MediaTypes mediaTypes, - Optional agentContext, - @Context Providers providers, com.atomgraph.linkeddatahub.Application system) - { - this.uriInfo = uriInfo; - this.mediaTypes = mediaTypes; - this.agentContext = agentContext; - this.system = system; - } - - /** - * Adds RDF data from a remote source to a named graph. - * Expects a model containing a resource with dct:source (source URI) and sd:name (target graph URI) properties. - * - * @param model the RDF model containing the import parameters - * @return JAX-RS response with the imported data - */ - @POST - public Response post(Model model) - { - ResIterator it = model.listSubjectsWithProperty(DCTerms.source); - try - { - if (!it.hasNext()) throw new BadRequestException("Argument resource not provided"); - - Resource arg = it.next(); - Resource source = arg.getPropertyResourceValue(DCTerms.source); - if (source == null) throw new BadRequestException("RDF source URI (dct:source) not provided"); - - Resource graph = arg.getPropertyResourceValue(SD.name); - if (graph == null || !graph.isURIResource()) throw new BadRequestException("Graph URI (sd:name) not provided"); - - GraphStoreClient gsc = GraphStoreClient.create(getSystem().getClient(), getMediaTypes()); // TO-DO: inject - Model importModel = gsc.getModel(source.getURI()); - // forward the stream to the named graph document -- do not directly append triples to graph because the agent might not have access to it - return forwardPost(Entity.entity(importModel, com.atomgraph.client.MediaType.APPLICATION_NTRIPLES_TYPE), graph.getURI()); - } - finally - { - it.close(); - } - } - - /** - * Forwards POST request to a graph. - * - * @param entity request entity - * @param graphURI the graph URI - * @return JAX-RS response - */ - protected Response forwardPost(Entity entity, String graphURI) - { - GraphStoreClient gsc = GraphStoreClient.create(getSystem().getClient(), getSystem().getMediaTypes()). - delegation(getUriInfo().getBaseUri(), getAgentContext().orElse(null)); - // forward the stream to the named graph document. Buffer the entity first so that the server response is not returned before the client response completes - try (Response response = gsc.post(URI.create(graphURI), entity, gsc.getReadableMediaTypes(Model.class))) - { - return Response.status(response.getStatus()). - entity(response.readEntity(Model.class)). - build(); - } - } - - /** - * Converts input stream to streaming output. - * @param is input stream - * @return streaming output - */ - public StreamingOutput getStreamingOutput(InputStream is) - { - return (OutputStream os) -> { - is.transferTo(os); - }; - } - - /** - * Returns the supported media types. - * - * @return media types - */ - public MediaTypes getMediaTypes() - { - return mediaTypes; - } - - /** - * Returns the current URI info. - * - * @return URI info - */ - public UriInfo getUriInfo() - { - return uriInfo; - } - - /** - * Returns the authenticated agent's context. - * - * @return optional agent context - */ - public Optional getAgentContext() - { - return agentContext; - } - - /** - * Returns the system application. - * - * @return system application - */ - public com.atomgraph.linkeddatahub.Application getSystem() - { - return system; - } - -} diff --git a/src/main/java/com/atomgraph/linkeddatahub/resource/Generate.java b/src/main/java/com/atomgraph/linkeddatahub/resource/Generate.java deleted file mode 100644 index 27e2124ff..000000000 --- a/src/main/java/com/atomgraph/linkeddatahub/resource/Generate.java +++ /dev/null @@ -1,313 +0,0 @@ -/** - * Copyright 2022 Martynas Jusevičius - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - * - */ -package com.atomgraph.linkeddatahub.resource; - -import com.atomgraph.core.MediaTypes; -import com.atomgraph.linkeddatahub.apps.model.Application; -import com.atomgraph.linkeddatahub.server.model.impl.DocumentHierarchyGraphStoreImpl; -import com.atomgraph.linkeddatahub.server.security.AgentContext; -import com.atomgraph.linkeddatahub.server.util.Skolemizer; -import com.atomgraph.linkeddatahub.vocabulary.LDH; -import com.atomgraph.linkeddatahub.vocabulary.VoID; -import com.atomgraph.linkeddatahub.vocabulary.DH; -import com.atomgraph.linkeddatahub.vocabulary.SIOC; -import com.atomgraph.spinrdf.vocabulary.SP; -import com.atomgraph.spinrdf.vocabulary.SPIN; -import java.net.URI; -import java.util.Calendar; -import java.util.Optional; -import java.util.UUID; -import jakarta.inject.Inject; -import jakarta.ws.rs.BadRequestException; -import jakarta.ws.rs.InternalServerErrorException; -import jakarta.ws.rs.POST; -import jakarta.ws.rs.container.ResourceContext; -import jakarta.ws.rs.core.Context; -import jakarta.ws.rs.core.Request; -import jakarta.ws.rs.core.Response; -import jakarta.ws.rs.core.Response.Status; -import jakarta.ws.rs.core.UriBuilder; -import jakarta.ws.rs.core.UriInfo; -import org.apache.jena.ontapi.model.OntModel; -import org.apache.jena.query.ParameterizedSparqlString; -import org.apache.jena.query.Query; -import org.apache.jena.query.QueryFactory; -import org.apache.jena.query.Syntax; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.ResIterator; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -/** - * JAX-RS resource that generates containers for given classes. - * - * @author {@literal Martynas Jusevičius } - */ -public class Generate -{ - - private static final Logger log = LoggerFactory.getLogger(Generate.class); - - private final UriInfo uriInfo; - private final MediaTypes mediaTypes; - private final Application application; - private final OntModel ontology; - private final Optional agentContext; - private final com.atomgraph.linkeddatahub.Application system; - private final ResourceContext resourceContext; - - /** - * Constructs endpoint for container generation. - * - * @param request current request - * @param uriInfo current URI info - * @param mediaTypes supported media types - * @param application matched application - * @param ontology ontology of the current application - * @param system system application - * @param agentContext authenticated agent's context - * @param resourceContext resource context for creating resources - */ - @Inject - public Generate(@Context Request request, @Context UriInfo uriInfo, MediaTypes mediaTypes, - com.atomgraph.linkeddatahub.apps.model.Application application, Optional ontology, Optional agentContext, - com.atomgraph.linkeddatahub.Application system, @Context ResourceContext resourceContext) - { - if (ontology.isEmpty()) throw new InternalServerErrorException("Ontology is not specified"); - this.uriInfo = uriInfo; - this.mediaTypes = mediaTypes; - this.application = application; - this.ontology = ontology.get(); - this.agentContext = agentContext; - this.system = system; - this.resourceContext = resourceContext; - } - - /** - * Generates containers for given classes. - * Expects a model containing a parent container (sioc:has_parent) and one or more class specifications - * with void:class and spin:query properties. Creates a new container for each class with a view based - * on the provided SPARQL SELECT query. - * - * @param model the RDF model containing the generation parameters - * @return JAX-RS response indicating success or failure - */ - @POST - public Response post(Model model) - { - ResIterator it = model.listSubjectsWithProperty(SIOC.HAS_PARENT); - try - { - if (!it.hasNext()) throw new BadRequestException("Argument resource not provided"); - - Resource arg = it.next(); - Resource service = arg.getPropertyResourceValue(LDH.service); - Resource parent = arg.getPropertyResourceValue(SIOC.HAS_PARENT); - if (parent == null) throw new BadRequestException("Parent container (sioc:has_parent) not provided"); - - ResIterator partIt = model.listSubjectsWithProperty(VoID._class); - try - { - while (partIt.hasNext()) - { - Resource part = partIt.next(); - Resource cls = part.getPropertyResourceValue(VoID._class); - Resource queryRes = part.getPropertyResourceValue(SPIN.query); - if (queryRes == null) throw new BadRequestException("Container query string (spin:query) not provided"); - - // Lookup query in ontology - Resource queryResource = getOntology().getResource(queryRes.getURI()); - if (queryResource == null || !queryResource.hasProperty(SP.text)) - throw new BadRequestException("Query resource not found in ontology: " + queryRes.getURI()); - - String queryString = queryResource.getProperty(SP.text).getString(); - Query query = QueryFactory.create(queryString, Syntax.syntaxARQ); - if (!query.isSelectType()) throw new BadRequestException("Container query is not of SELECT type"); - - ParameterizedSparqlString pss = new ParameterizedSparqlString(query.toString()); - pss.setIri(RDF.type.getLocalName(), cls.getURI()); // inject $type value - - URI containerGraphURI = UriBuilder.fromUri(parent.getURI()).path("{slug}/").build(UUID.randomUUID().toString()); - Model containerModel = ModelFactory.createDefaultModel(); - - createContainer(containerModel, - containerGraphURI, parent, - cls.getLocalName() + "s", - createView(containerModel, createContainerSelect(containerModel, - "Select " + cls.getLocalName(), - pss.asQuery(), - service))); - new Skolemizer(containerGraphURI.toString()).apply(containerModel); - - // append triples directly to the graph store without doing an HTTP request (and thus no ACL check) - try (Response containerResponse = getResourceContext().getResource(DocumentHierarchyGraphStoreImpl.class).post(containerModel, false, containerGraphURI)) - { - if (!containerResponse.getStatusInfo().getFamily().equals(Status.Family.SUCCESSFUL)) - { - if (log.isErrorEnabled()) log.error("Cannot create container"); - throw new InternalServerErrorException("Cannot create container"); - } - } - } - } - finally - { - partIt.close(); - } - - // ban the parent container URI from proxy cache to make sure the next query using it will be fresh (e.g. SELECT that loads children) - getSystem().ban(getSystem().getServiceContext(getApplication().getService()).getBackendProxy(), parent.getURI(), true); - - return Response.ok().build(); - } - finally - { - it.close(); - } - } - - /** - * Creates SELECT SPARQL query. - * - * @param model RDF model - * @param title query title - * @param query query object - * @param service optional SPARQL service resource - * @return query resource - */ - public Resource createContainerSelect(Model model, String title, Query query, Resource service) - { - Resource resource = model.createResource(). - addProperty(RDF.type, SP.Select). - addLiteral(DCTerms.title, title). - addProperty(SP.text, query.toString()); - - if (service != null) resource.addProperty(LDH.service, service); - - return resource; - } - - /** - * Creates a container document. - * - * @param model RDF model - * @param graphURI named graph URI - * @param parent parent document resource - * @param title document title - * @param content document content - * @return container resource - */ - public Resource createContainer(Model model, URI graphURI, Resource parent, String title, Resource content) - { - return model.createResource(graphURI.toString()). - addProperty(RDF.type, DH.Container). - addProperty(SIOC.HAS_PARENT, parent). - addLiteral(DCTerms.title, title). - addLiteral(DH.slug, UUID.randomUUID().toString()). - addLiteral(DCTerms.created, Calendar.getInstance()). - addProperty(model.createProperty(RDF.getURI(), "_1"), content); // TO-DO: make sure we're creating sequence value larger than the existing ones? - } - - /** - * Creates content resource. - * - * @param model RDF model - * @param query query resource - * @return content resource - */ - public Resource createView(Model model, Resource query) - { - return model.createResource(). - addProperty(RDF.type, LDH.View). - addProperty(SPIN.query, query); - } - - /** - * Returns the supported media types. - * - * @return media types - */ - public MediaTypes getMediaTypes() - { - return mediaTypes; - } - - /** - * Returns the current application. - * - * @return the application - */ - public Application getApplication() - { - return application; - } - - /** - * Returns the ontology. - * - * @return the ontology - */ - public OntModel getOntology() - { - return ontology; - } - - /** - * Returns the current URI info. - * - * @return URI info - */ - public UriInfo getUriInfo() - { - return uriInfo; - } - - /** - * Returns the authenticated agent's context. - * - * @return optional agent context - */ - public Optional getAgentContext() - { - return agentContext; - } - - /** - * Returns the system application. - * - * @return system application - */ - public com.atomgraph.linkeddatahub.Application getSystem() - { - return system; - } - - /** - * Returns the resource context. - * - * @return resource context - */ - public ResourceContext getResourceContext() - { - return resourceContext; - } - -} \ No newline at end of file diff --git a/src/main/java/com/atomgraph/linkeddatahub/server/model/impl/Dispatcher.java b/src/main/java/com/atomgraph/linkeddatahub/server/model/impl/Dispatcher.java index 670c8a0d2..bfbc2562b 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/server/model/impl/Dispatcher.java +++ b/src/main/java/com/atomgraph/linkeddatahub/server/model/impl/Dispatcher.java @@ -16,8 +16,6 @@ */ package com.atomgraph.linkeddatahub.server.model.impl; -import com.atomgraph.linkeddatahub.resource.Add; -import com.atomgraph.linkeddatahub.resource.Generate; import com.atomgraph.linkeddatahub.resource.Namespace; import com.atomgraph.linkeddatahub.resource.Transform; import com.atomgraph.linkeddatahub.resource.admin.ClearOntology; @@ -134,17 +132,6 @@ public Class getFileItem() return com.atomgraph.linkeddatahub.resource.upload.Item.class; } - /** - * Returns the endpoint for synchronous RDF imports. - * - * @return endpoint resource - */ - @Path("add") - public Class getAddEndpoint() - { - return Add.class; - } - /** * Returns the endpoint for synchronous RDF imports with a CONSTRUCT query transformation. * @@ -156,17 +143,6 @@ public Class getTransformEndpoint() return Transform.class; } - /** - * Returns the endpoint for container generation. - * - * @return endpoint resource - */ - @Path("generate") - public Class getGenerateEndpoint() - { - return Generate.class; - } - /** * Returns the endpoint that allows clearing ontologies from cache by URI. * diff --git a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/bootstrap/2.3.2/client/modal.xsl b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/bootstrap/2.3.2/client/modal.xsl index 82dc364e4..928169c8a 100644 --- a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/bootstrap/2.3.2/client/modal.xsl +++ b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/bootstrap/2.3.2/client/modal.xsl @@ -40,6 +40,9 @@ xmlns:ldt="&ldt;" xmlns:sd="&sd;" xmlns:sioc="&sioc;" xmlns:dct="&dct;" +xmlns:dh="&dh;" +xmlns:sp="&sp;" +xmlns:spin="&spin;" xmlns:bs2="http://graphity.org/xsl/bootstrap/2.3.2" extension-element-prefixes="ixsl" exclude-result-prefixes="#all" @@ -82,7 +85,7 @@ LIMIT 10 - + @@ -101,7 +104,10 @@ LIMIT 10
-
+ + + + This form uses RDF/POST encoding: https://atomgraph.github.io/RDF-POST/ @@ -200,7 +206,7 @@ LIMIT 10
-

Adding data this way will cause a blocking request, so use it for small amounts of data only (e.g. a few thousands of RDF triples). For larger data, use asynchronous RDF imports.

+

Adding data this way fetches the source through the server's Linked Data proxy and appends it to the target document in a blocking request, so use it for small amounts of data only (e.g. a few thousand RDF triples). For larger data, use asynchronous RDF imports.

@@ -210,9 +216,7 @@ LIMIT 10 - -
@@ -243,16 +247,9 @@ LIMIT 10
-
- This form uses RDF/POST encoding: https://atomgraph.github.io/RDF-POST/ - - - - - + +
- -