diff --git a/docs/integrations/integration-toolkit/configuration.md b/docs/integrations/integration-toolkit/configuration.md index 3efbfc17..197c7060 100644 --- a/docs/integrations/integration-toolkit/configuration.md +++ b/docs/integrations/integration-toolkit/configuration.md @@ -362,7 +362,16 @@ Everything beyond the configuration contract — the polling and acknowledgement "sharing": { "partners": [ { "partner_org_id": "string", "enabled": "boolean | JSONata string" } - ] + ], + "partner_lookups": [ + { + "unique_ids": [{ "attribute": "string", "field": "string" }], + "schema": "string", + "partner_org_id_attribute": "string", + "enabled": "boolean | JSONata string" + } + ], + "exclusive": "boolean" } } ], diff --git a/docs/integrations/integration-toolkit/inbound/partner-sharing.md b/docs/integrations/integration-toolkit/inbound/partner-sharing.md index c4fe502f..05c7262c 100644 --- a/docs/integrations/integration-toolkit/inbound/partner-sharing.md +++ b/docs/integrations/integration-toolkit/inbound/partner-sharing.md @@ -11,8 +11,13 @@ An inbound entity mapping can keep the entity it writes shared with [partner org Sharing is configured per entity mapping, next to `fields`, and kept in sync with every event: 1. The entity is written as usual. -2. It is **shared** with every listed partner whose `enabled` is true. -3. It is **unshared** from every listed partner whose `enabled` is false. +2. It is **shared** with every partner whose `enabled` is true. +3. It is **unshared** from every partner whose `enabled` is false. + +You name the partners in one of two ways, or both: + +- **[Listed partners](#configuration):** the mapping lists each partner organization ID. +- **[Partner lookups](#partner-lookups):** the payload names the partner (e.g. a sales partner number), and the partner organization is looked up in epilot at runtime. Use this when partner organizations are added over time, so the mapping does not change for each new partner. ## Prerequisites @@ -42,9 +47,11 @@ Add a `sharing` block to the entity mapping, with one entry per partner organiza | Property | Required | Description | | --- | --- | --- | -| `partners` | Yes | One entry per partner organization. At least one. | +| `partners` | No | One entry per partner organization. A `sharing` block needs `partners`, [`partner_lookups`](#partner-lookups), or both. | | `partners[].partner_org_id` | Yes | ID of the partner organization. | | `partners[].enabled` | No | `true`, `false`, or a JSONata expression. Evaluated against the payload item, like field-level [`enabled`](./mapping.md). Defaults to `true`. | +| `partner_lookups` | No | Partner organizations looked up at runtime. See [Partner lookups](#partner-lookups). | +| `exclusive` | No | `true` keeps the entity shared with the enabled partners only, and unshares it from every other partner organization. See [Exclusive sharing](#exclusive-sharing). Defaults to `false`. | With the configuration above, this event shares the contact with organization `739224`: @@ -65,9 +72,73 @@ If a later event names `SP-1002`, the contact is unshared from `739224` and shar To share several entities of one event, for example the contact and its contract, add a `sharing` block to each entity mapping. Only the mapped entity itself is shared, not its relations. +## Partner lookups + +Listing partners needs every partner organization ID in the mapping. When partner organizations are created over time, for example one per sales partner, look the organization up instead: + +1. The payload names the partner, e.g. `salesPartner.number`. +2. The lookup finds the entity in epilot that represents this partner: by default a `partner` entity, matched by `unique_ids`. +3. The entity is shared with the organization ID stored on that entity, by default in its `partner_org_id` attribute. + +```json +"sharing": { + "partner_lookups": [ + { + "unique_ids": [ + { "attribute": "sales_partner_number", "field": "salesPartner.number" } + ] + } + ], + "exclusive": true +} +``` + +| Property | Required | Description | +| --- | --- | --- | +| `partner_lookups[].unique_ids` | Yes | How to find the partner entity. Same format as a relation's `unique_ids`: one entry per attribute, each with `field`, `jsonataExpression`, or `constant`. | +| `partner_lookups[].schema` | No | Schema of the partner entity. Defaults to `partner`. | +| `partner_lookups[].partner_org_id_attribute` | No | Attribute of the partner entity that holds the partner organization ID. Defaults to `partner_org_id`. | +| `partner_lookups[].enabled` | No | Like `partners[].enabled`. When false, the entity is unshared from the organization the lookup finds. | + +With this configuration, an event with `"salesPartner": { "number": "SP-1001" }` shares the entity with the organization of the partner entity whose `sales_partner_number` is `SP-1001`. + +:::caution Declare the lookup attribute on the schema +The lookup searches the partner entities, and only attributes declared on the schema are searchable. An attribute that is set on the partner entity but missing from the schema never matches, and every event reports `partner_not_found`. Add the attribute (e.g. `sales_partner_number`) to the `partner` schema before you use it in a lookup. +::: + +What happens when a lookup does not find a partner organization: + +- **The payload names no partner** (the `unique_ids` evaluate to nothing): the item has no partner. The lookup is skipped without a warning. +- **No partner entity matches**, or **the match has no organization ID yet**: the sharing with that partner is left unchanged, and [`SHARING_PARTNER_NOT_PROVISIONED`](#errors-and-monitoring) is reported. `details.reason` is `partner_not_found` or `partner_org_missing`. This is expected while a partner is still being set up. The next event for the entity shares it once the organization ID is set. +- **The organization found has no partnership** with your organization: like a listed partner, it is reported as `SHARING_PARTNER_NOT_CONNECTED`. + +:::note +A lookup only runs when an event arrives. Entities that get no further events after a partner organization is set up are not shared automatically. Share them once when you set up the partner, e.g. by replaying their last event. +::: + +A new partner entity can take a few seconds to become searchable. An event in that window reports `partner_not_found`, and the next event shares the entity. + +## Exclusive sharing + +By default, sharing only touches the partners the mapping names. A partner that is no longer named, e.g. because the payload now has a different sales partner, keeps its access. + +With `"exclusive": true`, the enabled partners are the only partner organizations the entity stays shared with. Every other partner organization the entity is shared with is unshared. Your own organization always keeps its access. Use it when the payload decides the one partner an entity belongs to: + +- When the sales partner changes, the entity moves to the new partner. +- When the payload no longer names a sales partner, the entity is unshared from the old one. + +:::caution +Exclusive sharing also removes shares made outside this mapping, for example by a user in epilot or by another mapping. +::: + +With `exclusive`, a lookup that finds no partner organization (`SHARING_PARTNER_NOT_PROVISIONED`) still counts as "no partner". The entity is unshared from its current partner even though the new one is not set up yet. It is shared with the new partner on the first event after that partner's organization ID is set. The old partner must not keep access to a record that now belongs to someone else, so the gap is intended. It is reported as `SHARING_PARTNER_NOT_PROVISIONED`. + +Because of this, a misconfigured lookup (for example an attribute that is not declared on the partner schema) unshares every entity the mapping writes. Test a new lookup without `exclusive` first, and turn `exclusive` on once events stop reporting `partner_not_found`. A failed partner search is retried and never unshares. + ## How it works - **Only changes are sent.** The target state is compared with who the entity is currently shared with. A repeated event with the same outcome makes no sharing call, and partners get no duplicate notification. +- **Sharing becomes visible on the entity after a few seconds.** Who an entity is shared with is updated in the background after each share. An event for the same entity within those seconds still sees the previous state. It may share again with a partner that already has access (no harm done), and `exclusive` may not unshare yet. The next event catches up. - **Runs on unchanged entities too.** Sharing is checked even when the entity's attributes did not change. A changed `enabled` result therefore applies on the next event. - **Upserts only.** Sharing is skipped for the `delete` and `purge` modes. - **The entity is written first.** A sharing problem never undoes the entity write. @@ -78,11 +149,12 @@ Each case has its own monitoring code, so you can filter on it in the Monitoring | Situation | Behaviour | Code | | --- | --- | --- | -| No partnership with a listed partner organization | That partner is skipped, the others are synced. Not retried. | `SHARING_PARTNER_NOT_CONNECTED` (warning) | +| No partnership with a partner organization (listed or looked up) | That partner is skipped, the others are synced. Not retried. | `SHARING_PARTNER_NOT_CONNECTED` (warning) | +| A partner lookup finds no partner entity, or the entity has no organization ID yet | That lookup is skipped, the others are synced. Not retried. | `SHARING_PARTNER_NOT_PROVISIONED` (warning) | | The sharing service rejects the request | Nothing is synced for this entity. Not retried. | `SHARING_REJECTED` (warning) | -| Temporary failure (service unavailable, timeout, throttling) | The entity update is retried automatically. | `SHARING_FAILED` (error), once all retries are used up | +| Temporary failure (service unavailable, timeout, throttling), also of a partner lookup | The entity update is retried automatically. | `SHARING_FAILED` (error), once all retries are used up | -Warning events carry the affected `partner_org_ids` in their details. When sharing changed, the entity's success event carries `sharing.shared` and `sharing.unshared` with the partner organization IDs. +Warning events carry the affected `partner_org_ids` in their details. `SHARING_PARTNER_NOT_PROVISIONED` carries `reason` (`partner_not_found` or `partner_org_missing`), the `unique_ids` that were searched, and `partner_entity_id` when a partner entity was found. When sharing changed, the entity's success event carries `sharing.shared` and `sharing.unshared` with the partner organization IDs. See [Monitoring Codes](../monitoring/codes.md) for the full list. @@ -95,8 +167,17 @@ The mapping simulation in the Integration Hub's test view returns `sharing` on e "partners": [ { "partner_org_id": "739224", "enabled": true }, { "partner_org_id": "739225", "enabled": false } - ] + ], + "lookups": [ + { + "schema": "partner", + "unique_ids": { "sales_partner_number": "SP-1001" }, + "partner_org_id_attribute": "partner_org_id", + "enabled": true + } + ], + "exclusive": true } ``` -The simulation does not call the sharing service. It shows the target state, not the changes that would be sent. +The simulation does not call the sharing service. It shows the target state, not the changes that would be sent. Lookups are shown with their `unique_ids` evaluated, but the simulation does not search for the partner entity. The organization is only looked up when a real event is processed. diff --git a/docs/integrations/integration-toolkit/monitoring/codes.md b/docs/integrations/integration-toolkit/monitoring/codes.md index 98d189ef..7826bc3e 100644 --- a/docs/integrations/integration-toolkit/monitoring/codes.md +++ b/docs/integrations/integration-toolkit/monitoring/codes.md @@ -15,7 +15,7 @@ Every monitoring event carries a **code** and a **level**. The code says what happened; the level says how much you should care. Both are filterable in the Integration Hub's [Monitoring tab](./overview.md) and through the events API. -There are 102 codes. You are most likely here because you saw one in a failed +There are 118 codes. You are most likely here because you saw one in a failed event — find it below. :::tip @@ -42,6 +42,8 @@ Something failed and the event did not do what it was meant to do. These are wha | `DIRECT_PAYLOAD_INVALID` | The direct mode payload failed validation against the versioned payload schema | | `DIRECT_VERSION_UNSUPPORTED` | The direct mode payload version is not supported by the platform | | `ENTITY_REFERENCE_NOT_FOUND` | A direct-by-id entity reference points at an entity that does not exist or has a different schema | +| `ENV_VAR_REF_JSONATA_FAILED` | The JSONata expression of an env_var_ref field failed to evaluate, so the field was skipped | +| `ENV_VAR_REF_LOOKUP_FAILED` | The environment variables could not be looked up for an env_var_ref field, so the field was skipped | | `EVENT_NOT_CONFIGURED` | No mapping configuration found for this event type | | `EXTERNAL_API_ERROR` | An external API returned an error | | `EXTERNAL_ERROR` | An error span pushed by an external system (e.g. your integration middleware) via the external monitoring events endpoint. epilot assigns the code from the span's level; the middleware never sends one. | @@ -57,6 +59,8 @@ Something failed and the event did not do what it was meant to do. These are wha | `METERING_API_ERROR` | The metering API returned an error | | `METER_READING_GROUP_FAILED` | A batch write of meter readings failed permanently after all retries — summary event alongside the per-reading errors | | `METER_READING_GROUP_RETRYING` | A batch write of meter readings failed and will be retried automatically — one event per attempt covering the whole group (reading_count and external_ids in details) | +| `MISSING_METER_COUNTER_UNIQUE_IDS` | A meter reading was skipped because the event did not contain the unique IDs configured to identify its meter counter | +| `MISSING_METER_UNIQUE_IDS` | A meter reading was skipped because the event did not contain the unique IDs configured to identify its meter | | `MISSING_REQUIRED_PARAM` | A required parameter is missing from the request | | `MISSING_UNIQUE_IDENTIFIERS` | The event is missing the unique identifier field(s) required to match an entity | | `MSG_DEAD_LETTERED` | Outbound message moved to the dead-letter queue after exhausting delivery attempts, or via an operator skip | @@ -64,6 +68,8 @@ Something failed and the event did not do what it was meant to do. These are wha | `MSG_HEAD_BLOCKED` | Outbound stream halted by a poison head message (block policy) — requires operator unblock or consumer acknowledgement | | `OAUTH2_TOKEN_FAILURE` | Failed to obtain an OAuth2 access token | | `PAYLOAD_TOO_LARGE` | The payload exceeded the maximum size accepted by the receiving system | +| `PORTAL_REF_JSONATA_FAILED` | The JSONata expression of a portal_ref field failed to evaluate, so the field was skipped | +| `PORTAL_REF_LOOKUP_FAILED` | The portals could not be looked up for a portal_ref field, so the field was skipped | | `PROCESSING_STATE_LIMIT` | The processing record of an interactive event could not be written, because it or its correlation reached the size the store allows, so the event was ended as an error instead of being left processing: its correlation still completes, and anyone waiting on it is answered. The event's data may have been written all the same. Usually a payload with unusually many entities or very long unique ids | | `PRUNE_SCOPE_PARTIAL_FAILURE` | Scope pruning completed with some failures | | `RECURSION_DEPTH_EXCEEDED` | Maximum recursion depth was exceeded during processing | @@ -94,6 +100,7 @@ Something failed and the event did not do what it was meant to do. These are wha | `USE_CASE_INVALID_TYPE` | The use case type is invalid or unsupported | | `USE_CASE_MISSING_CONFIG` | The use case is missing required configuration | | `USE_CASE_NOT_FOUND` | The use case could not be found | +| `WEBHOOK_DELIVERY_FAILED` | The webhook could not be delivered and the target returned no HTTP status (for example a connection or timeout error). The webhook service's own error code is in the event detail | ## Warning @@ -109,15 +116,21 @@ Processing continued, but something needs a human eye — often a retry in fligh | `CORRELATION_LATE_EVENT` | An event arrived under a correlation the partner had already declared complete. It was processed, but anyone waiting on that correlation had already been answered, and once the correlation is complete the event cannot change that answer — the partner is sending `correlation_complete` too early | | `CORRELATION_WAIT_REJECTED` | A caller tried to wait on a correlation while too many waits were already open, so it was asked to retry. Its data is unaffected — only the answer it was waiting for is slower | | `CORRELATION_WAIT_TIMEOUT` | A caller waited on a correlation until its time ran out without getting what it asked for — see `missing_entities` and `missing_use_cases` for what had not arrived. Reported once per correlation. When nothing at all was sent under the correlation, it carries no integration: none is known before the first event | +| `ENV_VAR_REF_NOT_FOUND` | The environment variable an env_var_ref field points at does not exist, so the field was left unset | | `EXTERNAL_WARNING` | A warning span pushed by an external system via the external monitoring events endpoint. | | `FILE_PROXY_UPLOAD_RETRYING` | A file upload failed with a retryable error and will be retried automatically — one event per attempt | | `INTERACTIVE_CROSS_LANE_WRITE` | The same entity was updated from two different processing lanes within seconds of each other, one of them the interactive lane. Both updates were applied; this reports that they overlapped, so the order they landed in was decided by timing | | `INTERACTIVE_RATE_LIMITED` | The organization asked for interactive processing more often than its agreed rate, so these events were processed on its normal lane instead. Nothing was lost and the correlation can still be waited on — it will simply take as long as a bulk event does | | `LOOKUP_UNMAPPED` | A value was not listed in a lookup table and its fallback was used — see lookup_name and lookup_key for the gap | | `MSG_LATE_ARRIVAL` | An event arrived after the poll consumer had already received later events, so it was placed at the end of the stream instead of at its event time — it is delivered, but out of event-time order | +| `PORTAL_REF_AMBIGUOUS` | Several portals matched the portal_ref filter and the oldest one was used. Narrow the filter so it matches exactly one portal | +| `PORTAL_REF_NO_MATCH` | No portal matched the portal_ref filter, so the field was left unset. Check the filter against the portals in this organization | +| `RELATION_MISSING_UNIQUE_IDS` | A relation was skipped because the event did not contain all unique IDs of the relation's target. The entity was still written, without that relation — check that the mapping sends every unique ID the relation is configured with | | `SHARING_PARTNER_NOT_CONNECTED` | The entity was written but not shared with the partner orgs in partner_org_ids: this organization has no partnership with them. Set up the partnership, or fix the partner_org_id in the mapping | +| `SHARING_PARTNER_NOT_PROVISIONED` | The entity was written but a partner lookup in the mapping found no partner org to share it with: no entity matched details.unique_ids, or the match has no partner org id yet (details.reason). Expected while a partner org is still being set up: the next event for the entity shares it once the partner org id is set | | `SHARING_REJECTED` | The entity was written but the sharing service rejected sharing it with its partner orgs (details.status). Check the sharing configuration of the partners in partner_org_ids | | `SOFT_DELETED_ENTITY_MATCHED` | A soft-deleted entity matched the unique ID — it will be resurrected on upsert, or referenced as-is by a relation. Investigate why the ERP source is sending events for a deleted entity. | +| `UNIQUE_ID_TYPE_DERIVATION_FAILED` | Direct mode could not derive a unique ID's type from the entity schema, so the value was used exactly as sent | ## Success @@ -138,6 +151,7 @@ The event did what it was meant to do. Useful for confirming a sync actually lan | `METER_READING_UPSERTED` | One or more meter readings were created or updated — emitted once per batch, not per reading (reading_count and external_ids in details) | | `MSG_ACKED` | Outbound message delivered: the polling consumer acknowledged it and it was removed from the queue | | `PRUNE_SCOPE_COMPLETED` | Scope pruning completed successfully | +| `SECURE_PROXY_OK` | The secure proxy reached the target and returned its answer | | `WEBHOOK_DELIVERED` | Webhook was delivered successfully | ## Info @@ -155,6 +169,8 @@ Lifecycle markers rather than outcomes: a message was queued, a duplicate was ig | `FILE_PROXY_UPLOAD_ENQUEUED` | A per-file upload was accepted for delivery during fan-out | | `INTERACTIVE_NOT_ALLOWED` | Interactive processing was requested for a use case that is not configured for it, so the events were processed normally. Set `interactive` to `on_request` or `always` on the use case if the caller is meant to be able to wait for these events | | `MSG_ENQUEUED` | Outbound message enqueued to the poll queue, awaiting consumption by the ERP. When the use case maps the payload, the detail carries the mapped payload as delivered and its mapping_version (or payload_omitted when it exceeds 48 KiB) | +| `ORG_DROPPED` | Events from this organization are configured to be dropped, so the event was consumed without being processed | +| `WEBHOOK_SKIPPED` | The webhook was not delivered because its filters did not match the event | ## Status-code families Some codes are generated from an upstream response rather than drawn from the fixed list above.