From 0dca3510f26ced47222145c08d7fc901314fe0f5 Mon Sep 17 00:00:00 2001 From: Fabian Morgan Date: Tue, 4 Aug 2026 01:00:41 -0700 Subject: [PATCH 1/6] sts branch merge checklist --- .../03-merged-branches/18-hdds-13323-sts.md | 95 +++++++++++++++++++ 1 file changed, 95 insertions(+) create mode 100644 docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md diff --git a/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md b/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md new file mode 100644 index 0000000000..fdd96ee1be --- /dev/null +++ b/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md @@ -0,0 +1,95 @@ +# HDDS-13323: Ozone S3 Security Token Service (STS) + +Epic: [HDDS-13323](https://issues.apache.org/jira/browse/HDDS-13323) +Feature branch: https://github.com/apache/ozone/tree/HDDS-13323-sts + +## 1. Builds/intermittent test failures + +There are no intermittent failures specific to the HDDS-13323-sts branch as of now. During development, it was ensured that all CI checks were clean prior to every commit merge. + +The plan is to run repeated CI checks on the merge commit to master. + +## 2. Documentation + +[User documentation](https://ozone.apache.org/docs/next/administrator-guide/operations/s3/sts) for Ozone S3 Security Token Service (STS) has been added. + +## 3. Design, attached the docs + +Design document can be found here: [AWS STS Design for Ozone S3](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-hdds/docs/content/design/ozone-sts.md). + +## 4. S3 compatibility + +See sections [3.2](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-hdds/docs/content/design/ozone-sts.md#32-limitations-in-assumerole-api-support) and [3.3](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-hdds/docs/content/design/ozone-sts.md#33-limitations-in-iam-session-policy-support) in the design document for limitations in the AssumeRole API compatibility. + +Also **please note** that the Ranger feature flag (along with the Ozone feature flag) must be enabled in order to get AWS compatibility in terms of the requested actions and resulting actions the STS tokens have. + +## 5. Docker-compose / Acceptance tests + +New robot tests `ozone-secure-sts.robot` and `ozone-secure-sts-multitenant.robot` are being added. Furthermore, a shell script to test Polaris with Ozone STS (`polaris-smoketest.sh`) is being added. + +## 6. Support of containers / Kubernetes + +No addition. No change in existing support. + +## 7. Coverage / Code quality + +[New Code Coverage](https://sonarcloud.io/summary/new_code?id=hadoop-ozone&branch=HDDS-13323-sts) for STS (HDDS-13323-sts) is **89.1%** and [Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=HDDS-13323-sts) is **75.4%**. +[Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=master) for master is **78.5%**. + +## 8. Build time + +[Build time for the latest commit](https://github.com/apache/ozone/actions/runs/30604883966/job/91075071143) from HDDS-13323-sts branch is **11m 32s**. +[Build time for the latest commit](https://github.com/apache/ozone/actions/runs/30662032542/job/91260363408) from the master branch is **11m 48s**. + +## 9. Possible incompatible changes/used feature flag + +STS is disabled by default and gated by configuration flags. **Both** flags below must be enabled for correct, fine-grained authorization behavior. + +### Ozone feature flag + +Enable in `ozone-site.xml`: + +```xml + + ozone.s3g.sts.http.enabled + true + Enable the Ozone S3 Gateway STS endpoint. + +``` + +### Ranger feature flag (action-matches Policy Condition) + +Enable in `ranger-admin-site.xml`: + +```xml + + ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition + true + +``` + +A new RocksDB table stores revoked STS token entries. No OM versioning-framework feature flag is used. + +## 10. Third-party dependencies/License changes + +There are no third party dependencies introduced by this feature. + +## 11. Performance + +The AssumeRole invocations are write requests and therefore the throughput is shared with other OM write calls. + +## 12. Security considerations + +STS introduces a new network-accessible endpoint (ports 9880/9881) for AssumeRole. Key security properties: + +- **Temporary credentials only**: AssumeRole requires permanent S3 credentials (from `ozone s3 getsecret`) of a Kerberos identity to create the temporary credentials. Temporary credentials expire between 15 minutes and 12 hours. +- **Ranger-only authorization**: STS requires Apache Ranger with role-based policies and optional inline IAM session policies. +- **Fail-closed revocation checks**: If the revocation database is unavailable, token validation fails rather than allowing access. + +### Token revocation + +Ozone adds explicit revocation support: + +**Revoke a single session token** — only the token creator or an S3/tenant admin may revoke. This adds the token to an OM RocksDB revoked-token table. Subsequent S3 requests using that session token are denied immediately, even before the token's natural expiration. + +**Revoke all tokens for a user** — revoking a user's permanent S3 secret also invalidates all outstanding STS tokens created by that user. This is useful when a long-lived credential is compromised and all derived temporary credentials must be invalidated at once. From b23111a8c6b7c88de74db65e3848310a0f434096 Mon Sep 17 00:00:00 2001 From: Wei-Chiu Chuang Date: Tue, 4 Aug 2026 20:14:44 -0700 Subject: [PATCH 2/6] HDDS-16078. Fix markdownlint trailing spaces in STS merge checklist. Co-authored-by: Cursor --- .../03-merged-branches/18-hdds-13323-sts.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md b/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md index fdd96ee1be..9c2aaae70c 100644 --- a/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md +++ b/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md @@ -5,13 +5,13 @@ Feature branch: https://github.com/apache/ozone/tree/HDDS-13323-sts ## 1. Builds/intermittent test failures -There are no intermittent failures specific to the HDDS-13323-sts branch as of now. During development, it was ensured that all CI checks were clean prior to every commit merge. +There are no intermittent failures specific to the HDDS-13323-sts branch as of now. During development, it was ensured that all CI checks were clean prior to every commit merge. The plan is to run repeated CI checks on the merge commit to master. ## 2. Documentation -[User documentation](https://ozone.apache.org/docs/next/administrator-guide/operations/s3/sts) for Ozone S3 Security Token Service (STS) has been added. +[User documentation](https://ozone.apache.org/docs/next/administrator-guide/operations/s3/sts) for Ozone S3 Security Token Service (STS) has been added. ## 3. Design, attached the docs @@ -83,7 +83,7 @@ The AssumeRole invocations are write requests and therefore the throughput is sh STS introduces a new network-accessible endpoint (ports 9880/9881) for AssumeRole. Key security properties: - **Temporary credentials only**: AssumeRole requires permanent S3 credentials (from `ozone s3 getsecret`) of a Kerberos identity to create the temporary credentials. Temporary credentials expire between 15 minutes and 12 hours. -- **Ranger-only authorization**: STS requires Apache Ranger with role-based policies and optional inline IAM session policies. +- **Ranger-only authorization**: STS requires Apache Ranger with role-based policies and optional inline IAM session policies. - **Fail-closed revocation checks**: If the revocation database is unavailable, token validation fails rather than allowing access. ### Token revocation From 5340ec8ae7ca0991355a5f16eb25986c93729ca0 Mon Sep 17 00:00:00 2001 From: Fabian Morgan Date: Wed, 5 Aug 2026 20:06:23 -0700 Subject: [PATCH 3/6] pr review update for ChenSammi --- .../02-configuration/04-performance/04-rocksdb.md | 2 +- .../01-components/01-ozone-manager/02-rocksdb-schema.md | 3 ++- .../03-merged-branches/18-hdds-13323-sts.md | 4 ++-- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/05-administrator-guide/02-configuration/04-performance/04-rocksdb.md b/docs/05-administrator-guide/02-configuration/04-performance/04-rocksdb.md index 0b51e3036e..c5336f3436 100644 --- a/docs/05-administrator-guide/02-configuration/04-performance/04-rocksdb.md +++ b/docs/05-administrator-guide/02-configuration/04-performance/04-rocksdb.md @@ -25,7 +25,7 @@ RocksDB is utilized in the following Ozone components to store critical metadata - **Object Store Layout:** `volumeTable`, `bucketTable`, `keyTable` (for object store layout), `openKeyTable` (for tracking open keys during multipart uploads), and `multipartInfoTable` (for storing multipart upload information). - **File System Layout:** `directoryTable` and `fileTable` (for file system layout), `openFileTable` (for tracking open files), and `deletedDirectoryTable` (for tracking deleted directories). - **Access Control:** `prefixTable` (for storing prefix-based access control information). -- **Security:** `userTable`, `dTokenTable` (delegation tokens), and `s3SecretTable`. +- **Security:** `userTable`, `dTokenTable` (delegation tokens), `s3SecretTable`, and `s3RevokedStsTokenTable` (revoked STS session tokens). - **S3 Multi-Tenancy:** `tenantStateTable` (for storing tenant state information), `tenantAccessIdTable` (for storing access ID information), and `principalToAccessIdsTable` (for mapping user principals to access IDs). - **State Management:** - `transactionInfoTable` for tracking transactions. diff --git a/docs/07-system-internals/01-components/01-ozone-manager/02-rocksdb-schema.md b/docs/07-system-internals/01-components/01-ozone-manager/02-rocksdb-schema.md index 9f83af5559..89c97f50f8 100644 --- a/docs/07-system-internals/01-components/01-ozone-manager/02-rocksdb-schema.md +++ b/docs/07-system-internals/01-components/01-ozone-manager/02-rocksdb-schema.md @@ -72,7 +72,7 @@ Snapshot metadata and bookkeeping for snapshot-related garbage collection. ### 6. Multi-tenant and security -Tenants, access mappings, S3 secrets, and delegation tokens. +Tenants, access mappings, S3 secrets, delegation tokens, and revoked STS session tokens. | Table name | Key format | Value type | Description | | ---------- | ---------- | ---------- | ----------- | @@ -80,6 +80,7 @@ Tenants, access mappings, S3 secrets, and delegation tokens. | `tenantAccessIdTable` | `accessId` | `OmDBAccessIdInfo` | Access ID to secret and tenant | | `principalToAccessIdsTable` | `userPrincipal` | `OmDBUserPrincipalInfo` | Kerberos principal to access IDs | | `s3SecretTable` | `accessKeyId` | `S3SecretValue` | S3 secrets | +| `s3RevokedStsTokenTable` | `sts_session_token` | `Long` | Revoked STS session tokens (insertion time in millis) | | `dTokenTable` | `OzoneTokenID` | `Long` | Delegation tokens and renewal times | ### 7. Administrative and system diff --git a/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md b/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md index 9c2aaae70c..64a250655f 100644 --- a/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md +++ b/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md @@ -68,7 +68,7 @@ Enable in `ranger-admin-site.xml`: ``` -A new RocksDB table stores revoked STS token entries. No OM versioning-framework feature flag is used. +A new OM RocksDB table, `s3RevokedStsTokenTable` (`sts_session_token` → `insertionTimeMillis`), stores revoked STS token entries. No OM versioning-framework feature flag is used. ## 10. Third-party dependencies/License changes @@ -90,6 +90,6 @@ STS introduces a new network-accessible endpoint (ports 9880/9881) for AssumeRol Ozone adds explicit revocation support: -**Revoke a single session token** — only the token creator or an S3/tenant admin may revoke. This adds the token to an OM RocksDB revoked-token table. Subsequent S3 requests using that session token are denied immediately, even before the token's natural expiration. +**Revoke a single session token** — only the token creator or an S3/tenant admin may revoke. This adds the token to the OM RocksDB `s3RevokedStsTokenTable`. Subsequent S3 requests using that session token are denied immediately, even before the token's natural expiration. **Revoke all tokens for a user** — revoking a user's permanent S3 secret also invalidates all outstanding STS tokens created by that user. This is useful when a long-lived credential is compromised and all derived temporary credentials must be invalidated at once. From b6e83acf09280f3211230969a4cf00e7400d6a33 Mon Sep 17 00:00:00 2001 From: Wei-Chiu Chuang Date: Thu, 6 Aug 2026 07:56:59 -0700 Subject: [PATCH 4/6] HDDS-16078. Fix spelling check for STS RocksDB table description. Use "milliseconds" instead of "millis" in the s3RevokedStsTokenTable row. Co-authored-by: Cursor --- .../01-components/01-ozone-manager/02-rocksdb-schema.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/07-system-internals/01-components/01-ozone-manager/02-rocksdb-schema.md b/docs/07-system-internals/01-components/01-ozone-manager/02-rocksdb-schema.md index 89c97f50f8..c801785727 100644 --- a/docs/07-system-internals/01-components/01-ozone-manager/02-rocksdb-schema.md +++ b/docs/07-system-internals/01-components/01-ozone-manager/02-rocksdb-schema.md @@ -80,7 +80,7 @@ Tenants, access mappings, S3 secrets, delegation tokens, and revoked STS session | `tenantAccessIdTable` | `accessId` | `OmDBAccessIdInfo` | Access ID to secret and tenant | | `principalToAccessIdsTable` | `userPrincipal` | `OmDBUserPrincipalInfo` | Kerberos principal to access IDs | | `s3SecretTable` | `accessKeyId` | `S3SecretValue` | S3 secrets | -| `s3RevokedStsTokenTable` | `sts_session_token` | `Long` | Revoked STS session tokens (insertion time in millis) | +| `s3RevokedStsTokenTable` | `sts_session_token` | `Long` | Revoked STS session tokens (insertion time in milliseconds) | | `dTokenTable` | `OzoneTokenID` | `Long` | Delegation tokens and renewal times | ### 7. Administrative and system From 2585691635e7b5facbb26b7f4cd2f9b54bf510e5 Mon Sep 17 00:00:00 2001 From: Fabian Morgan Date: Tue, 18 Aug 2026 00:49:30 -0700 Subject: [PATCH 5/6] add links to smoke tests --- .../03-feature-branches/03-merged-branches/18-hdds-13323-sts.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md b/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md index 64a250655f..148836e56a 100644 --- a/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md +++ b/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md @@ -25,7 +25,7 @@ Also **please note** that the Ranger feature flag (along with the Ozone feature ## 5. Docker-compose / Acceptance tests -New robot tests `ozone-secure-sts.robot` and `ozone-secure-sts-multitenant.robot` are being added. Furthermore, a shell script to test Polaris with Ozone STS (`polaris-smoketest.sh`) is being added. +New robot tests [`ozone-secure-sts.robot`](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-ozone/dist/src/main/smoketest/security/ozone-secure-sts.robot) and [`ozone-secure-sts-multitenant.robot`](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-ozone/dist/src/main/smoketest/security/ozone-secure-sts-multitenant.robot) are being added. Furthermore, a shell script to test Polaris with Ozone STS ([`polaris-smoketest.sh`](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-ozone/dist/src/main/compose/ozonesecure-ha/polaris-smoketest.sh)) is being added. ## 6. Support of containers / Kubernetes From 83479652521d52dc1be5c027918c2bd8fe1b117b Mon Sep 17 00:00:00 2001 From: Fabian Morgan Date: Wed, 19 Aug 2026 15:53:45 -0700 Subject: [PATCH 6/6] update code coverage numbers --- .../03-merged-branches/18-hdds-13323-sts.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md b/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md index 148836e56a..6c89efeaae 100644 --- a/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md +++ b/docs/08-developer-guide/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md @@ -33,8 +33,8 @@ No addition. No change in existing support. ## 7. Coverage / Code quality -[New Code Coverage](https://sonarcloud.io/summary/new_code?id=hadoop-ozone&branch=HDDS-13323-sts) for STS (HDDS-13323-sts) is **89.1%** and [Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=HDDS-13323-sts) is **75.4%**. -[Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=master) for master is **78.5%**. +[New Code Coverage](https://sonarcloud.io/summary/new_code?id=hadoop-ozone&branch=HDDS-13323-sts) for STS (HDDS-13323-sts) is **81.58%** and [Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=HDDS-13323-sts) is **75.4%**. +[Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=master) for master is **78.6%**. ## 8. Build time