Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ RocksDB is utilized in the following Ozone components to store critical metadata
- **Object Store Layout:** `volumeTable`, `bucketTable`, `keyTable` (for object store layout), `openKeyTable` (for tracking open keys during multipart uploads), and `multipartInfoTable` (for storing multipart upload information).
- **File System Layout:** `directoryTable` and `fileTable` (for file system layout), `openFileTable` (for tracking open files), and `deletedDirectoryTable` (for tracking deleted directories).
- **Access Control:** `prefixTable` (for storing prefix-based access control information).
- **Security:** `userTable`, `dTokenTable` (delegation tokens), and `s3SecretTable`.
- **Security:** `userTable`, `dTokenTable` (delegation tokens), `s3SecretTable`, and `s3RevokedStsTokenTable` (revoked STS session tokens).
- **S3 Multi-Tenancy:** `tenantStateTable` (for storing tenant state information), `tenantAccessIdTable` (for storing access ID information), and `principalToAccessIdsTable` (for mapping user principals to access IDs).
- **State Management:**
- `transactionInfoTable` for tracking transactions.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -72,14 +72,15 @@ Snapshot metadata and bookkeeping for snapshot-related garbage collection.

### 6. Multi-tenant and security

Tenants, access mappings, S3 secrets, and delegation tokens.
Tenants, access mappings, S3 secrets, delegation tokens, and revoked STS session tokens.

| Table name | Key format | Value type | Description |
| ---------- | ---------- | ---------- | ----------- |
| `tenantStateTable` | `tenantId` | `OmDBTenantState` | Tenant configuration and state |
| `tenantAccessIdTable` | `accessId` | `OmDBAccessIdInfo` | Access ID to secret and tenant |
| `principalToAccessIdsTable` | `userPrincipal` | `OmDBUserPrincipalInfo` | Kerberos principal to access IDs |
| `s3SecretTable` | `accessKeyId` | `S3SecretValue` | S3 secrets |
| `s3RevokedStsTokenTable` | `sts_session_token` | `Long` | Revoked STS session tokens (insertion time in milliseconds) |
| `dTokenTable` | `OzoneTokenID` | `Long` | Delegation tokens and renewal times |

### 7. Administrative and system
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
# HDDS-13323: Ozone S3 Security Token Service (STS)

Epic: [HDDS-13323](https://issues.apache.org/jira/browse/HDDS-13323)
Feature branch: https://github.com/apache/ozone/tree/HDDS-13323-sts

## 1. Builds/intermittent test failures

There are no intermittent failures specific to the HDDS-13323-sts branch as of now. During development, it was ensured that all CI checks were clean prior to every commit merge.

The plan is to run repeated CI checks on the merge commit to master.

## 2. Documentation

[User documentation](https://ozone.apache.org/docs/next/administrator-guide/operations/s3/sts) for Ozone S3 Security Token Service (STS) has been added.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, it should be accessible once #510 is merged


## 3. Design, attached the docs

Design document can be found here: [AWS STS Design for Ozone S3](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-hdds/docs/content/design/ozone-sts.md).

## 4. S3 compatibility

See sections [3.2](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-hdds/docs/content/design/ozone-sts.md#32-limitations-in-assumerole-api-support) and [3.3](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-hdds/docs/content/design/ozone-sts.md#33-limitations-in-iam-session-policy-support) in the design document for limitations in the AssumeRole API compatibility.

Also **please note** that the Ranger feature flag (along with the Ozone feature flag) must be enabled in order to get AWS compatibility in terms of the requested actions and resulting actions the STS tokens have.

## 5. Docker-compose / Acceptance tests

New robot tests [`ozone-secure-sts.robot`](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-ozone/dist/src/main/smoketest/security/ozone-secure-sts.robot) and [`ozone-secure-sts-multitenant.robot`](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-ozone/dist/src/main/smoketest/security/ozone-secure-sts-multitenant.robot) are being added. Furthermore, a shell script to test Polaris with Ozone STS ([`polaris-smoketest.sh`](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-ozone/dist/src/main/compose/ozonesecure-ha/polaris-smoketest.sh)) is being added.

## 6. Support of containers / Kubernetes

No addition. No change in existing support.

## 7. Coverage / Code quality

[New Code Coverage](https://sonarcloud.io/summary/new_code?id=hadoop-ozone&branch=HDDS-13323-sts) for STS (HDDS-13323-sts) is **81.58%** and [Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=HDDS-13323-sts) is **75.4%**.
[Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=master) for master is **78.6%**.

## 8. Build time

[Build time for the latest commit](https://github.com/apache/ozone/actions/runs/30604883966/job/91075071143) from HDDS-13323-sts branch is **11m 32s**.
[Build time for the latest commit](https://github.com/apache/ozone/actions/runs/30662032542/job/91260363408) from the master branch is **11m 48s**.

## 9. Possible incompatible changes/used feature flag

STS is disabled by default and gated by configuration flags. **Both** flags below must be enabled for correct, fine-grained authorization behavior.

### Ozone feature flag

Enable in `ozone-site.xml`:

```xml
<property>
<name>ozone.s3g.sts.http.enabled</name>
<value>true</value>
<description>Enable the Ozone S3 Gateway STS endpoint.</description>
</property>
```

### Ranger feature flag (action-matches Policy Condition)

Enable in `ranger-admin-site.xml`:

```xml
<property>
<name>ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition</name>
<value>true</value>
</property>
```

A new OM RocksDB table, `s3RevokedStsTokenTable` (`sts_session_token` → `insertionTimeMillis`), stores revoked STS token entries. No OM versioning-framework feature flag is used.

## 10. Third-party dependencies/License changes

There are no third party dependencies introduced by this feature.

## 11. Performance

The AssumeRole invocations are write requests and therefore the throughput is shared with other OM write calls.

## 12. Security considerations

STS introduces a new network-accessible endpoint (ports 9880/9881) for AssumeRole. Key security properties:

- **Temporary credentials only**: AssumeRole requires permanent S3 credentials (from `ozone s3 getsecret`) of a Kerberos identity to create the temporary credentials. Temporary credentials expire between 15 minutes and 12 hours.
- **Ranger-only authorization**: STS requires Apache Ranger with role-based policies and optional inline IAM session policies.
- **Fail-closed revocation checks**: If the revocation database is unavailable, token validation fails rather than allowing access.

### Token revocation

Ozone adds explicit revocation support:

**Revoke a single session token** — only the token creator or an S3/tenant admin may revoke. This adds the token to the OM RocksDB `s3RevokedStsTokenTable`. Subsequent S3 requests using that session token are denied immediately, even before the token's natural expiration.

**Revoke all tokens for a user** — revoking a user's permanent S3 secret also invalidates all outstanding STS tokens created by that user. This is useful when a long-lived credential is compromised and all derived temporary credentials must be invalidated at once.