-
Notifications
You must be signed in to change notification settings - Fork 78
HDDS-16078. [STS] Branch merge checklist #523
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
fmorg-git
wants to merge
6
commits into
apache:master
Choose a base branch
from
fmorg-git:HDDS-16078
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+98
−2
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
0dca351
sts branch merge checklist
b23111a
HDDS-16078. Fix markdownlint trailing spaces in STS merge checklist.
jojochuang 5340ec8
pr review update for ChenSammi
b6e83ac
HDDS-16078. Fix spelling check for STS RocksDB table description.
jojochuang 2585691
add links to smoke tests
8347965
update code coverage numbers
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
95 changes: 95 additions & 0 deletions
95
...e/04-project/01-git/03-feature-branches/03-merged-branches/18-hdds-13323-sts.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,95 @@ | ||
| # HDDS-13323: Ozone S3 Security Token Service (STS) | ||
|
|
||
| Epic: [HDDS-13323](https://issues.apache.org/jira/browse/HDDS-13323) | ||
| Feature branch: https://github.com/apache/ozone/tree/HDDS-13323-sts | ||
|
|
||
| ## 1. Builds/intermittent test failures | ||
|
|
||
| There are no intermittent failures specific to the HDDS-13323-sts branch as of now. During development, it was ensured that all CI checks were clean prior to every commit merge. | ||
|
|
||
| The plan is to run repeated CI checks on the merge commit to master. | ||
|
|
||
| ## 2. Documentation | ||
|
|
||
| [User documentation](https://ozone.apache.org/docs/next/administrator-guide/operations/s3/sts) for Ozone S3 Security Token Service (STS) has been added. | ||
|
|
||
| ## 3. Design, attached the docs | ||
|
|
||
| Design document can be found here: [AWS STS Design for Ozone S3](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-hdds/docs/content/design/ozone-sts.md). | ||
|
|
||
| ## 4. S3 compatibility | ||
|
|
||
| See sections [3.2](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-hdds/docs/content/design/ozone-sts.md#32-limitations-in-assumerole-api-support) and [3.3](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-hdds/docs/content/design/ozone-sts.md#33-limitations-in-iam-session-policy-support) in the design document for limitations in the AssumeRole API compatibility. | ||
|
|
||
| Also **please note** that the Ranger feature flag (along with the Ozone feature flag) must be enabled in order to get AWS compatibility in terms of the requested actions and resulting actions the STS tokens have. | ||
|
|
||
| ## 5. Docker-compose / Acceptance tests | ||
|
|
||
| New robot tests [`ozone-secure-sts.robot`](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-ozone/dist/src/main/smoketest/security/ozone-secure-sts.robot) and [`ozone-secure-sts-multitenant.robot`](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-ozone/dist/src/main/smoketest/security/ozone-secure-sts-multitenant.robot) are being added. Furthermore, a shell script to test Polaris with Ozone STS ([`polaris-smoketest.sh`](https://github.com/apache/ozone/blob/HDDS-13323-sts/hadoop-ozone/dist/src/main/compose/ozonesecure-ha/polaris-smoketest.sh)) is being added. | ||
|
|
||
| ## 6. Support of containers / Kubernetes | ||
|
|
||
| No addition. No change in existing support. | ||
|
|
||
| ## 7. Coverage / Code quality | ||
|
|
||
| [New Code Coverage](https://sonarcloud.io/summary/new_code?id=hadoop-ozone&branch=HDDS-13323-sts) for STS (HDDS-13323-sts) is **81.58%** and [Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=HDDS-13323-sts) is **75.4%**. | ||
| [Overall Code Coverage](https://sonarcloud.io/summary/overall?id=hadoop-ozone&branch=master) for master is **78.6%**. | ||
|
|
||
| ## 8. Build time | ||
|
|
||
| [Build time for the latest commit](https://github.com/apache/ozone/actions/runs/30604883966/job/91075071143) from HDDS-13323-sts branch is **11m 32s**. | ||
| [Build time for the latest commit](https://github.com/apache/ozone/actions/runs/30662032542/job/91260363408) from the master branch is **11m 48s**. | ||
|
|
||
| ## 9. Possible incompatible changes/used feature flag | ||
|
|
||
| STS is disabled by default and gated by configuration flags. **Both** flags below must be enabled for correct, fine-grained authorization behavior. | ||
|
|
||
| ### Ozone feature flag | ||
|
|
||
| Enable in `ozone-site.xml`: | ||
|
|
||
| ```xml | ||
| <property> | ||
| <name>ozone.s3g.sts.http.enabled</name> | ||
| <value>true</value> | ||
| <description>Enable the Ozone S3 Gateway STS endpoint.</description> | ||
| </property> | ||
| ``` | ||
|
|
||
| ### Ranger feature flag (action-matches Policy Condition) | ||
|
|
||
| Enable in `ranger-admin-site.xml`: | ||
|
|
||
| ```xml | ||
| <property> | ||
| <name>ranger.servicedef.ozone.enableActionMatcherInPoliciesCondition</name> | ||
| <value>true</value> | ||
| </property> | ||
| ``` | ||
|
|
||
| A new OM RocksDB table, `s3RevokedStsTokenTable` (`sts_session_token` → `insertionTimeMillis`), stores revoked STS token entries. No OM versioning-framework feature flag is used. | ||
|
|
||
| ## 10. Third-party dependencies/License changes | ||
|
|
||
| There are no third party dependencies introduced by this feature. | ||
|
|
||
| ## 11. Performance | ||
|
|
||
| The AssumeRole invocations are write requests and therefore the throughput is shared with other OM write calls. | ||
|
|
||
| ## 12. Security considerations | ||
|
|
||
| STS introduces a new network-accessible endpoint (ports 9880/9881) for AssumeRole. Key security properties: | ||
|
|
||
| - **Temporary credentials only**: AssumeRole requires permanent S3 credentials (from `ozone s3 getsecret`) of a Kerberos identity to create the temporary credentials. Temporary credentials expire between 15 minutes and 12 hours. | ||
| - **Ranger-only authorization**: STS requires Apache Ranger with role-based policies and optional inline IAM session policies. | ||
| - **Fail-closed revocation checks**: If the revocation database is unavailable, token validation fails rather than allowing access. | ||
|
|
||
| ### Token revocation | ||
|
|
||
| Ozone adds explicit revocation support: | ||
|
|
||
| **Revoke a single session token** — only the token creator or an S3/tenant admin may revoke. This adds the token to the OM RocksDB `s3RevokedStsTokenTable`. Subsequent S3 requests using that session token are denied immediately, even before the token's natural expiration. | ||
|
|
||
| **Revoke all tokens for a user** — revoking a user's permanent S3 secret also invalidates all outstanding STS tokens created by that user. This is useful when a long-lived credential is compromised and all derived temporary credentials must be invalidated at once. | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
https://ozone.apache.org/docs/next/administrator-guide/operations/s3/sts is not accessible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
right, it should be accessible once #510 is merged