Skip to content

HDDS-15476. [Website V2] [Docs] [User Guide] The Ozone Client #455

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
amaliujia wants to merge 1 commit into
base: master
Choose a base branch
from
Open

HDDS-15476. [Website V2] [Docs] [User Guide] The Ozone Client #455

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 170 additions & 6 deletions versioned_docs/version-2.1.0/04-user-guide/02-clients/01-ozone.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,176 @@
sidebar_label: Ozone
---

# The Ozone Client
The Ozone client is the software that applications and operators use to read and write data in an Ozone cluster. It is not a standalone daemon; it runs inside the `ozone` command-line tools, Hadoop-compatible filesystem drivers, or embedded in Java applications through the Ozone client library.

Document how to install, configure and run Ozone's client. This should also include
This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other.

@Gargi-jais11 Gargi-jais11 Jun 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other.
This page describes how to install, configure, and run the client, and how the three primary access interfaces relate to each other: **the native RPC protocol**, **the Hadoop-compatible filesystem**, and **the S3-compatible REST AP**I.


- Which interfaces it supports (ofs, o3, Java).

@Gargi-jais11 Gargi-jais11 Jun 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no section for Hadoop filesystem integration (O3FS and OFS)

- Information about configuration files it reads (core-site.xml, client log4j).
- Environment variables and common JVM flags (`-D`) that can be used at the CLI.
## Installation

**TODO:** File a subtask under [HDDS-9858](https://issues.apache.org/jira/browse/HDDS-9858) and complete this page or section.
### Binary distribution

Download an official release tarball from the [Apache Ozone download page](https://dlcdn.apache.org/ozone) and extract it on every machine that will run client commands or applications:

```bash
curl -L -O https://dlcdn.apache.org/ozone/<version>/ozone-<version>.tar.gz
tar -xzf ozone-<version>.tar.gz
cd ozone-<version>
export OZONE_HOME=$(pwd)
export PATH=$OZONE_HOME/bin:$PATH
```

The distribution layout relevant to clients:

| Path | Purpose |
|------|---------|
| `bin/ozone` | Main entry point for CLI and client subcommands |
| `etc/hadoop/` | Configuration directory (`OZONE_CONF_DIR`) |
| `share/ozone/lib/` | Client and filesystem JARs |
| `share/ozone/lib/ozone-filesystem-hadoop3-*.jar` | Hadoop-compatible filesystem driver |

### Docker

For development and testing, the official `apache/ozone` image includes a preconfigured client. From inside a running container:

```bash
docker compose exec om ozone version
docker compose exec om ozone sh volume list /
```

See the [Docker installation guide](../../quick-start/installation/docker) for a full quick-start.

### Build from source

To build the distribution locally:

```bash
mvn clean package -DskipTests=true -Pdist
```

The tarball is produced under `hadoop-ozone/dist/target/`.

## Configuration

Before using the client, point it at a running Ozone cluster by placing configuration files in `OZONE_CONF_DIR` (defaults to `$OZONE_HOME/etc/hadoop`).

### Generate a starting configuration

Use `ozone genconf` to create a template `ozone-site.xml`:

@Gargi-jais11 Gargi-jais11 Jun 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Use `ozone genconf` to create a template `ozone-site.xml`:
Use `ozone genconf` to generate a template `ozone-site.xml `containing the minimally required Ozone properties and their default values:


```bash
ozone genconf /path/to/output
cp /path/to/output/ozone-site.xml $OZONE_HOME/etc/hadoop/
```

Edit the generated file and set at minimum the client-relevant properties:

| Property | Description | Default port |
|----------|-------------|--------------|
| `ozone.om.address` | Ozone Manager address used by the client and filesystem | 9862 |
| `ozone.scm.client.address` | SCM client service address | 9860 |
Comment on lines +68 to +71

@Gargi-jais11 Gargi-jais11 Jun 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| Property | Description | Default port |
|----------|-------------|--------------|
| `ozone.om.address` | Ozone Manager address used by the client and filesystem | 9862 |
| `ozone.scm.client.address` | SCM client service address | 9860 |
| Property | Description | Default |
|----------|-------------|--------------|
| `ozone.om.address` | Ozone Manager address used by the client and filesystem | 9862 |
| `ozone.scm.client.address` | SCM client service address |
No default address; set this to your SCM host (for example, scm1:9860) |
| `ozone.scm.client.port` | SCM client service port when the address omits a port | 9860 |


### Verify configuration

```bash
# Show resolved environment
ozone envvars

# Read a specific config value
ozone getconf -confKey ozone.om.address
```

Use `--config /path/to/conf` on any `ozone` command to override `OZONE_CONF_DIR` for a single invocation:

```bash
ozone --config /etc/ozone sh volume list /
```

## Running the client

All client operations are invoked through the `ozone` script. Common client subcommands:

| Command | Description |
|---------|-------------|
| `ozone sh` | Object-store shell (volumes, buckets, keys) via the native RPC protocol |
| `ozone fs` | Hadoop-compatible filesystem shell (equivalent to `hadoop fs`) |
| `ozone s3` | S3-related CLI operations |
| `ozone getconf` | Print configuration values |
| `ozone version` | Print the Ozone version |

@Gargi-jais11 Gargi-jais11 Jun 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The subcommand table is very incomplete The table lists five commands. The ozone script registers many more that are relevant :

`ozone admin:`Cluster administration (pipeline, container, replication ops)
`ozone debug:` Diagnostic tools (replaces deprecated auditparser, checknative)
`ozone tenant:` Multi-tenancy management
`ozone insight `: Metrics and log insight
`ozone freon :` Load generator / performance testing
`ozone dtutil :` Delegation token utilities
`ozone classpath :` Print effective classpath (useful for embedding the client)

If not specify all then atleast we should add ozone admin and ozone debug as well to the table.


Example workflow after configuration:

```bash
# Admin: create volumne and bucket

@Gargi-jais11 Gargi-jais11 Jun 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo :

Suggested change
# Admin: create volumne and bucket
# Admin: create volume and bucket

ozone sh volume create /vol1
ozone sh bucket create /vol1/bucket1
```

By default, CLI tools suppress logging output. Pass `--loglevel DEBUG` (or any Log4j level) to enable verbose client logging:

```bash
ozone --loglevel DEBUG sh volume info /vol1
```

### Client log4j configuration

Client-side logging is controlled by `log4j.properties` in `$OZONE_CONF_DIR`. The distribution ships a template at `etc/hadoop/log4j.properties`.

Key settings:

| Setting | Default | Description |
|---------|---------|-------------|
| `hadoop.root.logger` | `INFO,console` | Root Log4j logger and appenders |
| `hadoop.log.dir` | Set via JVM property | Directory for log files |
| `hadoop.log.file` | `hadoop.log` | Log file name |

The `ozone` script injects these as JVM system properties at runtime:

```
-Dhadoop.log.dir=${OZONE_LOG_DIR}
-Dhadoop.log.file=${OZONE_LOGFILE}
-Dhadoop.root.logger=${OZONE_ROOT_LOGGER}
```

For CLI subcommands (`ozone sh`, `ozone fs`, etc.), logging defaults to **OFF** unless you pass `--loglevel` or set `OZONE_ROOT_LOGGER` / `OZONE_LOGLEVEL` explicitly. This keeps routine command output clean.

@Gargi-jais11 Gargi-jais11 Jun 10, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe info also we should add and write it as a note.

Suggested change
For CLI subcommands (`ozone sh`, `ozone fs`, etc.), logging defaults to **OFF** unless you pass `--loglevel` or set `OZONE_ROOT_LOGGER` / `OZONE_LOGLEVEL` explicitly. This keeps routine command output clean.
**Note** - For CLI subcommands (`ozone sh`, `ozone fs`, etc.), logging defaults to **OFF** unless you pass `--loglevel` or set `OZONE_ROOT_LOGGER` / `OZONE_LOGLEVEL` explicitly. This keeps routine command output clean.
**Precedence:** The JVM system properties injected by the ozone script (`-Dhadoop.root.logger, -Dhadoop.log.dir, -Dhadoop.log.file)` take precedence over values in **log4j.properties.** Editing **log4j.properties** alone does not enable CLI logging; use `--loglevel`, `OZONE_ROOT_LOGGER`, or `OZONE_SH_OPTS / OZONE_FS_OPTS` as shown below.


To enable debug logging for a single command:

```bash
ozone --loglevel DEBUG sh bucket list /vol1
```

To persistently enable client logging, edit `etc/hadoop/ozone-env.sh`:

```bash
export OZONE_ROOT_LOGGER=DEBUG,console
```

Or override per command family:

```bash
export OZONE_SH_OPTS="-Dhadoop.root.logger=DEBUG,console"
export OZONE_FS_OPTS="-Dhadoop.root.logger=DEBUG,console"
```

Daemon processes (OM, SCM, DataNode) use additional Log4j2 audit configuration files (`om-audit-log4j2.properties`, etc.). Those apply to server-side components, not to client CLI tools.

## Environment variables

@jojochuang jojochuang Jun 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also mentioned here: https://ozone.apache.org/docs/next/administrator-guide/configuration/basic/environment-variables


Environment variables are processed by `bin/ozone` via `ozone-env.sh` in `$OZONE_CONF_DIR`. Copy and customize the template from the distribution or edit the file in place.

### Required and commonly used variables

| Variable | Description | Default |
|----------|-------------|---------|
| `JAVA_HOME` | Path to the JDK | Platform-specific; required on Linux |
| `OZONE_HOME` | Ozone installation directory | Derived from script location |
| `OZONE_CONF_DIR` | Configuration directory | `$OZONE_HOME/etc/hadoop` |
| `OZONE_LOG_DIR` | Log output directory | `$OZONE_HOME/logs` |

`HADOOP_CONF_DIR` is deprecated; use `OZONE_CONF_DIR` instead. If only `HADOOP_CONF_DIR` is set, the shell maps it to `OZONE_CONF_DIR` with a deprecation warning.

Verify effective values:

```bash
ozone envvars
```
Loading