Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
345 changes: 345 additions & 0 deletions client-libraries/java-dependency-configuration.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions client-libraries/java-migrate-to-v5.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ The two APIs can run **side by side in the same JVM**, so you can move one produ

- **Java 17 or later.** The combined Java client artifacts require this runtime for both APIs.
- **Pulsar 5.x brokers with scalable topics enabled.** The v5 client requires the scalable-topic protocol even when accessing existing `persistent://` topics. It cannot connect through this API to older brokers or brokers with `scalableTopicsEnabled=false`. These v5 connection requirements do not apply merely because a v4 application uses a combined dependency.
- **Aligned dependencies.** Complete the [dependency setup and runtime graph checks](java-setup.md#pulsar-bom) before migrating API usage.
- **Aligned dependencies.** Complete the [dependency setup and runtime graph checks](java-dependency-configuration.md#pulsar-bom) before migrating API usage.

## Dependencies

Follow [Java client setup](java-setup.md) for Maven and Gradle declarations, [Pulsar and Netty BOMs](java-setup.md#pulsar-bom), and [transitive exclusions](java-setup.md#replace-existing-dependencies). It is the canonical dependency guide, including runtime graph checks and the shaded fallback for unresolved conflicts.
Follow [Java client setup](java-setup.md) for Maven and Gradle declarations, [Pulsar and Netty BOMs](java-dependency-configuration.md#pulsar-bom), and [transitive exclusions](java-dependency-configuration.md#replace-existing-dependencies). The dependency configuration page is the canonical dependency guide, including runtime graph checks and the shaded fallback for unresolved conflicts.

Changing dependencies does not require changing v4 application code. Continue with the API migration below when you are ready to adopt v5.

Expand Down
182 changes: 12 additions & 170 deletions client-libraries/java-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ sidebar_label: "Set up"
description: Learn how to set up Java client library in Pulsar.
---

Use the combined Java dependency for applications using the v4 client, v5 client, the admin API, or any combination of them. The combined artifacts require **Java 17 or later**.
Use the combined Java dependency for applications using the v4 client, v5 client, the admin API, or any combination of them. The combined artifacts require **Java 17 or later**. **Choose Java 25 LTS for running new applications when you have the choice.**

Changing dependencies and changing client APIs are separate choices. Both combined artifacts support the existing v4 API (`org.apache.pulsar.client.api`), the v5 API (`org.apache.pulsar.client.api.v5`), and the admin API. You can update the dependency while keeping your v4 application code. Existing v4 applications using regular topics do not need to change their API or dependencies for a broker upgrade. Use this setup when configuring or updating application dependencies.

Expand All @@ -18,181 +18,23 @@ Use **`pulsar-client-v5-all`** by default for new dependency configurations. It
| `pulsar-client-v5-all` | Unshaded v4 client, v5 client, and admin implementations, with their transitive dependencies |
| `pulsar-client-v5-shaded` | One JAR containing relocated v4 client, v5 client, and admin implementations and bundled third-party dependencies |

Set `pulsar.version` in Maven or `pulsarVersion` in Gradle to your target client release from Pulsar 5 or later that publishes the chosen combined artifact. Use the same version for all Pulsar dependencies. These variables identify the dependency version; they do not select the v4 or v5 API.
:::info Strongly recommended: align dependencies and exclude conflicting clients

### Maven
To avoid classpath conflicts and incompatible libraries:

Add the following dependency to your `pom.xml`. The `${pulsar.version}` value comes from your Maven properties.
- **Import both the [Pulsar and Netty BOMs](java-dependency-configuration.md#pulsar-bom)** to align dependency versions.
- **[Remove and exclude conflicting clients](java-dependency-configuration.md#replace-existing-dependencies).** BOMs do not remove duplicate implementations. Never combine `pulsar-client-v5-all` with `pulsar-client-v5-shaded`.
- **[Verify the runtime dependency graph](java-dependency-configuration.md#verify-netty-alignment)**, including dependencies supplied by frameworks.

```xml
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-v5-all</artifactId>
<version>${pulsar.version}</version>
</dependency>
```

### Gradle

In Gradle Kotlin DSL, define the `pulsarVersion` project property and use it in `build.gradle.kts`:

```kotlin
val pulsarVersion: String by project

dependencies {
implementation("org.apache.pulsar:pulsar-client-v5-all:$pulsarVersion")
}
```

### Shaded fallback

If you need the shaded fallback, use `pulsar-client-v5-shaded` as an ordinary dependency, without a classifier, variant attributes, or implementation exclusions on this dependency. Despite its name, it includes the v4 client and admin implementation too.

```xml
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-v5-shaded</artifactId>
<version>${pulsar.version}</version>
</dependency>
```

```kotlin
val pulsarVersion: String by project

dependencies {
implementation("org.apache.pulsar:pulsar-client-v5-shaded:$pulsarVersion")
}
```

The shaded artifact has its own dependency-reduced POM. A Maven classifier shares the original artifact's POM and dependency graph: selecting a shaded classifier on an unshaded aggregate would still pull in its unshaded implementations. No `shaded` classifier is published for `pulsar-client-v5-all`.

The shaded artifact keeps `pulsar-client-api`, `pulsar-client-api-v5`, `pulsar-client-admin-api`, `pulsar-tls-factory-api`, and `pulsar-http-client-api` as unshaded external dependencies. Logging, Bouncy Castle, and other intentionally non-bundled libraries also remain external. Resolve the published metadata with Maven or Gradle instead of copying only the client JAR. For provider replacement and packaging requirements, see [Bouncy Castle providers](pathname:///docs/next/security-bouncy-castle).

Applications using protobuf schemas must also provide `com.google.protobuf:protobuf-java`. The client artifacts neither bundle it nor declare it transitively. Align generated Protobuf classes and dependency overrides with the resolved runtime. Applications using reflective Avro schemas should also review [Java Avro class trust](pathname:///docs/next/schema-get-started#java-avro-class-trust).

### Replace existing dependencies {#replace-existing-dependencies}

When adopting a combined dependency, replace separately declared `pulsar-client`, `pulsar-client-admin`, and the older `pulsar-client-all` aggregate. `pulsar-client-admin` is the Java artifact; `pulsar-admin` is the CLI name. Exclude older artifacts from dependencies that introduce them transitively. Keeping the separately shaded client/admin JARs alongside the combined dependency duplicates implementations and bundled libraries on the classpath. For the shaded fallback, also remove separately declared unshaded implementations.

For Gradle, apply exclusions to application configurations:

```kotlin
configurations.configureEach {
exclude(group = "org.apache.pulsar", module = "pulsar-client")
exclude(group = "org.apache.pulsar", module = "pulsar-client-admin")
exclude(group = "org.apache.pulsar", module = "pulsar-client-all")
}
```

In Maven, add exclusions to **each dependency** that introduces those artifacts. Maven exclusions apply to that dependency's subtree, not globally:

```xml
<exclusions>
<exclusion>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-admin</artifactId>
</exclusion>
<exclusion>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-all</artifactId>
</exclusion>
</exclusions>
```

Inspect the resolved runtime dependency graph:

```shell
mvn dependency:tree -Dscope=runtime '-Dincludes=org.apache.pulsar:*'
./gradlew dependencies --configuration runtimeClasspath
```

Verify that the replaced artifacts are gone and Pulsar versions agree. The unshaded aggregate resolves implementation modules such as `pulsar-client-v5`, `pulsar-client-original`, and `pulsar-client-admin-original`. The shaded fallback instead exposes the external APIs and intentionally non-bundled libraries through its dependency-reduced publication. Code that directly imports relocated implementation or third-party classes must move to public APIs or use the unshaded aggregate.

### Pulsar and Netty BOMs {#pulsar-bom}

Import `pulsar-bom` to align Pulsar artifacts and `io.netty:netty-bom` to align Netty modules. A BOM manages versions; it does not replace artifacts or remove duplicate implementations.

The recommended unshaded client uses **Netty 4.2.x**, currently **@pulsar:version:netty@**. Applications using Netty 4.1.x should upgrade their Netty dependencies to 4.2.x together. Netty 4.2 is largely backward compatible with 4.1, but the two lines cannot coexist on the same classpath. Review the [Netty migration guide](https://netty.io/wiki/netty-4.2-migration-guide.html), particularly TLS hostname verification, allocator defaults, and libraries that use Netty internally.

#### Maven {#pulsar-bom-maven}

Add the Netty version property and import both BOMs in `pom.xml`, using your existing `pulsar.version` property:

```xml
<properties>
<netty.version>@pulsar:version:netty@</netty.version>
</properties>

<dependencyManagement>
<dependencies>
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-bom</artifactId>
<version>${netty.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-bom</artifactId>
<version>${pulsar.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>

<dependencies>
<dependency>
<groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-v5-all</artifactId>
</dependency>
</dependencies>
```

Remove older explicit Netty versions and reconcile dependency management inherited from frameworks or parent POMs. When imported BOMs manage the same artifact, Maven gives precedence to the first import; direct dependency-management entries take precedence over imports.

#### Gradle {#pulsar-bom-gradle}

Use `platform` to import the BOMs. It allows normal dependency conflict resolution; inspect the resolved graph to confirm that Pulsar and Netty versions remain aligned. Avoid `enforcedPlatform` as a general default, especially for published libraries, because its enforced versions propagate to consumers. See [Gradle platform guidance](https://docs.gradle.org/current/userguide/platforms.html#sec:enforced-platform).

```kotlin
val pulsarVersion: String by project
val nettyVersion = "@pulsar:version:netty@"

dependencies {
implementation(platform("org.apache.pulsar:pulsar-bom:$pulsarVersion"))
implementation(platform("io.netty:netty-bom:$nettyVersion"))
implementation("org.apache.pulsar:pulsar-client-v5-all")
}
```

#### Verify Netty alignment

Inspect the resolved runtime graph after applying the BOMs:

```shell
mvn dependency:tree -Dscope=runtime '-Dincludes=io.netty:*'
./gradlew dependencyInsight --dependency io.netty --configuration runtimeClasspath
```

Ensure the resolved Netty core modules use one consistent 4.2.x version and that the packaged application contains no leftover 4.1.x or duplicate Netty JARs. Requested versions shown as replaced in a dependency report are not additional runtime copies. Netty components with independent version schemes, such as `netty-tcnative`, should use the versions managed by the Netty BOM rather than being assigned the core version manually.

### Spring Boot

When a framework supplies Pulsar dependencies, align its managed Pulsar and Netty versions with the [BOMs above](#pulsar-bom), apply the [transitive exclusions](#replace-existing-dependencies), and inspect the resulting runtime graph. Adding a combined dependency alone does not remove the framework's existing client dependency.

#### Spring Boot using Maven {#spring-boot-maven}
:::

Set Spring Boot's `pulsar.version` Maven property to the same target client version used above. Set `netty.version` to `@pulsar:version:netty@` as well. Add the combined dependency and exclude the replaced artifacts from dependencies that introduce them, such as the Pulsar starter. See [Spring Boot's Pulsar support](https://docs.spring.io/spring-boot/reference/messaging/pulsar.html) for its dependency management.
### Complete dependency configuration {#pulsar-bom}

#### Spring Boot using Gradle {#spring-boot-gradle}
For copyable build files with both BOMs and conflict handling, see [Java dependency configuration](java-dependency-configuration.md):

When using the Spring Dependency Management plugin (`io.spring.dependency-management`), set its `pulsar.version` property to the same `pulsarVersion`, for example `extra["pulsar.version"] = pulsarVersion`. Also set `extra["netty.version"] = "@pulsar:version:netty@"` to align Spring's managed Netty dependencies. Apply the combined dependency and exclusions above. See [Spring Boot's dependency version properties](https://docs.spring.io/spring-boot/appendix/dependency-versions/properties.html) for managed versions.
- [Complete Maven example](java-dependency-configuration.md#maven)
- [Complete Gradle example](java-dependency-configuration.md#gradle)
- [Replace existing dependencies](java-dependency-configuration.md#replace-existing-dependencies), [shaded fallback](java-dependency-configuration.md#shaded-fallback), and [Spring Boot](java-dependency-configuration.md#spring-boot)

## Step 2: Connect to Pulsar cluster

Expand Down
2 changes: 1 addition & 1 deletion client-libraries/java-v5.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The v5 client requires **Java 17** and Pulsar 5.x brokers with `scalableTopicsEn

## Install

Use **`pulsar-client-v5-all`** and follow [Java client setup](java-setup.md#step-1-install-java-client-library) for the Maven and Gradle dependency examples, [Pulsar and Netty BOMs](java-setup.md#pulsar-bom), and transitive dependency exclusions. The setup page also covers external dependencies and the shaded fallback for unresolved dependency conflicts.
Use **`pulsar-client-v5-all`** and follow [Java client setup](java-setup.md#step-1-install-java-client-library) for the Maven and Gradle dependency examples, [Pulsar and Netty BOMs](java-dependency-configuration.md#pulsar-bom), and transitive dependency exclusions. The dependency configuration page also covers external dependencies and the shaded fallback for unresolved dependency conflicts.

The v5 API lives under [`org.apache.pulsar.client.api.v5`](@pulsar:javadoc:client-v5@/org/apache/pulsar/client/api/v5/package-summary.html). Choosing a combined dependency does not switch an application from v4 to v5; changing the API is a separate [source migration](java-migrate-to-v5.md).

Expand Down
8 changes: 6 additions & 2 deletions docs/about.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,11 @@ import { docUrl } from "@site/src/utils/index";

This portal holds a variety of topics, tutorials, guides, and reference material to help you work with Pulsar.

Preparing to upgrade? Start with the [release highlights](release-highlights.md) and the [cluster upgrade guide](administration-upgrade.md).
:::tip Preparing to upgrade to Pulsar @pulsar:version:major@?

Start with the [release highlights](release-highlights.md) and the [cluster upgrade guide](administration-upgrade.md).

:::

## Choose your path
Select one of the content blocks below to begin your Pulsar journey. If you ...
Expand Down Expand Up @@ -59,4 +63,4 @@ The Pulsar community on GitHub is active, passionate, and knowledgeable. Join d

* Please go to [the community page](pathname:///community/#section-discussions) to find the contact information for joining various communication channels.
* The main communication channel for the Apache Pulsar project are [the mailing lists](pathname:///contact/).
* There's a separate page for Apache Pulsar [Security advisories and Security policy](pathname:///security/).
* There's a separate page for Apache Pulsar [Security advisories and Security policy](pathname:///security/).
2 changes: 1 addition & 1 deletion docs/administration-upgrade-to-5.0.x-applications.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Dependency migration is separate from v5 API adoption. An application using eith

When adopting the unshaded `pulsar-client-v5-all` dependency, upgrade application dependencies from Netty 4.1.x to **Netty 4.2.x**. Pulsar uses **@pulsar:version:netty@**. Netty 4.2 is largely backward compatible with 4.1, but both lines cannot coexist on the same classpath. This dependency alignment is part of updating the application; it is not required merely to upgrade brokers while retaining an existing v4 client dependency.

Import `io.netty:netty-bom` alongside `pulsar-bom`, update framework-managed Netty versions, and verify that the resolved runtime graph and packaged application contain a consistent set of Netty modules without old or duplicate JARs. See the [Maven and Gradle setup examples](pathname:///docs/client-libraries/java-setup#pulsar-bom) and the [Netty migration guide](https://netty.io/wiki/netty-4.2-migration-guide.html).
Import `io.netty:netty-bom` alongside `pulsar-bom`, update framework-managed Netty versions, and verify that the resolved runtime graph and packaged application contain a consistent set of Netty modules without old or duplicate JARs. See the [Maven and Gradle setup examples](pathname:///docs/client-libraries/java-dependency-configuration#pulsar-bom) and the [Netty migration guide](https://netty.io/wiki/netty-4.2-migration-guide.html).

## Check schema dependencies

Expand Down
6 changes: 1 addition & 5 deletions docs/functions-package-python.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,11 +40,7 @@ To package a Python function into **one Python file**, complete the following st
pip install 'pulsar-client==@pulsar:version:python@'
```

And install protobuf tools to generate the proto files:

```bash
pip install 'protobuf==3.20.*'
```
This example does not require generating Protocol Buffers files or installing `protobuf` separately. The Pulsar Docker image already includes the Python function runtime's dependencies.

3. Copy the Python function file to the Pulsar image.

Expand Down
9 changes: 9 additions & 0 deletions docs/functions-runtime-kubernetes.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ sidebar_label: "Configure Kubernetes runtime"
description: Configure Kubernetes runtime for functions in Pulsar.
---

:::warning Run only fully trusted code

Pulsar Functions and connectors execute user-provided code by design. This intended capability is not itself a remote code execution (RCE) vulnerability. **Run only code that you fully trust**, because it can modify its execution environment:

- **Thread and process runtimes** can read or modify any files and state accessible to the process they run in.
- **The Kubernetes runtime** does not, on its own, restrict access to Kubernetes cluster resources. Pulsar provides hooks for custom hardening, but the hardening itself is outside the project.

:::

The Kubernetes runtime works when a function worker generates and applies Kubernetes manifests. The manifests generated by a function worker include:
* a `StatefulSet`
By default, the `StatefulSet` manifest has a single pod with a number of replicas. The number is determined by the [parallelism](functions-deploy-cluster-parallelism.md) of the function. The pod downloads the function payload (via the function worker REST API) on pod boot. The pod's container image is configurable if the function runtime is configured.
Expand Down
Loading
Loading