Skip to content

feat: Add HBase Connector support for Spark 4.x - #797

Merged
razvan merged 3 commits into
mainfrom
feat/spark-4-hbase-connectors
Oct 28, 2026
Merged

razvan merged 3 commits into
mainfrom
feat/spark-4-hbase-connectors

Conversation

@razvan

@razvan razvan commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Description

Locally tested with images from stackabletech/docker-images#1723

--- PASS: kuttl (3425.56s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/resources_openshift-false_spark-4.1.3 (54.82s)
        --- PASS: kuttl/harness/config-overrides_openshift-false_spark-4.2.0 (43.57s)
        --- PASS: kuttl/harness/delta-lake_openshift-false_spark-delta-lake-4.1.3 (266.70s)
        --- PASS: kuttl/harness/logging_openshift-false_spark-logging-4.2.0_ny-tlc-report-0.3.0 (305.88s)
        --- PASS: kuttl/harness/spark-examples_openshift-false_spark-4.2.0 (46.68s)
        --- PASS: kuttl/harness/spark-examples_openshift-false_spark-4.1.3 (47.92s)
        --- PASS: kuttl/harness/logging_openshift-false_spark-logging-4.1.3_ny-tlc-report-0.3.0 (311.82s)
        --- PASS: kuttl/harness/overrides_openshift-false_spark-4.2.0 (137.50s)
        --- PASS: kuttl/harness/iceberg_openshift-false_spark-iceberg-4.1.3_iceberg-latest-1.11.0 (61.56s)
        --- PASS: kuttl/harness/overrides_openshift-false_spark-4.1.3 (123.56s)
        --- PASS: kuttl/harness/spark-connect-kerberos_openshift-false_iceberg-latest-1.11.0_hive-iceberg-4.0.0_spark-connect-4.1.3_krb5-1.21.1_kerberos-realm-CLUSTER.LOCAL_s3-use-tls-true (210.84s)
        --- PASS: kuttl/harness/spark-history-server_openshift-false_spark-4.2.0_s3-use-tls-true (245.88s)
        --- PASS: kuttl/harness/config-overrides_openshift-false_spark-4.1.3 (44.56s)
        --- PASS: kuttl/harness/graceful-shutdown_openshift-false_spark-4.2.0 (58.71s)
        --- PASS: kuttl/harness/graceful-shutdown_openshift-false_spark-4.1.3 (58.95s)
        --- PASS: kuttl/harness/signal-propagation_openshift-false_spark-4.2.0 (46.21s)
        --- PASS: kuttl/harness/smoke_openshift-false_spark-4.2.0_s3-use-tls-true (165.12s)
        --- PASS: kuttl/harness/spark-history-server_openshift-false_spark-4.1.3_s3-use-tls-true (228.54s)
        --- PASS: kuttl/harness/product-config-compat_openshift-false_spark-regression-4.2.0 (178.17s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3_openshift-false_spark-4.2.0 (246.01s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3_openshift-false_spark-4.1.3 (127.44s)
        --- PASS: kuttl/harness/signal-propagation_openshift-false_spark-4.1.3 (51.35s)
        --- PASS: kuttl/harness/spark-pi-private-s3_openshift-false_spark-4.2.0 (107.60s)
        --- PASS: kuttl/harness/spark-connect_openshift-false_iceberg-latest-1.11.0_hive-iceberg-4.0.0_spark-connect-4.1.3_s3-use-tls-true (269.99s)
        --- PASS: kuttl/harness/spark-pi-private-s3_openshift-false_spark-4.1.3 (112.84s)
        --- PASS: kuttl/harness/delta-lake_openshift-false_spark-delta-lake-4.2.0 (258.53s)
        --- PASS: kuttl/harness/custom-log-directory_openshift-false_spark-4.1.3_hdfs-latest-3.5.0_zookeeper-latest-3.9.6 (172.84s)
        --- PASS: kuttl/harness/custom-log-directory_openshift-false_spark-4.2.0_hdfs-latest-3.5.0_zookeeper-latest-3.9.6 (168.49s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3-image_openshift-false_spark-4.2.0_ny-tlc-report-0.3.0 (128.31s)
        --- PASS: kuttl/harness/pyspark-ny-public-s3-image_openshift-false_spark-4.1.3_ny-tlc-report-0.3.0 (124.00s)
        --- PASS: kuttl/harness/transit-encryption_openshift-false_spark-4.2.0 (51.63s)
        --- PASS: kuttl/harness/transit-encryption_openshift-false_spark-4.1.3 (50.63s)
        --- PASS: kuttl/harness/hbase-connector_openshift-false_spark-hbase-connector-4.1.3_hbase-2.6.7_hdfs-latest-3.5.0_zookeeper-latest-3.9.6 (184.62s)
        --- PASS: kuttl/harness/resources_openshift-false_spark-4.2.0 (56.82s)
        --- PASS: kuttl/harness/smoke_openshift-false_spark-4.1.3_s3-use-tls-true (194.97s)
        --- PASS: kuttl/harness/hbase-connector_openshift-false_spark-hbase-connector-4.2.0_hbase-2.6.7_hdfs-latest-3.5.0_zookeeper-latest-3.9.6 (1013.27s)
PASS

Definition of Done Checklist

  • Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
  • Please make sure all these things are done and tick the boxes

Author

  • Changes are OpenShift compatible
  • CRD changes approved
  • CRD documentation for all fields, following the style guide.
  • Helm chart can be installed and deployed operator works
  • Integration tests passed (for non trivial changes)
  • Changes need to be "offline" compatible
  • Links to generated (nightly) docs added
  • Release note snippet added

Reviewer

  • Code contains useful comments
  • Code contains useful logging statements
  • (Integration-)Test cases added
  • Documentation added or updated. Follows the style guide.
  • Changelog updated
  • Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

  • Feature Tracker has been updated
  • Proper release label has been added
  • Links to generated (nightly) docs added
  • Release note snippet added
  • Add type/deprecation label & add to the deprecation schedule
  • Add type/experimental label & add to the experimental features tracker

@razvan razvan self-assigned this Oct 26, 2026
@razvan
razvan marked this pull request as ready for review October 26, 2026 13:32
@razvan razvan moved this to Development: Waiting for Review in Stackable Engineering Oct 26, 2026
This causes tests with local product images to fail.
@Maleware
Maleware self-requested a review October 27, 2026 15:05
@Maleware Maleware moved this from Development: Waiting for Review to Development: In Review in Stackable Engineering Oct 27, 2026

@Maleware Maleware left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you. Tests passed on ARM64

@razvan
razvan added this pull request to the merge queue Oct 28, 2026
Merged via the queue into main with commit 22697c8 Oct 28, 2026
11 checks passed
@razvan
razvan deleted the feat/spark-4-hbase-connectors branch October 28, 2026 16:08
@razvan

razvan commented Oct 28, 2026

Copy link
Copy Markdown
Member Author

Release note in the associated issue: stackabletech/issues#904 (comment)

@razvan razvan moved this from Development: In Review to Development: Done in Stackable Engineering Oct 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Development: Done

Development

Successfully merging this pull request may close these issues.

2 participants