Skip to content

RANGER-5680: Make Solr optional in docker so Ranger can run without it - #1058

Open
paras200 wants to merge 1 commit into
apache:masterfrom
paras200:RANGER-5680
Open

RANGER-5680: Make Solr optional in docker so Ranger can run without it#1058
paras200 wants to merge 1 commit into
apache:masterfrom
paras200:RANGER-5680

Conversation

@paras200

@paras200 paras200 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

RANGER-5680 introduces a RANGER_AUDIT_STORE environment variable that selects the audit backend without requiring manual edits to install.properties, mirroring the existing RANGER_DB_TYPE pattern for database selection.

Audit store selection

export RANGER_AUDIT_STORE=opensearch # default — no change needed
export RANGER_AUDIT_STORE=solr # opt-in to Solr

ranger.sh rewrites the active audit block in install.properties at container startup based on RANGER_AUDIT_STORE, so setup.sh picks up the correct configuration. OpenSearch is now the default; Solr remains fully supported via the env-var.

Changes

  • .env: add RANGER_AUDIT_STORE=opensearch as the default
  • docker-compose.ranger.yml: pass RANGER_AUDIT_STORE into the ranger container environment
  • scripts/admin/ranger.sh: rewrite the active audit block in install.properties before setup.sh runs when RANGER_AUDIT_STORE=solr is set (OpenSearch is the no-op default, matching the properties file default)
  • ranger-admin-install-*.properties (postgres/mysql/oracle/sqlserver): flip default active audit block from Solr to OpenSearch; fix inconsistent dangling active elasticsearch param lines in mysql/oracle/sqlserver; add missing OpenSearch block to sqlserver
  • docker-compose.ranger-ozone.yml: add keytab provisioning wait to datanode/scm/om startup commands so they block on /etc/keytabs/.provisioned before attempting Kerberos login — fixes a race where Ozone services started before the KDC had finished generating keytabs
  • CI: switch services-docker-build and plugins-docker-build from docker-compose.ranger-solr.yml to docker-compose.ranger-opensearch.yml, consistent with OpenSearch being the default; update container checklists accordingly

How was this patch tested?

  • docker compose config validated across all documented stack combinations (core, pdp, ozone, all-containers, audit stacks, solr opt-in) — all resolve cleanly
  • Full end-to-end OpenSearch audit pipeline tested locally: OpenSearch index bootstrapped by Ranger Admin on startup, audit posted via ingestor → Kafka → dispatcher → indexed in OpenSearch, visible in Ranger Admin UI (audit_store=opensearch active, no Solr container running)
  • RANGER_AUDIT_STORE=solr path verified: ranger.sh rewrites install.properties correctly and ranger.audit.source.type=solr is written by setup.sh

@ramackri

Copy link
Copy Markdown
Contributor

@paras200
Check build is getting build it seems docker container start up issues

@paras200

Copy link
Copy Markdown
Contributor Author

Thanks @ramackri this is a draft PR, will review once I am marking it ready.
you are right, with removal of solr, this is the error - Error response from daemon: No such container: ranger-solr

@paras200
paras200 force-pushed the RANGER-5680 branch 4 times, most recently from 1229d50 to 3393f80 Compare August 5, 2026 13:06
@paras200
paras200 marked this pull request as ready for review August 5, 2026 13:11
@paras200
paras200 force-pushed the RANGER-5680 branch 2 times, most recently from 987a176 to ccc8ffb Compare August 6, 2026 11:32
@ramackri

ramackri commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

LGTM

@kumaab

kumaab commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Thanks @paras200 for the patch. Few questions:

  • Can we see if the official opensearch image can be used ? https://hub.docker.com/r/opensearchproject/opensearch, instead of building our own image of opensearch in Dockerfile.ranger-opensearch, similar to how it is done for Solr currently.
  • OPENSEARCH_VERSION=1.3.19 version in .env is EOL, why not use OpenSearch (3.x) ?

Thanks

CC: @mneethiraj

@paras200
paras200 force-pushed the RANGER-5680 branch 2 times, most recently from b83ac53 to c7e3364 Compare August 7, 2026 09:13
…e default

Mirrors the RANGER_DB_TYPE pattern so the audit store can be selected
without hand-editing install.properties:

  export RANGER_AUDIT_STORE=solr   # default is now opensearch

Changes:
- .env: add RANGER_AUDIT_STORE=opensearch (default)
- docker-compose.ranger.yml: pass RANGER_AUDIT_STORE into the ranger container env
- scripts/admin/ranger.sh: rewrite install.properties audit block before setup.sh
  runs — acts on RANGER_AUDIT_STORE=solr (opensearch is the no-op default, matching
  the properties file default)
- ranger-admin-install-*.properties (postgres/mysql/oracle/sqlserver): flip default
  active audit block from Solr to OpenSearch; comment out dangling active
  elasticsearch param lines for consistency
- docker-compose.ranger-pdp.yml: remove hard depends_on: ranger-solr
- docker-compose.ranger-ozone.yml: remove hard depends_on: ranger-solr (om service)
- Default bring-up now starts OpenSearch instead of Solr so the core stack matches
  the default audit store; Solr becomes opt-in (RANGER_AUDIT_STORE=solr + the solr
  compose file). Audits use the direct OpenSearchAuditDestination (like the old
  Solr path); the Kafka audit-server pipeline remains an optional advanced flow.
  - README.md: default core/all-containers/rebuild/ozone commands use
    docker-compose.ranger-opensearch.yml; document RANGER_AUDIT_STORE alongside
    RANGER_DB_TYPE; Solr sections kept as opt-in
  - .github/workflows/ci.yml: docker jobs download+build+start ranger-opensearch
    and assert it is running (replacing ranger-solr in the default path)
@paras200

paras200 commented Aug 7, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @kumaab for the review!
Done — switched to opensearchproject/opensearch:2.19.6 directly; custom Dockerfile and startup scripts removed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants