Skip to content

Commit a7abba8

Browse files
paulOsinskiMaffooch
paulOsinski
and
Maffooch
authored
[pro] add DD-Orchestrator upgrade instructions for on-prem customers (#14747)
* add ddorch notes * update * Apply suggestions from code review Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com> --------- Co-authored-by: Cody Maffucci <46459665+Maffooch@users.noreply.github.com>
1 parent 88144dc commit a7abba8

1 file changed

Lines changed: 174 additions & 0 deletions

File tree

‎docs/content/releases/pro/ddorch-database.md‎

Lines changed: 174 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,174 @@
1+
---
2+
title: "Adding the dd-orch Database on Upgrade"
3+
toc_hide: true
4+
weight: -20260501
5+
description: "Provisioning the dojodb-ddorch PostgreSQL database and pointing DefectDojo Pro at it on an existing self-hosted installation."
6+
audience: pro
7+
---
8+
9+
Starting with 2.57.3, DefectDojo Pro requires a second PostgreSQL database, `dojodb-ddorch`, used by the new `ddorch` orchestrator service. The existing `dojodb` database continues to be used by the main Django application.
10+
11+
This guide walks through adding `dojodb-ddorch` to an existing self-hosted PostgreSQL instance and pointing DefectDojo at it.
12+
13+
## Prerequisites
14+
15+
- PostgreSQL 16 is already installed and running on the DB server.
16+
- The `dojodbusr` role already exists with a known password.
17+
- `dojodb` is already created and reachable from the DefectDojo app server.
18+
- `listen_addresses` in `postgresql.conf` is already configured for remote access.
19+
- You have upgraded to the release that ships the `ddorch` and `ddorch-workers` services.
20+
21+
> **A note on the database name:** `dojodb-ddorch` contains a hyphen, so it must be double-quoted in every SQL statement (`"dojodb-ddorch"`).
22+
23+
## Part 1: Provision the Database
24+
25+
### 1. Create the new database
26+
27+
On the PostgreSQL server, open a `psql` session as the `postgres` superuser:
28+
29+
```bash
30+
sudo -i -u postgres psql --username postgres
31+
```
32+
33+
Create the database, grant privileges to the existing `dojodbusr` role, and transfer ownership:
34+
35+
```sql
36+
CREATE DATABASE "dojodb-ddorch";
37+
GRANT ALL PRIVILEGES ON DATABASE "dojodb-ddorch" TO dojodbusr;
38+
ALTER DATABASE "dojodb-ddorch" OWNER TO dojodbusr;
39+
\q
40+
```
41+
42+
**Example session:**
43+
44+
```
45+
root@dbserver:~# sudo -i -u postgres psql --username postgres
46+
psql (16.8)
47+
Type "help" for help.
48+
49+
postgres=# CREATE DATABASE "dojodb-ddorch";
50+
CREATE DATABASE
51+
postgres=# GRANT ALL PRIVILEGES ON DATABASE "dojodb-ddorch" TO dojodbusr;
52+
GRANT
53+
postgres=# ALTER DATABASE "dojodb-ddorch" OWNER TO dojodbusr;
54+
ALTER DATABASE
55+
postgres=# \q
56+
```
57+
58+
> **PostgreSQL 15+ note:** Ownership covers schema rights for the owner, but if you ever connect as a non-owner role you will also need to grant schema privileges inside the new database:
59+
>
60+
> ```sql
61+
> \c "dojodb-ddorch"
62+
> GRANT ALL ON SCHEMA public TO dojodbusr;
63+
> ```
64+
65+
### 2. Allow the app server to connect
66+
67+
Edit `/etc/postgresql/16/main/pg_hba.conf` and add a new line for `dojodb-ddorch` next to the existing `dojodb` entry.
68+
69+
**(a) Preferred — restrict to the DefectDojo app server's IP.**
70+
71+
Supposing the app server's IP is `9.9.9.9`, add:
72+
73+
```
74+
host dojodb-ddorch dojodbusr 9.9.9.9/32 scram-sha-256
75+
host postgres dojodbusr 9.9.9.9/32 scram-sha-256
76+
```
77+
78+
**(b) Alternative — allow from any host.**
79+
80+
```
81+
host dojodb-ddorch dojodbusr 0.0.0.0/0 scram-sha-256
82+
host postgres dojodbusr 0.0.0.0/0 scram-sha-256
83+
```
84+
85+
> **Note:** The lines in `pg_hba.conf` are whitespace-delimited. The easiest way to add this line is to copy/paste the existing `dojodb` line and change the database name.
86+
87+
**Alternative using `echo` (if no text editor is available):**
88+
89+
```bash
90+
# For specific IP (replace 9.9.9.9 with your app server IP):
91+
echo "host dojodb-ddorch dojodbusr 9.9.9.9/32 scram-sha-256" | sudo tee -a /etc/postgresql/16/main/pg_hba.conf
92+
echo "host postgres dojodbusr 9.9.9.9/32 scram-sha-256" | sudo tee -a /etc/postgresql/16/main/pg_hba.conf
93+
94+
# OR for all hosts:
95+
echo "host dojodb-ddorch dojodbusr 0.0.0.0/0 scram-sha-256" | sudo tee -a /etc/postgresql/16/main/pg_hba.conf
96+
echo "host postgres dojodbusr 0.0.0.0/0 scram-sha-256" | sudo tee -a /etc/postgresql/16/main/pg_hba.conf
97+
98+
```
99+
100+
### 3. Reload PostgreSQL
101+
102+
Changes to `pg_hba.conf` only require a reload — no restart is needed:
103+
104+
```bash
105+
sudo systemctl reload postgresql
106+
```
107+
108+
Verify the reload was picked up:
109+
110+
```bash
111+
sudo systemctl status postgresql
112+
```
113+
114+
### 4. Verify connectivity from the app server
115+
116+
From the **DefectDojo app server**, confirm `dojodbusr` can reach the new database. Replace `<db-server-ip>` with your DB server's IP and `<password>` with the password set for `dojodbusr`:
117+
118+
```bash
119+
psql "host=<db-server-ip> dbname=dojodb-ddorch user=dojodbusr password=<password>" -c "SELECT 1;"
120+
```
121+
122+
A successful response of `?column?` with a value of `1` confirms the database is reachable and the credentials are valid.
123+
124+
## Part 2: Point DefectDojo at the New Database
125+
126+
Only the `ddorch` service connects to the new database directly. The main Django application reaches the orchestrator over gRPC, so `DD_DATABASE_URL` does **not** change.
127+
128+
### 1. Set the orchestrator database URL
129+
130+
The `ddorch` service reads its connection string from the `DD_DATABASE_URL` environment variable and **automatically appends `-ddorch` to the database name** in whatever URL you pass it. This means you can reuse the same connection string you already use for the main Django application — no need to construct a second URL by hand.
131+
132+
On startup, ddorch rewrites the database name in this URL from `dojodb` to `dojodb-ddorch` and connects to the database you created in Part 1.
133+
134+
### 2. Restart the orchestrator services
135+
136+
From the deployment directory, recreate the two orchestrator containers so they pick up the new environment:
137+
138+
```bash
139+
docker compose up -d ddorch ddorch-workers
140+
```
141+
142+
Docker Compose will detect the environment change and recreate the containers. The `ddorch` service runs its own schema migrations against `dojodb-ddorch` on startup — no manual migration command is required.
143+
144+
### 3. Verify ddorch connected and migrated the new database
145+
146+
The most direct signal that the database is correctly wired up is the ddorch startup log. Check the last hundred lines:
147+
148+
```bash
149+
docker compose logs ddorch --tail=100
150+
```
151+
152+
Look for three log lines in sequence:
153+
154+
```
155+
{"level":"INFO","msg":"Appending database name to DATABASE_URL","from":"dojodb","to":"dojodb-ddorch"}
156+
INFO Running migrations current_schema_version=<N> next_version=<M> migrations_to_apply=<K>
157+
{"level":"INFO","msg":"starting server","port":9871}
158+
```
159+
160+
What each line proves:
161+
162+
- **`Appending database name to DATABASE_URL ... to: dojodb-ddorch`** — ddorch received your URL and derived the orch database name correctly.
163+
- **`Running migrations ... migrations_to_apply=0`** — ddorch connected to `dojodb-ddorch` and found the schema at the expected version. On a first-ever boot against a fresh database you may see `migrations_to_apply=<N>` with a non-zero value and no subsequent error — this means ddorch just created the tables from scratch. Both outcomes indicate success.
164+
- **`starting server ... port:9871`** — ddorch is up and listening.
165+
166+
If instead you see an error such as `FATAL: password authentication failed`, `no pg_hba.conf entry for host`, or `database "dojodb-ddorch" does not exist`, the database is not reachable — revisit Part 1 before proceeding.
167+
168+
Also confirm both orchestrator containers are running:
169+
170+
```bash
171+
docker compose ps ddorch ddorch-workers
172+
```
173+
174+
Both should report `Up`. With ddorch migrated and the workers container running, your installation is now using the new `dojodb-ddorch` database.

0 commit comments

Comments
 (0)