|
| 1 | +--- |
| 2 | +title: "Adding the dd-orch Database on Upgrade" |
| 3 | +toc_hide: true |
| 4 | +weight: -20260501 |
| 5 | +description: "Provisioning the dojodb-ddorch PostgreSQL database and pointing DefectDojo Pro at it on an existing self-hosted installation." |
| 6 | +audience: pro |
| 7 | +--- |
| 8 | + |
| 9 | +Starting with 2.57.3, DefectDojo Pro requires a second PostgreSQL database, `dojodb-ddorch`, used by the new `ddorch` orchestrator service. The existing `dojodb` database continues to be used by the main Django application. |
| 10 | + |
| 11 | +This guide walks through adding `dojodb-ddorch` to an existing self-hosted PostgreSQL instance and pointing DefectDojo at it. |
| 12 | + |
| 13 | +## Prerequisites |
| 14 | + |
| 15 | +- PostgreSQL 16 is already installed and running on the DB server. |
| 16 | +- The `dojodbusr` role already exists with a known password. |
| 17 | +- `dojodb` is already created and reachable from the DefectDojo app server. |
| 18 | +- `listen_addresses` in `postgresql.conf` is already configured for remote access. |
| 19 | +- You have upgraded to the release that ships the `ddorch` and `ddorch-workers` services. |
| 20 | + |
| 21 | +> **A note on the database name:** `dojodb-ddorch` contains a hyphen, so it must be double-quoted in every SQL statement (`"dojodb-ddorch"`). |
| 22 | +
|
| 23 | +## Part 1: Provision the Database |
| 24 | + |
| 25 | +### 1. Create the new database |
| 26 | + |
| 27 | +On the PostgreSQL server, open a `psql` session as the `postgres` superuser: |
| 28 | + |
| 29 | +```bash |
| 30 | +sudo -i -u postgres psql --username postgres |
| 31 | +``` |
| 32 | + |
| 33 | +Create the database, grant privileges to the existing `dojodbusr` role, and transfer ownership: |
| 34 | + |
| 35 | +```sql |
| 36 | +CREATE DATABASE "dojodb-ddorch"; |
| 37 | +GRANT ALL PRIVILEGES ON DATABASE "dojodb-ddorch" TO dojodbusr; |
| 38 | +ALTER DATABASE "dojodb-ddorch" OWNER TO dojodbusr; |
| 39 | +\q |
| 40 | +``` |
| 41 | + |
| 42 | +**Example session:** |
| 43 | + |
| 44 | +``` |
| 45 | +root@dbserver:~# sudo -i -u postgres psql --username postgres |
| 46 | +psql (16.8) |
| 47 | +Type "help" for help. |
| 48 | +
|
| 49 | +postgres=# CREATE DATABASE "dojodb-ddorch"; |
| 50 | +CREATE DATABASE |
| 51 | +postgres=# GRANT ALL PRIVILEGES ON DATABASE "dojodb-ddorch" TO dojodbusr; |
| 52 | +GRANT |
| 53 | +postgres=# ALTER DATABASE "dojodb-ddorch" OWNER TO dojodbusr; |
| 54 | +ALTER DATABASE |
| 55 | +postgres=# \q |
| 56 | +``` |
| 57 | + |
| 58 | +> **PostgreSQL 15+ note:** Ownership covers schema rights for the owner, but if you ever connect as a non-owner role you will also need to grant schema privileges inside the new database: |
| 59 | +> |
| 60 | +> ```sql |
| 61 | +> \c "dojodb-ddorch" |
| 62 | +> GRANT ALL ON SCHEMA public TO dojodbusr; |
| 63 | +> ``` |
| 64 | +
|
| 65 | +### 2. Allow the app server to connect |
| 66 | +
|
| 67 | +Edit `/etc/postgresql/16/main/pg_hba.conf` and add a new line for `dojodb-ddorch` next to the existing `dojodb` entry. |
| 68 | +
|
| 69 | +**(a) Preferred — restrict to the DefectDojo app server's IP.** |
| 70 | +
|
| 71 | +Supposing the app server's IP is `9.9.9.9`, add: |
| 72 | +
|
| 73 | +``` |
| 74 | +host dojodb-ddorch dojodbusr 9.9.9.9/32 scram-sha-256 |
| 75 | +host postgres dojodbusr 9.9.9.9/32 scram-sha-256 |
| 76 | +``` |
| 77 | +
|
| 78 | +**(b) Alternative — allow from any host.** |
| 79 | + |
| 80 | +``` |
| 81 | +host dojodb-ddorch dojodbusr 0.0.0.0/0 scram-sha-256 |
| 82 | +host postgres dojodbusr 0.0.0.0/0 scram-sha-256 |
| 83 | +``` |
| 84 | + |
| 85 | +> **Note:** The lines in `pg_hba.conf` are whitespace-delimited. The easiest way to add this line is to copy/paste the existing `dojodb` line and change the database name. |
| 86 | +
|
| 87 | +**Alternative using `echo` (if no text editor is available):** |
| 88 | + |
| 89 | +```bash |
| 90 | +# For specific IP (replace 9.9.9.9 with your app server IP): |
| 91 | +echo "host dojodb-ddorch dojodbusr 9.9.9.9/32 scram-sha-256" | sudo tee -a /etc/postgresql/16/main/pg_hba.conf |
| 92 | +echo "host postgres dojodbusr 9.9.9.9/32 scram-sha-256" | sudo tee -a /etc/postgresql/16/main/pg_hba.conf |
| 93 | + |
| 94 | +# OR for all hosts: |
| 95 | +echo "host dojodb-ddorch dojodbusr 0.0.0.0/0 scram-sha-256" | sudo tee -a /etc/postgresql/16/main/pg_hba.conf |
| 96 | +echo "host postgres dojodbusr 0.0.0.0/0 scram-sha-256" | sudo tee -a /etc/postgresql/16/main/pg_hba.conf |
| 97 | + |
| 98 | +``` |
| 99 | + |
| 100 | +### 3. Reload PostgreSQL |
| 101 | + |
| 102 | +Changes to `pg_hba.conf` only require a reload — no restart is needed: |
| 103 | + |
| 104 | +```bash |
| 105 | +sudo systemctl reload postgresql |
| 106 | +``` |
| 107 | + |
| 108 | +Verify the reload was picked up: |
| 109 | + |
| 110 | +```bash |
| 111 | +sudo systemctl status postgresql |
| 112 | +``` |
| 113 | + |
| 114 | +### 4. Verify connectivity from the app server |
| 115 | + |
| 116 | +From the **DefectDojo app server**, confirm `dojodbusr` can reach the new database. Replace `<db-server-ip>` with your DB server's IP and `<password>` with the password set for `dojodbusr`: |
| 117 | + |
| 118 | +```bash |
| 119 | +psql "host=<db-server-ip> dbname=dojodb-ddorch user=dojodbusr password=<password>" -c "SELECT 1;" |
| 120 | +``` |
| 121 | + |
| 122 | +A successful response of `?column?` with a value of `1` confirms the database is reachable and the credentials are valid. |
| 123 | + |
| 124 | +## Part 2: Point DefectDojo at the New Database |
| 125 | + |
| 126 | +Only the `ddorch` service connects to the new database directly. The main Django application reaches the orchestrator over gRPC, so `DD_DATABASE_URL` does **not** change. |
| 127 | + |
| 128 | +### 1. Set the orchestrator database URL |
| 129 | + |
| 130 | +The `ddorch` service reads its connection string from the `DD_DATABASE_URL` environment variable and **automatically appends `-ddorch` to the database name** in whatever URL you pass it. This means you can reuse the same connection string you already use for the main Django application — no need to construct a second URL by hand. |
| 131 | + |
| 132 | +On startup, ddorch rewrites the database name in this URL from `dojodb` to `dojodb-ddorch` and connects to the database you created in Part 1. |
| 133 | + |
| 134 | +### 2. Restart the orchestrator services |
| 135 | + |
| 136 | +From the deployment directory, recreate the two orchestrator containers so they pick up the new environment: |
| 137 | + |
| 138 | +```bash |
| 139 | +docker compose up -d ddorch ddorch-workers |
| 140 | +``` |
| 141 | + |
| 142 | +Docker Compose will detect the environment change and recreate the containers. The `ddorch` service runs its own schema migrations against `dojodb-ddorch` on startup — no manual migration command is required. |
| 143 | + |
| 144 | +### 3. Verify ddorch connected and migrated the new database |
| 145 | + |
| 146 | +The most direct signal that the database is correctly wired up is the ddorch startup log. Check the last hundred lines: |
| 147 | + |
| 148 | +```bash |
| 149 | +docker compose logs ddorch --tail=100 |
| 150 | +``` |
| 151 | + |
| 152 | +Look for three log lines in sequence: |
| 153 | + |
| 154 | +``` |
| 155 | +{"level":"INFO","msg":"Appending database name to DATABASE_URL","from":"dojodb","to":"dojodb-ddorch"} |
| 156 | +INFO Running migrations current_schema_version=<N> next_version=<M> migrations_to_apply=<K> |
| 157 | +{"level":"INFO","msg":"starting server","port":9871} |
| 158 | +``` |
| 159 | + |
| 160 | +What each line proves: |
| 161 | + |
| 162 | +- **`Appending database name to DATABASE_URL ... to: dojodb-ddorch`** — ddorch received your URL and derived the orch database name correctly. |
| 163 | +- **`Running migrations ... migrations_to_apply=0`** — ddorch connected to `dojodb-ddorch` and found the schema at the expected version. On a first-ever boot against a fresh database you may see `migrations_to_apply=<N>` with a non-zero value and no subsequent error — this means ddorch just created the tables from scratch. Both outcomes indicate success. |
| 164 | +- **`starting server ... port:9871`** — ddorch is up and listening. |
| 165 | + |
| 166 | +If instead you see an error such as `FATAL: password authentication failed`, `no pg_hba.conf entry for host`, or `database "dojodb-ddorch" does not exist`, the database is not reachable — revisit Part 1 before proceeding. |
| 167 | + |
| 168 | +Also confirm both orchestrator containers are running: |
| 169 | + |
| 170 | +```bash |
| 171 | +docker compose ps ddorch ddorch-workers |
| 172 | +``` |
| 173 | + |
| 174 | +Both should report `Up`. With ddorch migrated and the workers container running, your installation is now using the new `dojodb-ddorch` database. |
0 commit comments