feat: rotate the internal PostgreSQL logs

README.md

@@ -104,6 +104,7 @@ volumes:
 Logs are stored in the `logs/` directory in the persistent directory. In a docker container, it should be at `/nc_app_context_chat_backend/logs/`. The log file is named `ccb.log` and is set to otate at 20 MB with 10 backups. These logs are in JSONL format, i.e. each line is a valid JSON object.
 Now only warning and above logs are printed to the console. All the debug logs are written to the log file if `debug` is set to `true` in the config file.
 The logs of the embedding server are written to `logs/embedding_server_[date].log` in the persistent directory, it rotates with date change and is not in JSONL format, just raw stdout and stderr from the embedding server's process.
+The internal vector database (PostgreSQL) writes its logs to `vector_db_data/pgsql/log/` in the persistent directory, one file per weekday (e.g. `postgresql-Mon.log`); each is overwritten when that weekday comes round again, so the logs are kept for about a week.
 
 ## Configuration
 Configuration resides inside the persistent storage as `config.yaml`. The location is `$APP_PERSISTENT_STORAGE`. By default it would be at `/nc_app_context_chat_backend_data/config.yaml` inside the container.

changelog.md

@@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 - add gh workflows for docker builds and do separate cpu, cuda and rocm (vulkan) images (#295) @kyteinsky
 
 ### Changed
+- rotate the internal PostgreSQL logs into one file per weekday instead of a single ever-growing logfile (#312) @sanzakicesarr
 - update readme according to the latest changes (#300) @kyteinsky
 - bump llama_cpp_python to 0.3.23 (#301) @kyteinsky
 

dockerfile_scripts/pgsql/setup.sh

@@ -11,7 +11,7 @@ source "$(dirname $(realpath $0))/env"
 
 if [ x"$1" == "xstop" ]; then
     echo -n Stopping PostgreSQL...
-    sudo -u postgres ${PG_BIN}/pg_ctl -D "$DATA_DIR" -l "${DATA_DIR}/logfile" -o "-p ${PG_PORT}" stop
+    sudo -u postgres ${PG_BIN}/pg_ctl -D "$DATA_DIR" -o "-p ${PG_PORT}" stop
     exit 0
 fi
 
@@ -47,8 +47,19 @@ if [ ! -d "$DATA_DIR/base" ]; then
     sudo -u postgres ${PG_BIN}/initdb -D "$DATA_DIR" -E UTF8
 fi
 
+# Rotate the internal PostgreSQL logs with the built-in logging collector: one
+# file per weekday (postgresql-Mon.log ... postgresql-Sun.log) under
+# "$DATA_DIR/log". The collector overwrites a weekday file only on time-based
+# rotation, not on startup, so a frequently-restarted container would otherwise
+# append to last week's same-named file. Before starting, drop the pre-rotation
+# single logfile and any weekday log at least six days old (find -mtime +5), so
+# last week's file is always gone before its name is reused. See #303.
+rm -f "${DATA_DIR}/logfile"
+find "${DATA_DIR}/log" -maxdepth 1 -type f -name 'postgresql-*.log' -mtime +5 -delete 2>/dev/null || true
+PG_LOG_OPTS="-c logging_collector=on -c log_directory=log -c log_filename=postgresql-%a.log -c log_rotation_age=1d -c log_rotation_size=0 -c log_truncate_on_rotation=on"
+
 echo "Starting PostgreSQL..."
-sudo -u postgres ${PG_BIN}/pg_ctl -D "$DATA_DIR" -l "${DATA_DIR}/logfile" -o "-p ${PG_PORT}" start
+sudo -u postgres ${PG_BIN}/pg_ctl -D "$DATA_DIR" -o "-p ${PG_PORT} ${PG_LOG_OPTS}" start
 
 echo "Waiting for PostgreSQL to start..."
 until sudo -u postgres ${PG_SQL} -c "SELECT 1" > /dev/null 2>&1; do