diff --git a/README.md b/README.md index 0cc3b450..a0303183 100644 --- a/README.md +++ b/README.md @@ -104,6 +104,7 @@ volumes: Logs are stored in the `logs/` directory in the persistent directory. In a docker container, it should be at `/nc_app_context_chat_backend/logs/`. The log file is named `ccb.log` and is set to otate at 20 MB with 10 backups. These logs are in JSONL format, i.e. each line is a valid JSON object. Now only warning and above logs are printed to the console. All the debug logs are written to the log file if `debug` is set to `true` in the config file. The logs of the embedding server are written to `logs/embedding_server_[date].log` in the persistent directory, it rotates with date change and is not in JSONL format, just raw stdout and stderr from the embedding server's process. +The internal vector database (PostgreSQL) writes its logs to `vector_db_data/pgsql/log/` in the persistent directory, one file per weekday (e.g. `postgresql-Mon.log`); each is overwritten when that weekday comes round again, so the logs are kept for about a week. ## Configuration Configuration resides inside the persistent storage as `config.yaml`. The location is `$APP_PERSISTENT_STORAGE`. By default it would be at `/nc_app_context_chat_backend_data/config.yaml` inside the container. diff --git a/changelog.md b/changelog.md index e5e5b063..b0f5e18c 100644 --- a/changelog.md +++ b/changelog.md @@ -17,6 +17,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/). - add gh workflows for docker builds and do separate cpu, cuda and rocm (vulkan) images (#295) @kyteinsky ### Changed +- rotate the internal PostgreSQL logs into one file per weekday instead of a single ever-growing logfile (#312) @sanzakicesarr - update readme according to the latest changes (#300) @kyteinsky - bump llama_cpp_python to 0.3.23 (#301) @kyteinsky diff --git a/dockerfile_scripts/pgsql/setup.sh b/dockerfile_scripts/pgsql/setup.sh index 7578ed83..ccae73ce 100755 --- a/dockerfile_scripts/pgsql/setup.sh +++ b/dockerfile_scripts/pgsql/setup.sh @@ -11,7 +11,7 @@ source "$(dirname $(realpath $0))/env" if [ x"$1" == "xstop" ]; then echo -n Stopping PostgreSQL... - sudo -u postgres ${PG_BIN}/pg_ctl -D "$DATA_DIR" -l "${DATA_DIR}/logfile" -o "-p ${PG_PORT}" stop + sudo -u postgres ${PG_BIN}/pg_ctl -D "$DATA_DIR" -o "-p ${PG_PORT}" stop exit 0 fi @@ -47,8 +47,19 @@ if [ ! -d "$DATA_DIR/base" ]; then sudo -u postgres ${PG_BIN}/initdb -D "$DATA_DIR" -E UTF8 fi +# Rotate the internal PostgreSQL logs with the built-in logging collector: one +# file per weekday (postgresql-Mon.log ... postgresql-Sun.log) under +# "$DATA_DIR/log". The collector overwrites a weekday file only on time-based +# rotation, not on startup, so a frequently-restarted container would otherwise +# append to last week's same-named file. Before starting, drop the pre-rotation +# single logfile and any weekday log at least six days old (find -mtime +5), so +# last week's file is always gone before its name is reused. See #303. +rm -f "${DATA_DIR}/logfile" +find "${DATA_DIR}/log" -maxdepth 1 -type f -name 'postgresql-*.log' -mtime +5 -delete 2>/dev/null || true +PG_LOG_OPTS="-c logging_collector=on -c log_directory=log -c log_filename=postgresql-%a.log -c log_rotation_age=1d -c log_rotation_size=0 -c log_truncate_on_rotation=on" + echo "Starting PostgreSQL..." -sudo -u postgres ${PG_BIN}/pg_ctl -D "$DATA_DIR" -l "${DATA_DIR}/logfile" -o "-p ${PG_PORT}" start +sudo -u postgres ${PG_BIN}/pg_ctl -D "$DATA_DIR" -o "-p ${PG_PORT} ${PG_LOG_OPTS}" start echo "Waiting for PostgreSQL to start..." until sudo -u postgres ${PG_SQL} -c "SELECT 1" > /dev/null 2>&1; do