Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -90,8 +90,9 @@ MINIO_SECRET_KEY=minioadmin1234
# OLLAMA_SERVER_CONNECT_TIMEOUT=3s
# OLLAMA_SERVER_READ_TIMEOUT=18s

# Redis (로그아웃 토큰 블랙리스트) - application.yml에 이미 기본값(localhost:6379, 인증 없음)이 있어
# Valkey (로그아웃 토큰 블랙리스트) - application.yml에 이미 기본값(localhost:6379, 인증 없음)이 있어
# docker-compose 기본 포트를 쓰면 설정 불필요. 기본값과 다를 때만 주석 해제
# Valkey는 Redis 프로토콜과 호환되고 Spring Data Redis가 접속하므로 변수명은 REDIS_*를 그대로 사용합니다.
# REDIS_HOST=localhost
# REDIS_PORT=6379
# REDIS_PASSWORD=
Expand Down
16 changes: 8 additions & 8 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ DocGrid는 조직에 흩어진 PDF·DOCX 문서를 자동으로 인덱싱하고,
| Database | OpenSQL 17.8, PostgreSQL 17, pgvector 0.8.1 |
| AI | BAAI/bge-m3, Ollama, qwen2.5:7b |
| Storage | Local Filesystem, MinIO, AWS S3 |
| Cache | Redis 7 |
| Cache | Valkey 9.1 (Redis 프로토콜 호환, 클라이언트: Spring Data Redis) |
| Observability | RAGOps Dashboard, Prometheus |
| Test | JUnit 5, Gradle, Node.js Test Runner |

Expand Down Expand Up @@ -69,10 +69,10 @@ npm --prefix frontend install
`.env.example`은 로컬 PostgreSQL, MinIO와 자동 인덱싱 Worker를 실행할 수 있는 기본값을 제공합니다.
실제 비밀번호와 외부 서버 인증정보는 `.env`, README, Issue 또는 Commit에 기록하지 않습니다.

### 3. PostgreSQL, MinIO, Redis, BGE-M3 실행
### 3. PostgreSQL, MinIO, Valkey, BGE-M3 실행

```bash
docker compose up -d --build postgres minio redis embedding-server
docker compose up -d --build postgres minio valkey embedding-server
docker compose logs -f embedding-server
```

Expand All @@ -96,7 +96,7 @@ ollama pull qwen2.5:7b
ollama run qwen2.5:7b "안녕"
```

초기 설치 후에는 Redis, BGE-M3, Ollama를 전용 스크립트로 한 번에 기동하고 실제 연결 상태까지 확인합니다.
초기 설치 후에는 Valkey, BGE-M3, Ollama를 전용 스크립트로 한 번에 기동하고 실제 연결 상태까지 확인합니다.

```bash
./scripts/local-services.sh start
Expand Down Expand Up @@ -147,7 +147,7 @@ Embedding Provider가 준비되지 않아도 먼저 기동해 수집 실패와
### 종료

```bash
docker compose stop postgres minio redis embedding-server
docker compose stop postgres minio valkey embedding-server
# macOS에서 Homebrew로 Ollama를 실행한 경우
brew services stop ollama
```
Expand Down Expand Up @@ -180,19 +180,19 @@ SSH Key는 저장소에 포함하지 않습니다. 환경별 설정은 [.env.exa

## 테스트

일반 회귀 테스트는 외부 장시간 Benchmark와 실제 인프라 E2E를 제외하지만 PostgreSQL과 Redis를
일반 회귀 테스트는 외부 장시간 Benchmark와 실제 인프라 E2E를 제외하지만 PostgreSQL과 Valkey를
사용하는 통합 테스트를 포함합니다. 두 Service를 먼저 준비합니다.

```bash
docker compose up -d --wait postgres redis
docker compose up -d --wait postgres valkey
./backend/gradlew -p backend test
npm --prefix frontend test
```

파일 저장소 Adapter와 실제 자동 Worker 전체 흐름은 별도 E2E로 검증합니다.

```bash
docker compose up -d --build --wait postgres minio redis embedding-server
docker compose up -d --build --wait postgres minio valkey embedding-server

DB_HOST=127.0.0.1 DB_PORT=55432 \
./backend/gradlew -p backend storageWorkerE2eTest
Expand Down
12 changes: 6 additions & 6 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,16 +118,16 @@ services:
networks:
- docgrid-local

redis:
image: redis:7-alpine
container_name: docgrid-redis
valkey:
image: valkey/valkey:9.1-alpine
container_name: docgrid-valkey
restart: unless-stopped
ports:
- "127.0.0.1:${REDIS_PORT:-6379}:6379"
volumes:
- redis-data:/data
- valkey-data:/data
healthcheck:
test: ["CMD", "redis-cli", "ping"]
test: ["CMD", "valkey-cli", "ping"]
interval: 10s
timeout: 5s
retries: 5
Expand All @@ -142,6 +142,6 @@ volumes:
name: docgrid_postgres17_data
minio-data:
huggingface-cache:
redis-data:
valkey-data:
Comment thread
coderabbitai[bot] marked this conversation as resolved.
prometheus-data:
alertmanager-data:
320 changes: 320 additions & 0 deletions docs/design/kangcheolung-#399-valkey-migration.md

Large diffs are not rendered by default.

221 changes: 221 additions & 0 deletions docs/test-results/evidence/issue-399/full-redis-7.4.10-summary.txt

Large diffs are not rendered by default.

221 changes: 221 additions & 0 deletions docs/test-results/evidence/issue-399/full-valkey-9.1.2-summary.txt

Large diffs are not rendered by default.

26 changes: 26 additions & 0 deletions docs/test-results/evidence/issue-399/summarize.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
"""Gradle 테스트 결과 XML을 클래스별로 집계한다.

사용법: python3 summarize.py <build/test-results/test 디렉터리>
"""
import glob
import sys
import xml.etree.ElementTree as ET

if len(sys.argv) != 2:
sys.exit("사용법: python3 summarize.py <build/test-results/test 디렉터리>")

directory = sys.argv[1]
paths = sorted(glob.glob(directory + "/*.xml"))
if not paths:
sys.exit(f"테스트 결과 XML이 없습니다: {directory}")

totals = [0, 0, 0, 0]
rows = []
for path in paths:
root = ET.parse(path).getroot()
counts = [int(root.get(key, 0)) for key in ("tests", "failures", "errors", "skipped")]
totals = [a + b for a, b in zip(totals, counts)]
rows.append((root.get("name").split(".")[-1], counts))
for name, c in rows:
print(f"{name:70s} tests={c[0]:<4} fail={c[1]:<3} err={c[2]:<3} skipped={c[3]}")
print("합계 tests=%d failures=%d errors=%d skipped=%d" % tuple(totals))
205 changes: 205 additions & 0 deletions docs/test-results/evidence/issue-399/verify-e2e.log
Original file line number Diff line number Diff line change
@@ -0,0 +1,205 @@
Unable to find image 'prom/prometheus:v3.5.5' locally
v3.5.5: Pulling from prom/prometheus
a03b0fa940b1: Pulling fs layer
7a21057f59f5: Pulling fs layer
905e8aa01b13: Pulling fs layer
aa2b3c16fb92: Pulling fs layer
788b085682b4: Pulling fs layer
66d02544ade1: Pulling fs layer
55493eaebe55: Pulling fs layer
b718c71eb0d8: Pulling fs layer
d7a574078986: Pulling fs layer
3d49b2164575: Pulling fs layer
a03b0fa940b1: Download complete
905e8aa01b13: Download complete
3d49b2164575: Download complete
aa2b3c16fb92: Download complete
55493eaebe55: Download complete
7a21057f59f5: Download complete
788b085682b4: Download complete
d7a574078986: Download complete
905e8aa01b13: Pull complete
7a21057f59f5: Pull complete
66d02544ade1: Download complete
66d02544ade1: Pull complete
b718c71eb0d8: Download complete
a03b0fa940b1: Pull complete
3d49b2164575: Pull complete
aa2b3c16fb92: Pull complete
55493eaebe55: Pull complete
788b085682b4: Pull complete
d7a574078986: Pull complete
b718c71eb0d8: Pull complete
Digest: sha256:332c2f43e7e389d74d3893b55bb02fbbd684208e681eeb604641d5d769c0fe2a
Status: Downloaded newer image for prom/prometheus:v3.5.5
Checking /etc/prometheus/prometheus.yml
SUCCESS: 3 rule files found
SUCCESS: /etc/prometheus/prometheus.yml is valid prometheus config file syntax

Checking /etc/prometheus/rules/docgrid-backend-alerts.yml
SUCCESS: 3 rules found

Checking /etc/prometheus/rules/docgrid-pipeline-alerts.yml
SUCCESS: 9 rules found

Checking /etc/prometheus/rules/embedding-provider-alerts.yml
SUCCESS: 7 rules found

Checking /etc/prometheus/rules/embedding-provider-alerts.yml
SUCCESS: 7 rules found

Checking /etc/prometheus/rules/docgrid-backend-alerts.yml
SUCCESS: 3 rules found

Checking /etc/prometheus/rules/docgrid-pipeline-alerts.yml
SUCCESS: 9 rules found

SUCCESS

SUCCESS

SUCCESS

Checking /test/rules-firing.yml
SUCCESS: 3 rules found

Checking /test/rules-resolved.yml
SUCCESS: 3 rules found

Unable to find image 'quay.io/prometheus/alertmanager:v0.33.1' locally
v0.33.1: Pulling from prometheus/alertmanager
4f4fb700ef54: Pulling fs layer
bc568a425d6c: Pulling fs layer
b206c4e2f56d: Pulling fs layer
91b78d9e43f9: Pulling fs layer
b90af14651b3: Pulling fs layer
4f4fb700ef54: Already exists
b90af14651b3: Download complete
91b78d9e43f9: Download complete
bc568a425d6c: Download complete
bc568a425d6c: Pull complete
b206c4e2f56d: Download complete
4f4fb700ef54: Pull complete
b90af14651b3: Pull complete
91b78d9e43f9: Pull complete
b206c4e2f56d: Pull complete
Digest: sha256:9e082985f56f4c8c9f724e18f2288c6708f472e56a5286b8863d080434ea065d
Status: Downloaded newer image for quay.io/prometheus/alertmanager:v0.33.1
Checking '/etc/alertmanager/alertmanager.yml' SUCCESS
Found:
- global config
- route
- 2 inhibit rules
- 1 receivers
- 0 templates

Checking '/etc/alertmanager/alertmanager.yml' SUCCESS
Found:
- global config
- route
- 1 inhibit rules
- 1 receivers
- 0 templates

Checking '/etc/alertmanager/alertmanager.yml' SUCCESS
Found:
- global config
- route
- 2 inhibit rules
- 1 receivers
- 0 templates

Checking '/etc/alertmanager/alertmanager.yml' SUCCESS
Found:
- global config
- route
- 2 inhibit rules
- 1 receivers
- 0 templates

Checking '/etc/alertmanager/alertmanager.yml' SUCCESS
Found:
- global config
- route
- 2 inhibit rules
- 1 receivers
- 0 templates

Checking '/etc/alertmanager/alertmanager.yml' SUCCESS
Found:
- global config
- route
- 2 inhibit rules
- 1 receivers
- 0 templates

Checking '/etc/alertmanager/alertmanager.yml' SUCCESS
Found:
- global config
- route
- 2 inhibit rules
- 1 receivers
- 0 templates

Checking '/etc/alertmanager/alertmanager.yml' SUCCESS
Found:
- global config
- route
- 2 inhibit rules
- 1 receivers
- 0 templates

Checking '/etc/alertmanager/alertmanager.yml' SUCCESS
Found:
- global config
- route
- 2 inhibit rules
- 1 receivers
- 0 templates

webhook-receiver Pulling
6db16a24faac Pulling fs layer
4617d3b35936 Pulling fs layer
02bd2e91174a Pulling fs layer
02bd2e91174a Downloading [==================================================>] 419.1kB/419.1kB
02bd2e91174a Download complete
02bd2e91174a Pull complete
6db16a24faac Download complete
4617d3b35936 Downloading [======> ] 2.097MB/15.65MB
4617d3b35936 Downloading [==========================> ] 8.389MB/15.65MB
4617d3b35936 Downloading [===========================================> ] 13.63MB/15.65MB
4617d3b35936 Download complete
4617d3b35936 Extracting 1 s
4617d3b35936 Extracting 1 s
4617d3b35936 Extracting 1 s
4617d3b35936 Extracting 1 s
6db16a24faac Pull complete
4617d3b35936 Pull complete
webhook-receiver Pulled
Network docgrid-alertmanager-e2e-431_default Creating
Network docgrid-alertmanager-e2e-431_default Created
Container docgrid-alertmanager-e2e-431-webhook-receiver-1 Creating
Container docgrid-alertmanager-e2e-431-webhook-receiver-1 Created
Container docgrid-alertmanager-e2e-431-alertmanager-1 Creating
Container docgrid-alertmanager-e2e-431-alertmanager-1 Created
Container docgrid-alertmanager-e2e-431-prometheus-1 Creating
Container docgrid-alertmanager-e2e-431-prometheus-1 Created
Container docgrid-alertmanager-e2e-431-webhook-receiver-1 Starting
Container docgrid-alertmanager-e2e-431-webhook-receiver-1 Started
Container docgrid-alertmanager-e2e-431-webhook-receiver-1 Waiting
Container docgrid-alertmanager-e2e-431-webhook-receiver-1 Healthy
Container docgrid-alertmanager-e2e-431-alertmanager-1 Starting
Container docgrid-alertmanager-e2e-431-alertmanager-1 Started
Container docgrid-alertmanager-e2e-431-alertmanager-1 Waiting
Container docgrid-alertmanager-e2e-431-alertmanager-1 Healthy
Container docgrid-alertmanager-e2e-431-prometheus-1 Starting
Container docgrid-alertmanager-e2e-431-prometheus-1 Started
Container docgrid-alertmanager-e2e-431-webhook-receiver-1 Waiting
Container docgrid-alertmanager-e2e-431-alertmanager-1 Waiting
Container docgrid-alertmanager-e2e-431-prometheus-1 Waiting
Container docgrid-alertmanager-e2e-431-alertmanager-1 Healthy
Container docgrid-alertmanager-e2e-431-webhook-receiver-1 Healthy
Container docgrid-alertmanager-e2e-431-prometheus-1 Healthy
Alertmanager routing E2E: SUCCESS (firing 2s, resolved 2s)
Monitoring configuration validation: SUCCESS
verify exit=0
6 changes: 3 additions & 3 deletions monitoring/drills/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@ services:
timeout: 2s
retries: 30

redis:
image: redis:7-alpine
valkey:
image: valkey/valkey:9.1-alpine
ports:
- "127.0.0.1::6379"
healthcheck:
test: ["CMD", "redis-cli", "ping"]
test: ["CMD", "valkey-cli", "ping"]
interval: 1s
timeout: 1s
retries: 30
Expand Down
4 changes: 2 additions & 2 deletions monitoring/drills/run_drill.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,14 +159,14 @@ def compose(self, *arguments, capture=True, check=True):

def start_core(self, *, include_provider):
"""Start fresh database, cache, local receiver, Alertmanager, and optionally BGE-M3."""
services = ["postgres", "redis", "webhook-receiver", "alertmanager"]
services = ["postgres", "valkey", "webhook-receiver", "alertmanager"]
if include_provider:
run(["docker", "volume", "create", "docgrid_huggingface-cache"])
services.append("embedding-server")
print(f"Starting isolated services for {self.scenario}...", flush=True)
self.compose("up", "-d", "--wait", *services, capture=False)
self.postgres_port = self.service_port("postgres", 5432)
self.redis_port = self.service_port("redis", 6379)
self.redis_port = self.service_port("valkey", 6379)
self.alertmanager_url = f"http://127.0.0.1:{self.service_port('alertmanager', 9093)}"
if include_provider:
self.embedding_port = self.service_port("embedding-server", 8000)
Expand Down
Loading
Loading