Distributed Crawler & Price Tracker
Calisan Servisler
-----------------
- Frontend: http://localhost:3000
- Controller API: http://localhost:8000
- RabbitMQ Panel: http://localhost:15672
- PostgreSQL: localhost:5433
- Worker Pool: docker-compose scale ile yatay olceklenebilir
Klasor Yapisi
-------------
- controller/: FastAPI controller uygulamasi
- worker/: Python crawler worker servisi
- frontend/: React + Vite frontend uygulamasi
- infra/: Veritabani baslangic semasi
- docker-compose.yml: Servis orkestrasyonu
Calistirma
----------
Proje dizinine girin:
cd ~/Desktop/distributed-crawler
Servisleri arka planda baslatin:
docker-compose up -d --build --force-recreate --scale worker=4
Servis durumunu kontrol edin:
docker-compose ps
Backend saglik kontrolu:
curl http://localhost:8000/health
Demo verisini hazirlayin:
curl -X POST http://localhost:8000/demo/setup
Frontend'e girin:
http://localhost:3000
Demo kullanici bilgileri:
Kullanici adi: admin
Sifre: admin
Temel API Endpointleri
----------------------
- GET /health: Controller, veritabani ve RabbitMQ saglik durumu
- POST /enqueue: Tek URL tarama gorevi olusturur
- POST /enqueue/batch: Birden fazla URL gorevi olusturur
- GET /results: Son crawler sonuclarini listeler
- GET /stats: Sistem metriklerini getirir
- GET /products: Takip edilen urunleri listeler
- POST /products/track: Yeni urun takibi baslatir
- GET /notifications: Fiyat ve stok bildirimlerini listeler
- POST /demo/setup: Demo market verisini hazirlar
- GET /pipeline: Kuyruk, worker ve isleme olaylarini getirir
RabbitMQ Panel Bilgileri
------------------------
Adres:
http://localhost:15672
Giris:
Kullanici adi: guest
Sifre: guest
Sorun Giderme
-------------
Container'lar calismiyorsa:
docker-compose down --remove-orphans
docker-compose up -d --build --force-recreate --scale worker=4
Health endpoint cevap vermiyorsa:
docker-compose logs controller
Worker gorev almiyorsa:
docker-compose logs worker
RabbitMQ kuyruğunu kontrol etmek icin:
http://localhost:15672
Projeyi durdurmak icin:
docker-compose down