CSB 311 Data Frontend (Excellence Project)

[bhuelsman]

Overview

Adds a React-based frontend dashboard that visualizes CSB 311 service request data, making it accessible to non-technical users (residents, city officials, community organizers) without requiring API knowledge.
Live Demo: http://localhost:5001/ (after setup)

New Files

• frontend/csb.htm - Interactive HTML/CSS/JS dashboard

  • Displays active 311 service requests with filtering
  • Search by service name, description, or neighborhood
  • Filter by ward and service type
  • Real-time statistics (total requests, service types, wards)
  • Responsive design (mobile-friendly)

• src/write_service/consumers/csb_full_pipeline.py - Complete data pipeline

  • Downloads CSB data ZIP from St. Louis Open Data portal
  • Extracts and processes CSV
  • Loads into PostgreSQL database
  • Uses temporary directory (auto-cleanup)
  • Marks old data inactive, new data active
  • Single command execution (~30 seconds)

Modified Files

• frontend/index.htm - Added links to both ARPA and CSB dashboards
• src/read_service/app.py - Added /csb-dashboard route, organized code with comments
• src/read_service/processors/arpa_processor.py - Fixed import path issues
• src/write_service/consumers/models.py - Consolidated dynamic table creation from sqlalchemy.py
• src/write_service/consumers/json_consumer.py - Updated imports after consolidation

Removed Files

• src/write_service/consumers/sqlalchemy.py - Merged into models.py to resolve import conflicts

How to Run

1. Load CSB Data (one-time setup)

cd src/write_service/consumers
python3 csb_full_pipeline.py

Output: ✓ Pipeline complete! Active records: 92

2. Start Backend

cd src/read_service
python3 app.py

Server runs on: http://localhost:5001

3. Access the Application

For Non-Technical Users:

• Main page: http://localhost:5001/
  • Landing page with links to all dashboards
• CSB Dashboard: http://localhost:5001/csb-dashboard
  • Interactive visualization of 311 service requests
  • Filter by ward, service type, search terms
  • View statistics and active requests

For Developers:

• CSB API: http://localhost:5001/csb
  • Returns JSON array of active service requests
• ARPA API: http://localhost:5001/api/arpa
  • Returns ARPA funding data
• Swagger Docs: http://localhost:5001/swagger
  • Complete API documentation

Dashboard Capabilities

• Real-time filtering: Search across service names, descriptions, and neighborhoods
• Ward filtering: View requests by specific ward (01-14)
• Service type filtering: Filter by service categories (sidewalk repair, tree inspection, etc.)
• Statistics: Live counts of total requests, service types, and wards
• Data attribution: Links to original St. Louis Open Data source

Data Pipeline Features

• Single command: One script handles download → process → store
• Automatic cleanup: Uses temp directories, no CSV files in codebase
• Incremental updates: Marks old data inactive, adds new data
• Error handling: Robust parsing of various date formats
• Batch processing: Efficient bulk inserts (1000 records at a time)

Notes

• There is a mock data fallback to populate the frontend with data if no csb data has been ingested