AI Debate System - CrewAI Implementation

A multi-agent AI debate system built with CrewAI that simulates structured debates on complex topics. This project demonstrates how AI agents can take opposing sides of an argument and have their performance evaluated by an impartial judge agent.

🎯 Overview

This debate system creates two opposing arguments on a given topic using a single debater agent that alternates perspectives, followed by evaluation from a judge agent. The system is designed to explore different viewpoints on controversial topics and provide balanced analysis.

Current Implementation: The system debates "Is AI a threat to humanity?" by generating supporting and opposing arguments, then having a judge evaluate both sides.

🏗️ System Architecture

Agent Roles

1. Debater Agent (gpt-4o-mini)

   • Role: Knowledgeable debater on the specified topic
   • Goal: Generate compelling arguments for both sides of the debate
   • Tasks: Creates both proposer (supporting) and opposer (opposing) arguments

2. Judge Agent (gpt-4o-mini)

   • Role: Impartial judge with expertise on the topic
   • Goal: Evaluate the debate fairly and provide scoring
   • Tasks: Analyzes both sides and provides comprehensive evaluation

📋 Prerequisites

• Python >=3.10 <3.14
• UV package manager
• OpenAI API key

🛠️ Installation

1. Install UV (if not already installed):

pip install uv

2. Navigate to project directory:

cd crew/debate

3. Install dependencies:

crewai install

4. Set up environment variables:
   • Add your OPENAI_API_KEY to the .env file

🎮 Usage

Running the Debate

Execute the debate system from the project root:

crewai run

This will:

1. Generate 10 supporting arguments for the motion (saved to proposer.md)
2. Generate 10 opposing arguments against the motion (saved to opposer.md)
3. Have the judge evaluate both sides and provide a comprehensive decision

Custom Topics

To debate a different topic, modify the inputs in src/debate/main.py:

inputs = {
    'topic': 'Your custom debate topic here',
}

Training the System

Train the crew for improved performance:

crewai train <n_iterations> <filename>

Testing

Test the crew's performance:

crewai test <n_iterations> <eval_llm>

Replay

Replay a specific task execution:

crewai replay <task_id>

📁 Project Structure

debate/
├── src/debate/
│   ├── __init__.py
│   ├── config/
│   │   ├── agents.yaml          # Agent configurations
│   │   └── tasks.yaml           # Task definitions
│   ├── tools/
│   │   ├── __init__.py
│   │   └── custom_tool.py       # Custom tool template
│   ├── crew.py                  # Main crew orchestration
│   └── main.py                  # Entry point and execution logic
├── knowledge/
│   └── user_preference.txt      # User context information
├── proposer.md                  # Generated supporting arguments
├── opposer.md                   # Generated opposing arguments
└── README.md                    # This file

⚙️ Configuration

Agents Configuration (config/agents.yaml)

debater:
  role: "You are a debater with a nice amount of knowledge about the topic {topic}"
  goal: "Debate the topic {topic}"
  backstory: "You're a seasoned debater with a knack for debating the topic {topic}. Known for your ability to debate the topic in a clear and concise manner."
  model: gpt-4o-mini

judge:
  role: "You are a judge with a nice amount of knowledge about the topic {topic}"
  goal: "Judge the debate between the debater and the opponent"
  backstory: "You're a meticulous judge with a keen eye for detail. You are just and unbiased."
  model: gpt-4o-mini

Tasks Configuration (config/tasks.yaml)

The system includes three main tasks:

1. Proposer Task: Generates 10 supporting arguments
2. Opposer Task: Generates 10 opposing arguments
3. Decider Task: Judges both sides and provides comprehensive evaluation

📊 Output Format

Supporting Arguments (proposer.md)

The system generates 10 bullet points supporting the motion, covering aspects like:

• Augmentation of human abilities
• Economic growth and innovation
• Healthcare improvements
• Global challenge solutions
• Safety enhancements

Opposing Arguments (opposer.md)

The system generates 10 bullet points opposing the motion, covering concerns like:

• Exponential AI growth risks
• Autonomous weaponry concerns
• Job displacement issues
• Privacy and surveillance threats
• Existential risks

Judge Evaluation

The judge provides:

• Scores for both sides
• Reasoning for the scores
• Overall conclusion
• Recommendations for the debate

🔧 Customization

Adding Custom Tools

Extend functionality by creating custom tools in the tools/ directory:

from crewai.tools import BaseTool
from typing import Type
from pydantic import BaseModel, Field

class MyCustomToolInput(BaseModel):
    argument: str = Field(..., description="Description of the argument.")

class MyCustomTool(BaseTool):
    name: str = "Name of my tool"
    description: str = "Tool description"
    args_schema: Type[BaseModel] = MyCustomToolInput

    def _run(self, argument: str) -> str:
        # Implementation goes here
        return "Tool output"

Modifying Agent Behavior

Edit the YAML configuration files to:

• Change agent personalities and expertise
• Adjust LLM models and parameters
• Modify task descriptions and outputs
• Add new debate formats

Knowledge Integration

The system includes a knowledge base (knowledge/user_preference.txt) that can be expanded with:

• Domain-specific information
• User preferences and context
• Historical debate data
• Expert opinions and research

🎯 Example Debate Topics

The system works well with controversial topics such as:

• "Is AI a threat to humanity?" (default)
• "Should social media be regulated?"
• "Is remote work better than office work?"
• "Should genetic engineering be allowed?"
• "Is cryptocurrency the future of money?"

🔍 Advanced Features

Sequential Processing

The crew uses sequential processing to ensure:

1. Supporting arguments are generated first
2. Opposing arguments are created independently
3. Judge evaluation considers both perspectives

Verbose Output

Enable detailed logging to see:

• Agent reasoning processes
• Task execution steps
• Decision-making chains
• Performance metrics

Knowledge Sources

Integrate external knowledge sources to enhance argument quality and factual accuracy.

🧪 Testing Different Scenarios

Modify the training and testing functions in main.py to experiment with:

• Different topics and complexity levels
• Various evaluation criteria
• Multiple debate formats
• Different LLM models

📈 Performance Optimization

• Model Selection: Currently uses gpt-4o-mini for cost-effectiveness
• Sequential Processing: Ensures logical flow of arguments
• Output Files: Saves results for analysis and comparison
• Error Handling: Robust exception management for reliable execution

🤝 Contributing

To extend this debate system:

1. Add New Agent Types: Create specialized roles (researcher, fact-checker, audience)
2. Implement New Debate Formats: Add parliamentary, town hall, or academic styles
3. Enhance Evaluation: Create more sophisticated scoring systems
4. Add Real-time Features: Implement live debate capabilities
5. Integrate External Data: Connect to research databases and current events

📚 CrewAI Resources

• CrewAI Documentation
• GitHub Repository
• Discord Community
• Interactive Docs Chat

📄 License

This project is part of the larger "Using CrewAI and Multiple LLMs" repository and follows the same licensing terms.

🎓 Educational Value

This project demonstrates:

• Multi-agent system design patterns
• Structured debate simulation
• AI-powered argument generation
• Automated evaluation systems
• CrewAI framework implementation

Perfect for learning about AI agents, debate analysis, and collaborative AI systems.

---

Current Status: The system successfully generates balanced arguments on AI's impact on humanity and provides impartial evaluation. Ready for extension with additional topics and enhanced evaluation criteria.