AWS Lambda Function for Data Processing

This repository contains an AWS Lambda function that processes restaurant data using AWS Bedrock Agent Runtime.

Overview

The lambda function (lambda_function.py) serves as an API endpoint that:

• Accepts queries about data
• Loads relevant business data files from S3 based on configuration
• Processes requests through AWS Bedrock Agent Runtime
• Returns AI-generated responses with insights

Architecture

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│   API Gateway   │───▶│  Lambda Function │───▶│  Bedrock Agent  │
└─────────────────┘    └──────────────────┘    └─────────────────┘
                              │
                              ▼
                       ┌─────────────────┐
                       │   S3 Storage    │
                       │  (Data Files)   │
                       └─────────────────┘

Configuration Files

The function uses two external configuration files that are loaded at runtime:

1. S3 Configuration (s3_config.yaml)

Defines the S3 bucket and path structure for data files.

2. File Type Mappings (file_type_mappings.json)

Maps business information types to their corresponding file prefixes.

Setup Instructions

1. Create Configuration Files

Create the following configuration files in your deployment package:

s3_config.yaml

s3:
  bucket_name: "your-data-bucket-name"
  base_path: "-data"

file_type_mappings.json

{
  "Customer Hub": "clients",
  "Help Centre": "support",
  "Loyalty": "rewards",
  "Live Report Hub": "analytics",
  "Activities": "events"
}

2. Environment Variables

Set the following environment variables in your Lambda function:

• AGENT_ID: Your Bedrock Agent ID
• AGENT_ALIAS_ID: Your Bedrock Agent Alias ID

3. S3 Data Structure

Ensure your S3 bucket follows this structure:

s3://your-bucket-name/restaurant-data/
├── business-id-1/
│   ├── x.csv
│   ├── z.csv
│   └── c.csv
├── business-id-2/
│   ├── x.csv
│   └── c.csv
└── ...

API Usage

Request Format

{
  "query": "What were my top selling items last month?",
  "business_id": ["business-123", "business-456"],
  "information_type": ["x", "z"],
  "session_id": "optional-session-id",
  "enable_trace": false,
  "end_session": false,
  "memory_id": "optional-memory-id"
}

Response Format

{
  "statusCode": 200,
  "body": {
    "response": "AI-generated response about your data",
    "session_id": "session-id",
    "trace": {...}
  }
}

File Structure

├── lambda_function.py          # Main Lambda handler
├── util.py                     # Utility functions
├── s3_config.yaml             # S3 configuration (gitignored)
├── file_type_mappings.json    # File type mappings (gitignored)
├── .gitignore                 # Git ignore rules
└── README.md                  # This file

Dependencies

• boto3: AWS SDK for Python
• pyyaml: YAML parsing
• AWS Bedrock Agent Runtime permissions
• S3 read permissions for the configured bucket

Security Notes

• Configuration files are gitignored as they may contain sensitive information
• The function validates all inputs and handles errors gracefully
• S3 paths are constructed securely to prevent path traversal attacks

Error Handling

The function includes comprehensive error handling for:

• Invalid JSON requests
• Missing business IDs
• Empty queries
• Configuration file loading errors
• S3 access issues
• Bedrock service errors

Development

Local Testing

1. Create the configuration files with your test data
2. Install dependencies: pip install boto3 pyyaml
3. Test with sample events

Deployment

1. Package the Lambda function with configuration files
2. Upload to AWS Lambda
3. Configure environment variables
4. Set up API Gateway trigger

Example Configuration Files

Example s3_config.yaml

s3:
  bucket_name: "my-data-bucket"
  base_path: "business-data"

Example file_type_mappings.json

{
  "Orders": "sales",
  "Menus": "catalog",
  "Payment": "transactions",
  "Promo": "discounts",
  "Inventory": "stock",
  "Customer Hub": "clients",
  "Help Centre": "support",
  "Loyalty": "rewards",
  "Live Report Hub": "analytics",
  "Activities": "events"
}

Troubleshooting

Common Issues

1. Configuration files not found: Ensure s3_config.yaml and file_type_mappings.json are in the Lambda deployment package
2. S3 access denied: Verify Lambda execution role has S3 read permissions
3. Bedrock access denied: Ensure Lambda has Bedrock Agent Runtime permissions
4. Invalid b IDs: Check that business IDs exist in S3 and match the expected format

Logs

The function provides detailed CloudWatch logs including:

• Configuration loading status
• S3 file loading progress
• Bedrock agent responses
• Error details and stack traces