Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 14 additions & 0 deletions task-1/output/cleaned_sales.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
transaction_id,product_name,category,price,quantity,customer_email,date,revenue,vat
1,Laptop Pro,Electronics,999.99,2,alice@example.com,2024-03-15,1999.98,420.0
2,Wireless Mouse,Electronics,29.99,5,bob@company.com,2024-03-15,149.95,31.49
3,Usb Cable,Electronics,4.99,10,,2024-03-16,49.9,10.48
4,Office Chair,Furniture,349.5,1,charlie@work.org,2024-03-16,349.5,73.39
5,Standing Desk,Furniture,599.0,1,charlie@work.org,not_a_date,599.0,125.79
8,Monitor Arm,Furniture,79.99,0,frank@corp.com,2024-03-18,0.0,0.0
9,Webcam Hd,Electronics,54.99,1,,2024-03-18,54.99,11.55
10,Desk Lamp,Furniture,34.99,4,grace@university.edu,2024-03-19,139.96,29.39
11,Noise Cancelling Headphones,Electronics,199.99,1,alice@example.com,2024-03-19,199.99,42.0
12,Cable Management Kit,Furniture,15.99,6,henry@business.com,2024-03-20,95.94,20.15
13,Ergonomic Mouse Pad,Furniture,24.99,3,ivan@email.com,2024-03-20,74.97,15.74
14,Laptop Stand,Furniture,45.99,2,jenny@work.org,2024-03-21,91.98,19.32
15,Bluetooth Speaker,Unknown,39.99,1,karl@startup.io,2024-03-21,39.99,8.4
Binary file added task-1/src/__pycache__/__init__.cpython-311.pyc
Binary file not shown.
Binary file added task-1/src/__pycache__/config.cpython-311.pyc
Binary file not shown.
Binary file added task-1/src/__pycache__/models.cpython-311.pyc
Binary file not shown.
Binary file added task-1/src/__pycache__/pipeline.cpython-311.pyc
Binary file not shown.
Binary file added task-1/src/__pycache__/transforms.cpython-311.pyc
Binary file not shown.
15 changes: 8 additions & 7 deletions task-1/src/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,8 @@
3. Raise ValueError if either is missing — do NOT let None silently propagate.
"""
import os

from dotenv import load_dotenv

#print("dotenv works!")

# Load .env values into os.environ before they're read by _required().
# (Step 1 from the docstring above; already wired up so the rest of the
Expand All @@ -21,15 +20,17 @@

def _required(name: str) -> str:
"""Read an env var; fail loudly if missing."""
# TODO 2: Read os.environ[name]; if not set, raise ValueError with a
# message that names the missing variable AND points at .env.example.
raise NotImplementedError("Implement _required: see TODO 2 in config.py")
path = os.environ.get(name)
if path is None:
raise ValueError(f"Missing required environment path: {name},See .env.example for details.")
return path



# TODO 3: Replace the placeholder lines below by calling _required(...) for
# each variable. INPUT_PATH and OUTPUT_PATH must be importable from this
# module by the rest of the pipeline as a relative import
# (`from .config import INPUT_PATH, ...`), since the pipeline runs as
# `python -m src.pipeline`.
INPUT_PATH: str = "" # TODO: _required("INPUT_PATH")
OUTPUT_PATH: str = "" # TODO: _required("OUTPUT_PATH")
INPUT_PATH: str = _required("INPUT_PATH")
OUTPUT_PATH: str = _required("OUTPUT_PATH")
36 changes: 17 additions & 19 deletions task-1/src/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,24 +10,22 @@
"""
from dataclasses import dataclass


# TODO 1: Define a @dataclass called Transaction with these fields:
# transaction_id: int
# product_name: str
# category: str
# price: float
# quantity: int
# customer_email: str
# date: str
# revenue: float = 0.0
# vat: float = 0.0
#
# TODO 2: Add __post_init__ that raises ValueError when:
# - self.price < 0 (with a message naming the bad value)
# - not self.product_name.strip() (empty / whitespace-only product name)


# Replace this stub with your dataclass:
@dataclass
class Transaction:
transaction_id: int # TODO: replace this stub with the full field list above
transaction_id: int
product_name: str
category: str
price: float
quantity: int
customer_email: str
date: str
revenue: float = 0.0
vat: float = 0.0

def __post_init__(self):
self.price = float(self.price)
if self.price < 0:
raise ValueError(f"Price should not be negtive: {self.price}")
if not self.product_name.strip():
raise ValueError("Product name should not be empty or whitespace-only")

15 changes: 11 additions & 4 deletions task-1/src/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -31,14 +31,20 @@

def read_csv(path: str) -> list[dict]:
"""Read a CSV file into a list of dicts. I/O only — no business rules."""
# TODO: implement using csv.DictReader.
raise NotImplementedError
with open(path, mode='r', encoding='utf-8') as file:
reader = csv.DictReader(file)
return list(reader)


def write_csv(rows: list[dict], path: str) -> None:
"""Write a list of dicts to CSV. I/O only — no business rules."""
# TODO: implement using csv.DictWriter.
raise NotImplementedError
if not rows:
return
keys_ofdictionry = rows[0].keys()
with open(path, mode='w', encoding='utf-8', newline='') as file:
writer = csv.DictWriter(file, fieldnames=keys_ofdictionry)
writer.writeheader()
writer.writerows(rows)


def run() -> None:
Expand All @@ -53,6 +59,7 @@ def run() -> None:
# TODO: cast price / quantity / revenue / vat to the right types here
# if your transforms left them as strings, then iterate over `data`
# to build Transaction(**row) for each cleaned row.

transactions = [Transaction(**row) for row in data]

# Output dir must exist. Use pathlib for cross-platform safety.
Expand Down
47 changes: 39 additions & 8 deletions task-1/src/transforms.py

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

filter_zero_quantity is not working because quantity comes out of the CSV as a string "0", not an integer 0, so "0" != 0 is always true and zero-quantity rows slip through. Wrap it in int() to fix this.

Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,16 @@ def remove_invalid(rows: list[dict]) -> list[dict]:

Empty here means missing, "", or whitespace-only.
"""
# TODO: implement. Return a new list, do not mutate `rows`.
raise NotImplementedError
result = []
for row in rows:
product_name = row.get("product_name", "")
try:
price = float(row.get("price", 0))
except ValueError:
continue
if product_name.strip() and price>0:
result.append(row.copy())
return result


def clean_fields(rows: list[dict]) -> list[dict]:
Expand All @@ -31,20 +39,43 @@ def clean_fields(rows: list[dict]) -> list[dict]:

Return a new list. Do not mutate the input rows.
"""
# TODO: implement.
raise NotImplementedError
result = []
for row in rows:
product_name = row.get("product_name", "").strip().title()
customer_email = row.get("customer_email", "").strip().lower()
category = row.get("category", "").strip()
category = category if category else "Unknown"
new_row = row.copy()
new_row["product_name"] = product_name
new_row["customer_email"] = customer_email
new_row["category"] = category
result.append(new_row)
return result


def calculate_revenue(rows: list[dict], vat_rate: float = 0.21) -> list[dict]:
"""Add 'revenue' (price * quantity) and 'vat' (revenue * vat_rate) fields.

Round both to 2 decimal places. Coerce price/quantity from string if needed.
"""
# TODO: implement.
raise NotImplementedError
result = []
for row in rows:
price = float(row["price"])
quantity = int(row["quantity"])
revenue = round(price * quantity, 2)
vat = round(revenue * vat_rate, 2)
new_row = row.copy()
new_row["revenue"] = revenue
new_row["vat"] = vat
result.append(new_row)
return result


def filter_zero_quantity(rows: list[dict]) -> list[dict]:
"""Remove rows where quantity is 0."""
# TODO: implement.
raise NotImplementedError
result = []
for row in rows:
quantity = row.get("quantity", 0)
if quantity != 0:
result.append(row.copy())
return result
Binary file added task-1/tests/__pycache__/__init__.cpython-311.pyc
Binary file not shown.
Binary file not shown.
52 changes: 32 additions & 20 deletions task-1/tests/test_transforms.py
Original file line number Diff line number Diff line change
@@ -1,12 +1,5 @@
"""Tests for the pure transform functions (chapter Task 5).

Write at least 4 tests:
- test_remove_invalid_drops_empty_names
- test_clean_fields_normalizes_names
- test_calculate_revenue_adds_fields
- test_no_mutation
"""
import pytest
import copy

from src.transforms import (
calculate_revenue,
Expand All @@ -17,24 +10,43 @@


def test_remove_invalid_drops_empty_names():
# TODO: feed in 3 rows (one valid, one empty product_name, one
# whitespace-only product_name); assert only the valid one survives.
raise NotImplementedError
data = [
{"product_name": "Laptop", "price": 999.99},
{"product_name": "", "price": 50.0},
{"product_name": " ", "price": 25.0},
]
result = remove_invalid(data)
assert len(result) == 1
assert result[0]["product_name"] == "Laptop"


def test_clean_fields_normalizes_names():
# TODO: feed a row with messy product_name and uppercase email; assert
# the output has stripped + title-cased name and lowercase email.
raise NotImplementedError
data = [
{"product_name": " laptop", "customer_email": "BOB@Company.COM"}
]
result = clean_fields(data)

assert len(result) == 1
assert result[0]["product_name"] == "Laptop"
assert result[0]["customer_email"] == "bob@company.com"


def test_calculate_revenue_adds_fields():
# TODO: feed a row with price=100, quantity=3; assert output has
# revenue=300.0 and vat=63.0 (default VAT rate is 0.21).
raise NotImplementedError
data = [
{"product_name": "Laptop", "price": 100.0, "quantity": 3}
]
result = calculate_revenue(data)

assert len(result) == 1
assert result[0]["revenue"] == 300.0
assert result[0]["vat"] == 63.0


def test_no_mutation():
# TODO: feed in a list, run any transform on it, assert the original
# list is unchanged. This is the most important test in the file.
raise NotImplementedError
original_data = [
{"product_name": "Laptop", "price": 100.0, "quantity": 1}
]

data_to_transform = copy.deepcopy(original_data)
calculate_revenue(data_to_transform)
assert data_to_transform == original_data
97 changes: 94 additions & 3 deletions task-2/AI_DEBUG.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,27 +11,118 @@ Aim for 100-200 words per section. Bullet points are fine.
What went wrong? Paste the traceback or the wrong-output sample. Include the file and the line you were running when it broke.

```
(paste here)
python C:\Users\Beheerder\c55-data-week-2\task-1\src\pipeline.py
C:\Users\Beheerder\AppData\Local\Microsoft\WindowsApps\python.exe: can't open file 'C:\\Users\\Beheerder\\c55-data-week-2\\UsersBeheerderc55-data-week-2task-1srcpipeline.py': [Errno 2] No such file or directory

Beheerder@Marah MINGW64 ~/c55-data-week-2 (main)
$ python -m task-1.src.pipeline
dotenv works!
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Beheerder\c55-data-week-2\task-1\src\pipeline.py", line 77, in <module>
    run()
  File "C:\Users\Beheerder\c55-data-week-2\task-1\src\pipeline.py", line 51, in run
    raw = read_csv(INPUT_PATH)
          ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Beheerder\c55-data-week-2\task-1\src\pipeline.py", line 34, in read_csv
    with open(path, mode='r', encoding='utf-8') as file:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'data/messy_sales.csv'
```

## The Prompt

What did you ask the AI? Paste the actual prompt verbatim. (Include the code or stack trace you pasted alongside it; do NOT include any real `.env` values, API keys, or PII — replace those with `<REDACTED>`.)

```
(paste here)
python C:\Users\Beheerder\c55-data-week-2\task-1\src\pipeline.py
C:\Users\Beheerder\AppData\Local\Microsoft\WindowsApps\python.exe: can't open file 'C:\\Users\\Beheerder\\c55-data-week-2\\UsersBeheerderc55-data-week-2task-1srcpipeline.py': [Errno 2] No such file or directory

Beheerder@Marah MINGW64 ~/c55-data-week-2 (main)
$ python -m task-1.src.pipeline
dotenv works!
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "C:\Users\Beheerder\c55-data-week-2\task-1\src\pipeline.py", line 77, in <module>
    run()
  File "C:\Users\Beheerder\c55-data-week-2\task-1\src\pipeline.py", line 51, in run
    raw = read_csv(INPUT_PATH)
          ^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Beheerder\c55-data-week-2\task-1\src\pipeline.py", line 34, in read_csv
    with open(path, mode='r', encoding='utf-8') as file:
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'data/messy_sales.csv'
Is this error because of the data file in a different folder?
```

## The Solution

What did the AI suggest? Did it work on the first try? Did you have to follow up? Paste the final code change (a small diff is best).

```
(paste here)
#LLM Response like that:
Yes, exactly! You hit the nail on the head.

The error happens because of **where you are standing** (your current directory) versus **where the computer is looking**.

### The Problem

You are standing in `~/c55-data-week-2`.
Your code is trying to find `data/messy_sales.csv`.
But looking at your file structure, the data is actually inside `task-1/data/messy_sales.csv`.

Because you are in the "Root" folder, Python looks for a folder named `data` right next to you, doesn't see it, and panics.

### The Fix

You have two ways to fix this. **Option 1 is usually what instructors want.**

#### Option 1: Fix the `INPUT_PATH` (Recommended)

Open your `config.py` (or wherever `INPUT_PATH` is defined) and change it to include the `task-1` folder:

```python
# Change this:
INPUT_PATH = "data/messy_sales.csv"

# To this:
INPUT_PATH = "task-1/data/messy_sales.csv"
OUTPUT_PATH = "task-1/output/cleaned_sales.csv"

```

#### Option 2: Run from inside the `task-1` folder

If you don't want to change the code, you have to move your terminal "closer" to the data:

1. **Move into the task folder:**
```bash
cd task-1

```


2. **Run the module from there:**
```bash
python -m src.pipeline

```


*(Note: Since you moved inside `task-1`, you drop the `task-1.` prefix from the command.)*

### Why "dotenv works!" is a good sign

The fact that you see `dotenv works!` means your Python environment is working and it successfully found your `pipeline.py`. You are 99% of the way there—you just need to point the "boss" (the script) to the right "filing cabinet" (the data folder).

**Try Option 1 first**, as it makes your project more stable when running from the main directory!
```

## Reflection

Did you understand *why* the original code was broken before the AI told you? If not, what was the gap in your mental model? If you understood it before asking, why did you still ask the AI — speed, second opinion, or something else?

(write here, ~100 words)
Actually, yes, I understand what the problem was, but I wanted to check if I understand the trace in the correct way, and also to add something in task-2 as well ^_^.
Binary file added task-3/assets/azure_blob_week2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions task-3/assets/blob_url.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://sthyfstudentsdemo.blob.core.windows.net/week2-mareh-aboghanem/cleaned_sales.csv