Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions task-1/output/clean_sales.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
transaction_id,product_name,category,price,quantity,customer_email,date,revenue,vat
1,Laptop Pro,Electronics,999.99,2,alice@example.com,2024-03-15,1999.98,420.0
2,Wireless Mouse,Electronics,29.99,5,bob@company.com,2024-03-15,149.95,31.49
3,Usb Cable,Electronics,4.99,10,,2024-03-16,49.9,10.48
4,Office Chair,Furniture,349.5,1,charlie@work.org,2024-03-16,349.5,73.39
5,Standing Desk,Furniture,599.0,1,charlie@work.org,not_a_date,599.0,125.79
9,Webcam Hd,Electronics,54.99,1,,2024-03-18,54.99,11.55
10,Desk Lamp,Furniture,34.99,4,grace@university.edu,2024-03-19,139.96,29.39
11,Noise Cancelling Headphones,Electronics,199.99,1,alice@example.com,2024-03-19,199.99,42.0
12,Cable Management Kit,Furniture,15.99,6,henry@business.com,2024-03-20,95.94,20.15
13,Ergonomic Mouse Pad,Furniture,24.99,3,ivan@email.com,2024-03-20,74.97,15.74
14,Laptop Stand,Furniture,45.99,2,jenny@work.org,2024-03-21,91.98,19.32
15,Bluetooth Speaker,Unknown,39.99,1,karl@startup.io,2024-03-21,39.99,8.4
11 changes: 7 additions & 4 deletions task-1/src/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
from dotenv import load_dotenv



# Load .env values into os.environ before they're read by _required().
# (Step 1 from the docstring above; already wired up so the rest of the
# module can rely on os.environ being populated.)
Expand All @@ -23,13 +24,15 @@ def _required(name: str) -> str:
"""Read an env var; fail loudly if missing."""
# TODO 2: Read os.environ[name]; if not set, raise ValueError with a
# message that names the missing variable AND points at .env.example.
raise NotImplementedError("Implement _required: see TODO 2 in config.py")

value = os.environ.get(name)
if value is None :

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small edge case: os.environ.get(name) returns "" (empty string) for a set-but-empty variable, not None. Your check if value is None would let an empty path through.

Safer pattern would be:

value = os.environ.get(name)
if not value:
    raise ValueError(f"Missing environment variable {name}. See .env.example")

raise ValueError(f"Missing environment variable {name}. see .env file")
return value

# TODO 3: Replace the placeholder lines below by calling _required(...) for
# each variable. INPUT_PATH and OUTPUT_PATH must be importable from this
# module by the rest of the pipeline as a relative import
# (`from .config import INPUT_PATH, ...`), since the pipeline runs as
# `python -m src.pipeline`.
INPUT_PATH: str = "" # TODO: _required("INPUT_PATH")
OUTPUT_PATH: str = "" # TODO: _required("OUTPUT_PATH")
INPUT_PATH: str = _required("INPUT_PATH") # TODO: _required("INPUT_PATH")
OUTPUT_PATH: str = _required("OUTPUT_PATH") # TODO: _required("OUTPUT_PATH")
17 changes: 16 additions & 1 deletion task-1/src/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,29 @@
# customer_email: str
# date: str
# revenue: float = 0.0
# vat: float = 0.0
# vat: float = 0.0
#
# TODO 2: Add __post_init__ that raises ValueError when:
# - self.price < 0 (with a message naming the bad value)
# - not self.product_name.strip() (empty / whitespace-only product name)



# Replace this stub with your dataclass:
@dataclass
class Transaction:
transaction_id: int # TODO: replace this stub with the full field list above
product_name: str
category: str
price: float
quantity: int
customer_email: str
date: str
revenue: float = 0.0
vat: float = 0.0

def __post_init__(self):
if self.price < 0 :
return ValueError(f"Invalid price {self.price} the price can not be negative!")
if not self.product_name.strip():
return ValueError("product name can not empty or has a whitespace!")
Comment on lines +45 to +48

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return ValueError(...) does not stop execution or reject bad data, it just returns a ValueError object that gets thrown away. Nothing ever raises. Replace return with raise.

23 changes: 19 additions & 4 deletions task-1/src/pipeline.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,13 +32,19 @@
def read_csv(path: str) -> list[dict]:
"""Read a CSV file into a list of dicts. I/O only — no business rules."""
# TODO: implement using csv.DictReader.
raise NotImplementedError

with open(path, newline="", encoding="utf-8") as f:
reader = csv.DictReader(f)
return [dict(row) for row in reader]

def write_csv(rows: list[dict], path: str) -> None:
"""Write a list of dicts to CSV. I/O only — no business rules."""
# TODO: implement using csv.DictWriter.
raise NotImplementedError
if not rows :
return
with open(path, "w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=rows[0].keys())
writer.writeheader()
writer.writerows(rows)


def run() -> None:
Expand All @@ -53,7 +59,16 @@ def run() -> None:
# TODO: cast price / quantity / revenue / vat to the right types here
# if your transforms left them as strings, then iterate over `data`
# to build Transaction(**row) for each cleaned row.
transactions = [Transaction(**row) for row in data]
transactions = []
for row in data:
typed_row ={**row,
"transaction_id" : int(row.get("transaction_id",0)),
"price": float(row.get("price",0)),
"quantity":int(row.get("quantity",0)),
"revenue": float(row.get("revenue",0)),
"vat": float(row.get("vat",0))
}
transactions.append(Transaction(**typed_row))

# Output dir must exist. Use pathlib for cross-platform safety.
Path(OUTPUT_PATH).parent.mkdir(parents=True, exist_ok=True)
Expand Down
47 changes: 39 additions & 8 deletions task-1/src/transforms.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,12 @@ def remove_invalid(rows: list[dict]) -> list[dict]:
Empty here means missing, "", or whitespace-only.
"""
# TODO: implement. Return a new list, do not mutate `rows`.
raise NotImplementedError

return [
row for row in rows
if str(row.get("product_name","")).strip() and float(row.get("price",0))>= 0
]


def clean_fields(rows: list[dict]) -> list[dict]:
"""Clean string fields:

Expand All @@ -32,7 +35,24 @@ def clean_fields(rows: list[dict]) -> list[dict]:
Return a new list. Do not mutate the input rows.
"""
# TODO: implement.
raise NotImplementedError
cleaned =[]
for row in rows :
new_row = {**row,
"product_name": row.get("product_name", "").strip().title(),
"customer_email": row.get("customer_email", "").strip().lower(),
"category": row.get("category") or "Unknown",
}
cleaned.append(new_row)
return cleaned

def filter_zero_quantity(rows: list[dict]) -> list[dict]:
"""Remove rows where quantity is 0."""
# TODO: implement.
if rows is None:
return[]
Comment on lines +51 to +52

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The if rows is None: return [] guard is fine after your debugging experience. Your upstream function returned None instead of a list. Fixing the source is often better than guarding everywhere, but both approaches are valid to discuss.

return [
row for row in rows
if int(row.get("quantity", 0)) != 0 ]


def calculate_revenue(rows: list[dict], vat_rate: float = 0.21) -> list[dict]:
Expand All @@ -41,10 +61,21 @@ def calculate_revenue(rows: list[dict], vat_rate: float = 0.21) -> list[dict]:
Round both to 2 decimal places. Coerce price/quantity from string if needed.
"""
# TODO: implement.
raise NotImplementedError
result =[]
for row in rows :

price = float(row.get("price",0))
quantity =int(row.get("quantity",0))

revenue = round(price * quantity,2)
vat = round(revenue * vat_rate,2)

new_row = { **row,
"revenue" : revenue,
"vat" : vat }

result.append(new_row)
return result



def filter_zero_quantity(rows: list[dict]) -> list[dict]:
"""Remove rows where quantity is 0."""
# TODO: implement.
raise NotImplementedError
38 changes: 34 additions & 4 deletions task-1/tests/test_transforms.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
- test_no_mutation
"""
import pytest
import copy

from src.transforms import (
calculate_revenue,
Expand All @@ -19,22 +20,51 @@
def test_remove_invalid_drops_empty_names():
# TODO: feed in 3 rows (one valid, one empty product_name, one
# whitespace-only product_name); assert only the valid one survives.
raise NotImplementedError
data = [
{"product_name": "Laptop", "price": 999.99},
{"product_name": "", "price": 50.0},
{"product_name": " ", "price": 25.0},
]

result = remove_invalid(data)

assert len(result) == 1
assert result[0]["product_name"] == "Laptop"


def test_clean_fields_normalizes_names():
# TODO: feed a row with messy product_name and uppercase email; assert
# the output has stripped + title-cased name and lowercase email.
raise NotImplementedError
data = [
{"product_name": " laptop ", "customer_email": "TEST@EMAIL.COM", "category": ""},
]

result = clean_fields(data)

assert result[0]["product_name"] == "Laptop"
assert result[0]["customer_email"] == "test@email.com"
assert result[0]["category"] == "Unknown"


def test_calculate_revenue_adds_fields():
# TODO: feed a row with price=100, quantity=3; assert output has
# revenue=300.0 and vat=63.0 (default VAT rate is 0.21).
raise NotImplementedError
data = [{"price": 100.0, "quantity": 3},]

result = calculate_revenue(data, vat_rate=0.21)

assert result[0]["revenue"] == 300.0
assert result[0]["vat"] == 63.0 #300 * 0.21



def test_no_mutation():
# TODO: feed in a list, run any transform on it, assert the original
# list is unchanged. This is the most important test in the file.
raise NotImplementedError
data = [ {"product_name": "Laptop", "price": 100.0, "quantity": 1}, ]

original = copy.deepcopy(data)

_ = clean_fields(data)

assert data == original
27 changes: 26 additions & 1 deletion task-2/AI_DEBUG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,21 @@ What went wrong? Paste the traceback or the wrong-output sample. Include the fil
```
(paste here)
```
data = filter_zero_quantity(data)
File "/Users/barbari/Desktop/c55-data-week-2/task-1/src/transforms.py", line 50, in filter_zero_quantity
return []
TypeError: 'NoneType' object is not iterable


def filter_zero_quantity(rows: list[dict]) -> list[dict]:
"""Remove rows where quantity is 0."""
# TODO: implement.
return [
row for row in rows
if row.get("quantity", 0) != 0 ]




## The Prompt

Expand All @@ -21,6 +36,11 @@ What did you ask the AI? Paste the actual prompt verbatim. (Include the code or
```
(paste here)
```
i pasted this data = filter_zero_quantity(data)
in filter_zero_quantity
return [
TypeError: 'NoneType' object is not iterable]
and asked what is the problem

## The Solution

Expand All @@ -29,9 +49,14 @@ What did the AI suggest? Did it work on the first try? Did you have to follow up
```
(paste here)
```

the AI suggested that the issue caused bt rows being none which made the list comprehension fail because none is not iterable .
it recomended either fixing the source of rows so that it always returns a list or adding a safety check inside the function to handle none.
It did not work on the first try because the problem was not in the list comprehension itself but in how rows was being passed into the function, so a follow-up explanation was needed to clarify that the real fix had to be applied upstream or by guarding against None. The final code change suggested was adding a simple check at the start of the function
## Reflection

Did you understand *why* the original code was broken before the AI told you? If not, what was the gap in your mental model? If you understood it before asking, why did you still ask the AI — speed, second opinion, or something else?

(write here, ~100 words)

At first, I didn’t fully understand the real cause of the error. I assumed the issue might be inside the list comprehension itself or the condition on quantity, but I didn’t immediately realize that rows itself could be None. The gap in my understanding was not considering that the input variable might be coming from an earlier function that returned None instead of a list.
I still asked the AI because I wanted confirmation and a faster way to pinpoint the exact cause instead of guessing through trial and error. It helped me quickly narrow it down to the real source of the bug.
Binary file added task-3/assets/azure_blob_week2.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions task-3/assets/blob_url.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
https://sthyfstudentsdemo.blob.core.windows.net/week2-thebaraah/clean_sales.csv