diff --git a/AI_ASSIST.md b/AI_ASSIST.md index 8585a41..f216a0f 100644 --- a/AI_ASSIST.md +++ b/AI_ASSIST.md @@ -4,31 +4,33 @@ Document one place you used an LLM during this assignment. ## The problem - +When I tried to start the Streamlit dashboard, it could not connect to Azure PostgreSQL. The terminal reported an invalid SSL mode value. -TODO +My PostgreSQL connection URL was stored in an ignored .env file, so I needed help investigating the problem without sharing the complete URL or any credentials. ## The prompt - - -TODO +My Streamlit application cannot connect to Azure PostgreSQL. The terminal reports an invalid SSL mode value. My connection URL is stored in an ignored .env file. ## The response - +ChatGPT explained that the error was probably caused by an incorrect SSL setting in the PostgreSQL URL. It advised me to check that the URL in my local .env file ended with: -TODO +?sslmode=require ## Reflection - +I checked the connection URL in my .env file and discovered that I had written: + +?sslmode=requir + +The final letter e was missing. I corrected it to: + +?sslmode=require + +After saving the .env file and restarting Streamlit, the application connected successfully to Azure PostgreSQL and loaded the dashboard. -TODO +From this error, I learned to pay closer attention when copying connection URLs and configuration values. Even a small typo can prevent an application from connecting, so I should carefully compare copied values with the original instructions before running the application. --- diff --git a/README.md b/README.md index 91de124..138e23e 100644 --- a/README.md +++ b/README.md @@ -52,9 +52,9 @@ Fill in `week11-streamlit/metric_definitions.md`: a five-field definition (name, 2) Paste its link below, plus screenshots or a PDF export in this repo. 3) Paste your 5-minute presentation recording link (keep it PRIVATE). --> -- Metabase dashboard (in the **Week 11 Submissions** collection): TODO -- Screenshots / PDF export: TODO -- Presentation recording (private, hosted in the Azure `student-submissions` container): TODO +- Metabase dashboard (in the **Week 11 Submissions** collection): https://metabase-hyf.politepebble-abd3ebc2.westeurope.azurecontainerapps.io/dashboard/48-nyc-taxi-analytics-mohammed-alfakih +- Screenshots / PDF export: [Metabase](screenshots/metabase-dashboard.png), [Streamlit](screenshots/streamlit-dashboard.png) +- Presentation recording (private, hosted in the Azure `student-submissions` container): https://hyfstoragedev.blob.core.windows.net/student-submissions/week-11/mohammed-alfakih.mp4 > ⚠️ **Keep the recording private.** It shows your screen and voice. Never make it public and never commit the `.mp4` (git history is forever). Check the frame for passwords, `.env` contents, or connection strings before uploading. @@ -66,7 +66,7 @@ To upload by hand, [open the `hyfstoragedev` Storage browser in the Azure Portal Your pull request should review itself: a reviewer should be able to understand and check it without asking you anything. When you open the PR, GitHub loads a template (`.github/PULL_REQUEST_TEMPLATE.md`) into the description, fill in every section. Two things carry the most weight: -- **Reproducible run instructions.** The Streamlit steps above must work from a clean clone against the reviewer's *own* Postgres: `uv sync`, copy `.env.example` to `.env`, set their own `POSTGRES_URL` (with `?sslmode=require`) and `DB_SCHEMA`, then `uv run streamlit run app.py`. Name every prerequisite, including your own `fct_trips` mart from Week 10. If a step only works on your machine, it is not reproducible. +- **Reproducible run instructions.** The Streamlit steps above must work from a clean clone against the reviewer's _own_ Postgres: `uv sync`, copy `.env.example` to `.env`, set their own `POSTGRES_URL` (with `?sslmode=require`) and `DB_SCHEMA`, then `uv run streamlit run app.py`. Name every prerequisite, including your own `fct_trips` mart from Week 10. If a step only works on your machine, it is not reproducible. - **Proof for what a reviewer cannot run.** A reviewer cannot open your private `dev_` schema or your Metabase Questions, so commit screenshots (or a PDF export) of your Metabase dashboard and your running Streamlit app. Screenshots are how you prove "it runs on my data." See "Package your pull request for review" in the Week 11 Assignment chapter for the full rationale. diff --git a/screenshots/metabase-dashboard.png b/screenshots/metabase-dashboard.png new file mode 100644 index 0000000..8d1e539 Binary files /dev/null and b/screenshots/metabase-dashboard.png differ diff --git a/screenshots/streamlit-dashboard.png b/screenshots/streamlit-dashboard.png new file mode 100644 index 0000000..8eb5f3f Binary files /dev/null and b/screenshots/streamlit-dashboard.png differ diff --git a/week11-streamlit/app.py b/week11-streamlit/app.py index cf06bcb..18b62a8 100644 --- a/week11-streamlit/app.py +++ b/week11-streamlit/app.py @@ -30,16 +30,116 @@ def run_query(sql: str) -> pd.DataFrame: with engine.connect() as conn: return pd.read_sql(sql, conn) +# Payment-type filter +payment_types_df = run_query( + f""" + SELECT DISTINCT payment_type_label + FROM {DB_SCHEMA}.fct_trips + WHERE payment_type_label IS NOT NULL + ORDER BY payment_type_label + """ +) + +payment_types = ["All"] + payment_types_df["payment_type_label"].tolist() + +selected_payment_type = st.sidebar.selectbox( + "Payment type", + payment_types, +) +if selected_payment_type == "All": + where_clause = "" +else: + safe_payment_type = selected_payment_type.replace("'", "''") + where_clause = ( + "WHERE payment_type_label = " + f"'{safe_payment_type}'" + ) + + +# Panel 1: Headline KPIs st.subheader("Headline KPIs") -# TODO: query total trip count, average trip_distance, and average -# fare_per_mile from {DB_SCHEMA}.fct_trips through run_query(), then -# render three tiles side by side with st.columns(3) and .metric(). -# This is deliberately not the total-trips/avg-fare/total-revenue trio -# from the chapter: trip_distance and fare_per_mile are different columns, -# so copying the chapter's SQL verbatim will not answer this. -raise NotImplementedError( - "TODO: implement the headline KPIs panel (total trips, avg trip " - "distance, avg fare per mile) from fct_trips." +kpi_df = run_query( + f""" + SELECT + COUNT(*) AS total_trips, + AVG(trip_distance) AS avg_trip_distance, + AVG(fare_per_mile) AS avg_fare_per_mile + FROM {DB_SCHEMA}.fct_trips + {where_clause} + """ ) + +kpis = kpi_df.iloc[0] + +col1, col2, col3 = st.columns(3) + +col1.metric( + "Total trips", + f"{int(kpis['total_trips']):,}", +) + +col2.metric( + "Average trip distance", + f"{kpis['avg_trip_distance']:.2f} miles", +) + +col3.metric( + "Average fare per mile", + f"${kpis['avg_fare_per_mile']:.2f}", +) + + +# Panel 2: Trip count by hour +st.subheader("Trip count by hour of day") + +hourly_trips = run_query( + f""" + SELECT + EXTRACT(HOUR FROM pickup_datetime) AS pickup_hour, + COUNT(*) AS trip_count + FROM {DB_SCHEMA}.fct_trips + {where_clause} + GROUP BY EXTRACT(HOUR FROM pickup_datetime) + ORDER BY pickup_hour + """ +) + +st.line_chart( + hourly_trips, + x="pickup_hour", + y="trip_count", + x_label="Pickup hour (0–23)", + y_label="Number of trips", +) + + +# Panel 3: Data freshness +st.subheader("Data freshness") + +freshness_df = run_query( + f""" + SELECT + COUNT(*) AS row_count, + MAX(pickup_datetime) AS latest_pickup_datetime + FROM {DB_SCHEMA}.fct_trips + {where_clause} + """ +) + +freshness = freshness_df.iloc[0] + +fresh_col1, fresh_col2 = st.columns(2) + +fresh_col1.metric( + "Row count", + f"{int(freshness['row_count']):,}", +) + +fresh_col2.metric( + "Latest pickup datetime", + freshness["latest_pickup_datetime"].strftime( + "%Y-%m-%d %H:%M:%S" + ), +) \ No newline at end of file diff --git a/week11-streamlit/metric_definitions.md b/week11-streamlit/metric_definitions.md new file mode 100644 index 0000000..4725afa --- /dev/null +++ b/week11-streamlit/metric_definitions.md @@ -0,0 +1,68 @@ +# Metric definitions + +Five fields per metric: Name, Description, Calculation, Data source, Refresh frequency. One block per panel. Copy this file to `metric_definitions.md` inside your `week11-streamlit/` folder and fill it in. + +## Metabase panels + + + +### Panel 1: Trip Count by Payment Type + +- **Name**: Trip Count by Payment Type +- **Description**: Number of taxi trips grouped by payment method. Missing payment labels are shown as Unknown. +- **Calculation**: Count all rows using COUNT(\*), grouped by COALESCE(payment_type_label, 'Unknown'). +- **Data source**: fct_trips in the Week 10 development schema; field payment_type_label. +- **Refresh frequency**: After every successful refresh of the Week 10 dbt mart. + +### Panel 2: Average fare per mile by dropoff borough + +- **Name**: Average fare per mile by dropoff borough +- **Description**: Average fare charged per mile for trips ending in each borough. Missing dropoff boroughs are shown as Unknown. +- **Calculation**: AVG(fare_per_mile) grouped by COALESCE(dropoff_borough, 'Unknown'); null fare_per_mile values are excluded by the average. +- **Data source**: fct_trips in the Week 10 development schema; fields fare_per_mile and dropoff_borough. +- **Refresh frequency**: After every successful refresh of the Week 10 dbt mart. + +### Panel 3: Average trip duration by hour of day + +- **Name**: Average trip duration by hour of day +- **Description**: Average trip duration in minutes grouped by the pickup hour from 0 through 23 +- **Calculation**: Average of EXTRACT(EPOCH FROM (dropoff_datetime - pickup_datetime)) / 60, grouped by EXTRACT(HOUR FROM pickup_datetime). Trips with missing timestamps or negative durations are excluded. +- **Data source**: fct_trips in the Week 10 development schema; fields pickup_datetime and dropoff_datetime. +- **Refresh frequency**: After every successful refresh of the Week 10 dbt mart. + +## Streamlit panels + + + +### Panel 1: Headline KPIs + +- **Name**: Headline trip KPIs +- **Description**: Shows total trips, average trip distance in miles, and average fare per mile. Results are recalculated for the selected payment type. +- **Calculation**: COUNT(\*), AVG(trip_distance), and AVG(fare_per_mile). When a payment type is selected, rows are filtered using payment_type_label before aggregation. +- **Data source**: fct_trips in the schema configured by DB_SCHEMA; fields trip_distance, fare_per_mile, and payment_type_label. + +- **Refresh frequency**: After every successful dbt mart refresh; Streamlit query results can remain cached for up to five minutes. + + + +### Panel 2: Trip count by hour of day + +- **Name**: Trip count by hour of day +- **Description**: Shows the number of trips beginning during each hour from 0 through 23. Results follow the selected payment-type filter. +- **Calculation**: COUNT(\*) grouped by EXTRACT(HOUR FROM pickup_datetime). +- **Data source**: fct_trips in the schema configured by DB_SCHEMA; fields pickup_datetime and payment_type_label. + +- **Refresh frequency**: After every successful dbt mart refresh; Streamlit query results can remain cached for up to five minutes. + +### Panel 3: Data freshness + +- **Name**: Trip-data freshness +- **Description**: Shows the number of rows and newest pickup timestamp in the current payment-type selection. The timestamp indicates data coverage and is not the pipeline execution time. +- **Calculation**: COUNT(\*) for row count and MAX(pickup_datetime) for the newest trip. +- **Data source**: fct_trips in the schema configured by DB_SCHEMA; fields pickup_datetime and payment_type_label. + +- **Refresh frequency**: After every successful dbt mart refresh; Streamlit query results can remain cached for up to five minutes.