Pavel Tisner

AI_ASSIST.md

@@ -4,31 +4,19 @@ Document one place you used an LLM during this assignment.
 
 ## The problem
 
-<!-- TODO: describe the specific problem you asked an LLM about.
-     Example: "My Streamlit KPI panel kept re-querying Postgres on every
-     sidebar interaction even though I wrapped run_query in @st.cache_data." -->
-
-TODO
+I needed to implement a payment type filter in my Streamlit dashboard that would update all dashboard panels (KPIs, hourly trend, and data freshness) using a single sidebar selection.
 
 ## The prompt
 
-<!-- TODO: paste the exact prompt you sent to the LLM. -->
-
-TODO
+How can I implement a Streamlit sidebar filter for `payment_type_label` so that it updates all dashboard panels? I already have KPI queries, an hourly trip chart, and a data freshness panel that read from `fct_trips`. I want one `st.sidebar.selectbox` with an "All" option that applies the same SQL filter to every query while keeping `@st.cache_data`.
 
 ## The response
 
-<!-- TODO: summarise or paste what the LLM returned. -->
-
-TODO
+The LLM suggested creating a sidebar `selectbox`, building a reusable SQL `WHERE` clause based on the selected payment type, and inserting that clause into every query. It also suggested escaping single quotes in the selected value before building the SQL string.
 
 ## Reflection
 
-<!-- TODO: what did you change, keep, or discard after reviewing the LLM's answer?
-     Be specific: "I kept the cache_data suggestion but changed ttl from 60 to 300
-     to match the mart's once-a-day refresh cadence." -->
-
-TODO
+I kept the overall approach of creating one reusable `where_clause` and applying it to every query. I also kept the handling of the "All" option. I reviewed the generated code, integrated it into my existing application, and verified that all dashboard panels updated correctly when the payment type changed.
 
 ---
 

README.md

@@ -48,13 +48,16 @@ Fill in `week11-streamlit/metric_definitions.md`: a five-field definition (name,
 
 ## My submission
 
-<!-- TODO: 1) Save your Metabase dashboard into the shared "Week 11 Submissions" collection.
-     2) Paste its link below, plus screenshots or a PDF export in this repo.
-     3) Paste your 5-minute presentation recording link (keep it PRIVATE). -->
+- **Metabase dashboard (Week 11 Submissions collection):**
+  https://metabase-hyf.politepebble-abd3ebc2.westeurope.azurecontainerapps.io/dashboard/49-nyc-taxi-analytics-pavel-tisner
 
-- Metabase dashboard (in the **Week 11 Submissions** collection): TODO
-- Screenshots / PDF export: TODO
-- Presentation recording (private, hosted in the Azure `student-submissions` container): TODO
+- **Screenshots / PDF export:**
+  - `docs/Metabase - NYC Taxi Analytics_ Pavel Tisner.pdf`
+  - `docs/streamlit-dashboard.png`
+  - `docs/streamlit-dashboard-filter.png`
+
+- **Presentation recording (private, hosted in the Azure `student-submissions` container:**
+  TODO (will be added after recording)
 
 > ⚠️ **Keep the recording private.** It shows your screen and voice. Never make it public and never commit the `.mp4` (git history is forever). Check the frame for passwords, `.env` contents, or connection strings before uploading.
 

week11-streamlit/app.py

@@ -30,16 +30,110 @@ def run_query(sql: str) -> pd.DataFrame:
     with engine.connect() as conn:
         return pd.read_sql(sql, conn)
 
+payment_types_query = f"""
+SELECT DISTINCT payment_type_label
+FROM {DB_SCHEMA}.fct_trips
+WHERE payment_type_label IS NOT NULL
+ORDER BY payment_type_label
+"""
+
+payment_types = run_query(payment_types_query)["payment_type_label"].tolist()
+
+selected_payment = st.sidebar.selectbox(
+    "Payment Type",
+    ["All"] + payment_types,
+)
+
+if selected_payment == "All":
+    where_clause = ""
+else:
+    escaped_payment = selected_payment.replace("'", "''")
+    where_clause = (
+        f"WHERE payment_type_label = '{escaped_payment}'"
+    )
 
 st.subheader("Headline KPIs")
 
-# TODO: query total trip count, average trip_distance, and average
-# fare_per_mile from {DB_SCHEMA}.fct_trips through run_query(), then
-# render three tiles side by side with st.columns(3) and .metric().
-# This is deliberately not the total-trips/avg-fare/total-revenue trio
-# from the chapter: trip_distance and fare_per_mile are different columns,
-# so copying the chapter's SQL verbatim will not answer this.
-raise NotImplementedError(
-    "TODO: implement the headline KPIs panel (total trips, avg trip "
-    "distance, avg fare per mile) from fct_trips."
+kpi_query = f"""
+SELECT
+    COUNT(*) AS total_trips,
+    AVG(trip_distance) AS avg_trip_distance,
+    AVG(fare_per_mile) AS avg_fare_per_mile
+FROM {DB_SCHEMA}.fct_trips
+{where_clause}
+"""
+
+kpi_data = run_query(kpi_query).iloc[0]
+
+col1, col2, col3 = st.columns(3)
+
+col1.metric(
+    label="Total Trips",
+    value=f"{int(kpi_data['total_trips']):,}",
+)
+
+col2.metric(
+    label="Average Trip Distance",
+    value=(
+        f"{kpi_data['avg_trip_distance']:.2f} miles"
+        if pd.notna(kpi_data["avg_trip_distance"])
+        else "No data"
+    ),
 )
+
+col3.metric(
+    label="Average Fare per Mile",
+    value=(
+        f"${kpi_data['avg_fare_per_mile']:.2f}"
+        if pd.notna(kpi_data["avg_fare_per_mile"])
+        else "No data"
+    ),
+)
+
+st.subheader("Trips by Hour of Day")
+
+hourly_query = f"""
+SELECT
+    EXTRACT(HOUR FROM pickup_datetime) AS pickup_hour,
+    COUNT(*) AS trip_count
+FROM {DB_SCHEMA}.fct_trips
+{where_clause}
+GROUP BY pickup_hour
+ORDER BY pickup_hour
+"""
+
+hourly_data = run_query(hourly_query)
+
+st.line_chart(
+    hourly_data,
+    x="pickup_hour",
+    y="trip_count",
+)
+
+st.subheader("Data Freshness")
+
+freshness_query = f"""
+SELECT
+    COUNT(*) AS row_count,
+    MAX(pickup_datetime) AS latest_pickup
+FROM {DB_SCHEMA}.fct_trips
+{where_clause}
+"""
+
+freshness = run_query(freshness_query).iloc[0]
+
+col1, col2 = st.columns(2)
+
+col1.metric(
+    label="Row Count",
+    value=f"{int(freshness['row_count']):,}",
+)
+
+col2.metric(
+    label="Latest Pickup",
+    value=(
+        freshness["latest_pickup"].strftime("%Y-%m-%d %H:%M:%S")
+        if pd.notna(freshness["latest_pickup"])
+        else "No data"
+    ),
+)

week11-streamlit/metric_definitions.md

@@ -0,0 +1,84 @@
+# Metric definitions
+
+Five fields per metric: Name, Description, Calculation, Data source, Refresh frequency. One block per panel.
+
+## Metabase panels
+
+### Panel 1: Trip Count by Payment Type
+
+- **Name**: Trip Count by Payment Type
+- **Description**: The number of NYC taxi trips recorded for each payment type. This metric shows which payment methods are used most frequently and highlights less common payment categories.
+- **Calculation**: Count all rows in `fct_trips`, grouped by `payment_type_label`: `COUNT(*) GROUP BY payment_type_label`.
+- **Data source**: `dev_pavel_tisner.fct_trips` in the Azure PostgreSQL `team1` database.
+- **Refresh frequency**: Recalculated when the Metabase Question or dashboard is refreshed. The underlying data changes when the dbt mart is rebuilt.
+
+### Panel 2: Average Fare per Mile by Dropoff Borough
+
+- **Name**: Average Fare per Mile by Dropoff Borough
+- **Description**: The average fare per mile for trips ending in each dropoff borough. It compares normalized trip costs across destinations and includes the `Unknown` and `NaN` location categories present in the mart.
+- **Calculation**: Calculate `AVG(fare_per_mile)` for all trips, grouped by `dropoff_borough`.
+- **Data source**: `dev_pavel_tisner.fct_trips` in the Azure PostgreSQL `team1` database. The `fare_per_mile` field is calculated in the dbt mart from fare amount and trip distance.
+- **Refresh frequency**: Recalculated when the Metabase Question or dashboard is refreshed. The underlying data changes when the dbt mart is rebuilt.
+
+### Panel 3: Average Trip Duration by Hour of Day
+
+- **Name**: Average Trip Duration by Hour of Day
+- **Description**: The average duration in minutes of trips grouped by their pickup hour. It shows how average journey duration changes throughout the day.
+- **Calculation**: For every trip, calculate duration as `EXTRACT(EPOCH FROM (dropoff_datetime - pickup_datetime)) / 60`. Average the result and group it by `EXTRACT(HOUR FROM pickup_datetime)`.
+- **Data source**: `dev_pavel_tisner.fct_trips` in the Azure PostgreSQL `team1` database.
+- **Refresh frequency**: Recalculated when the Metabase Question or dashboard is refreshed. The underlying data changes when the dbt mart is rebuilt.
+
+## Streamlit panels
+
+The selected `payment_type_label` filter is applied to every Streamlit metric.
+When `All` is selected, no payment-type filter is applied.
+
+### Panel 1: Headline KPIs
+
+#### Metric 1: Total Trips
+
+- **Name**: Total Trips
+- **Description**: The total number of taxi trips in the current payment-type selection.
+- **Calculation**: Count all rows after applying the selected payment-type filter: `COUNT(*)`.
+- **Data source**: `dev_pavel_tisner.fct_trips` in the Azure PostgreSQL `team1` database.
+- **Refresh frequency**: Recalculated when the Streamlit app reruns or the filter changes. Query results are cached for up to five minutes, and the underlying data changes when the dbt mart is rebuilt.
+
+#### Metric 2: Average Trip Distance
+
+- **Name**: Average Trip Distance
+- **Description**: The average trip distance in miles for trips in the current payment-type selection.
+- **Calculation**: Calculate `AVG(trip_distance)` after applying the selected payment-type filter.
+- **Data source**: `dev_pavel_tisner.fct_trips` in the Azure PostgreSQL `team1` database.
+- **Refresh frequency**: Recalculated when the Streamlit app reruns or the filter changes. Query results are cached for up to five minutes, and the underlying data changes when the dbt mart is rebuilt.
+
+#### Metric 3: Average Fare per Mile
+- **Name**: Average Fare per Mile
+- **Description**: The average fare charged per mile for trips in the current payment-type selection.
+- **Calculation**: Calculate `AVG(fare_per_mile)` after applying the selected payment-type filter.
+- **Data source**: `dev_pavel_tisner.fct_trips` in the Azure PostgreSQL `team1` database. The `fare_per_mile` field is calculated in the dbt mart from fare amount and trip distance.
+- **Refresh frequency**: Recalculated when the Streamlit app reruns or the filter changes. Query results are cached for up to five minutes, and the underlying data changes when the dbt mart is rebuilt.
+
+### Panel 2: Trips by Hour of Day
+
+#### Metric 4: Trip Count by Pickup Hour
+- **Name**: Trip Count by Pickup Hour
+- **Description**: The number of trips beginning during each hour of the day for the current payment-type selection. It shows daily demand patterns across the 24-hour period.
+- **Calculation**: Extract the hour from `pickup_datetime`, count rows, and group by pickup hour: `COUNT(*) GROUP BY EXTRACT(HOUR FROM pickup_datetime)`.
+- **Data source**: `dev_pavel_tisner.fct_trips` in the Azure PostgreSQL `team1` database.
+- **Refresh frequency**: Recalculated when the Streamlit app reruns or the filter changes. Query results are cached for up to five minutes, and the underlying data changes when the dbt mart is rebuilt.
+
+### Panel 3: Data Freshness
+
+#### Metric 5: Row Count
+- **Name**: Row Count
+- **Description**: The number of rows currently available in `fct_trips` for the selected payment type. It provides a basic completeness check and should match Total Trips for the same selection.
+- **Calculation**: Count all rows after applying the selected payment-type filter: `COUNT(*)`.
+- **Data source**: `dev_pavel_tisner.fct_trips` in the Azure PostgreSQL `team1` database.
+- **Refresh frequency**: Recalculated when the Streamlit app reruns or the filter changes. Query results are cached for up to five minutes, and the underlying data changes when the dbt mart is rebuilt.
+
+#### Metric 6: Latest Pickup Datetime
+- **Name**: Latest Pickup Datetime
+- **Description**: The most recent pickup timestamp represented in the current payment-type selection. It indicates how far the dataset extends in event time, but it is not the timestamp of the latest pipeline run.
+- **Calculation**: Select the maximum pickup timestamp after applying the selected payment-type filter: `MAX(pickup_datetime)`.
+- **Data source**: `dev_pavel_tisner.fct_trips` in the Azure PostgreSQL `team1` database.
+- **Refresh frequency**: Recalculated when the Streamlit app reruns or the filter changes. Query results are cached for up to five minutes, and the underlying data changes when the dbt mart is rebuilt.