Built using Microsoft Azure, Python & Power BI
By Krishna Chaitanya Muttevi
youtube demo: https://youtu.be/wWYqg14s9uY
RetailX is a cloud-native, real-time retail data analytics pipeline that simulates live transaction data and processes it through an event-driven Azure ecosystem. The data is visualized using interactive Power BI dashboards.
This project showcases hands-on experience with real-time data ingestion, transformation, cloud storage, and visualization using tools like Azure Event Hub, Stream Analytics, ADLS Gen2, and Power BI.
| Component | Purpose |
|---|---|
| Azure Event Hub | Real-time ingestion |
| Azure Stream Analytics | SQL-based stream processing |
| Azure Data Lake Gen2 | Cloud storage for JSON data |
| Power BI | Data visualization |
| Python | Transaction simulator |
| JSON | Streaming data format |
The following architecture demonstrates the complete flow from simulation to visualization:
-
Publishes transaction events to Azure Event Hub in real-time.
-
Written in Python (
simulate transactions.py). -
Each message contains fields like timestamp, product, quantity, price, etc.
- Created namespace and Event Hub.
- Generated Shared Access Policy for authentication.
- Connected Python script to push data.
-
Created storage account with Hierarchical namespace.
-
Created container and directory to hold processed output files (JSON).
-
Created job to read from Event Hub.
-
Defined input schema.
-
Output written to ADLS Gen2 as JSON.
-
Live queries tested on streaming input.
Power BI connects directly to ADLS Gen2 to visualize real-time output.
-
Connected to Azure Blob Storage.
-
Navigated to the container directory.
-
Transformed raw JSON into tabular format.
-
Built interactive dashboards with charts and KPIs.
