Skip to content

Commit 24079ea

Browse files
authored
Merge 5fe5a5d into effaaa9
2 parents effaaa9 + 5fe5a5d commit 24079ea

5 files changed

Lines changed: 3061 additions & 11 deletions

File tree

AzurePortal/1_MedallionArch/README.md

Lines changed: 21 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Costa Rica
55
[![GitHub](https://img.shields.io/badge/--181717?logo=github&logoColor=ffffff)](https://github.com/)
66
[brown9804](https://github.com/brown9804)
77

8-
Last updated: 2025-01-29
8+
Last updated: 2025-02-21
99

1010
------------------------------------------
1111

@@ -73,34 +73,45 @@ Implementing a medallion architecture provides several benefits:
7373

7474
### Step 1: Set Up Your Environment
7575

76+
> [!NOTE]
77+
> If you need more visual help, check out the video that walks you through the steps. Below is a detailed step-by-step guide.
78+
7679
1. **Create a Fabric Workspace**: This will be your central hub for all activities.
77-
- Navigate to the Fabric portal.
78-
- Click on `Create a resource` and select `Fabric Workspace`.
80+
- Click on `Create a resource` and create your Fabric Capacity if you haven't done so already.
81+
82+
<img width="550" alt="image" src="https://github.com/user-attachments/assets/8f259cac-1dcb-4129-9070-0b31899c4ab4">
83+
84+
- Go to the [Fabric portal](https://app.fabric.microsoft.com/).
85+
- Click on `Workspaces`, then select either your existing workspace or create a new one by clicking `New Workspace`:
7986
- Provide a name and other required details, then create the workspace.
8087

81-
<img width="550" alt="image" src="https://github.com/user-attachments/assets/8f259cac-1dcb-4129-9070-0b31899c4ab4">
8288

8389
<img width="550" alt="image" src="https://github.com/user-attachments/assets/2f3225fc-6aa6-4eeb-8207-75038b36f18f">
8490

91+
- Now, assign the Fabric Capacity to your workspace by clicking on `Workspace settings` and selecting the fabric capacity under the license.
92+
8593
<img width="550" alt="image" src="https://github.com/user-attachments/assets/1831c97d-6b9a-4470-968d-e7803bc58b80">
8694

95+
https://github.com/user-attachments/assets/c524741c-be91-4fe4-82bc-c841fae8c6c9
96+
8797
2. **Create Lakehouses**: Set up three lakehouses for the Bronze, Silver, and Gold layers.
98+
8899
- In your Fabric workspace, create three lakehouses named `raw_Bronze`, `cleansed_Silver`, and `curated_Gold`.
89100

90-
| Go to Data Engineering | Create the 3 lakehouses |
91-
| --- | --- |
92-
|<img src="https://github.com/user-attachments/assets/f0f5baf2-9ad2-4422-9089-14aa28ae5181" width="250"> | <img src="https://github.com/user-attachments/assets/5827c73a-1291-4fb6-a2c6-0c61f4808164" width="550"> |
101+
<img src="https://github.com/user-attachments/assets/5827c73a-1291-4fb6-a2c6-0c61f4808164" width="550">
93102

94103
<img width="550" alt="image" src="https://github.com/user-attachments/assets/ca3c64c7-d414-4ec0-8cbc-cf39548bf758">
95104

96105
<img width="958" alt="image" src="https://github.com/user-attachments/assets/828adf9d-8722-4bef-8694-8c22de330797">
97106

107+
https://github.com/user-attachments/assets/fdb64dd2-a6ec-4da0-a385-e55f875c8f8e
108+
98109
### Step 2: Ingest Data into the Bronze Layer
99110
1. **Identify Data Sources**:
100111
- Determine the sources from which you'll ingest data.
101112
- List all the data sources such as databases, APIs, file systems, etc.
102113

103-
> Let's suppose you have your data locally as csv, you can upload your data to the raw layer:
114+
> Let's suppose you have your data locally as csv, you can upload your data to the raw layer. Click [to access sample files in case you don't have any data at the moment](./sample_files)
104115
105116
<img width="550" alt="image" src="https://github.com/user-attachments/assets/0ebd4ca9-80f6-4d1c-a28c-fc7ab6988538">
106117

@@ -110,6 +121,8 @@ Implementing a medallion architecture provides several benefits:
110121
| --- | --- |
111122
| <img width="550" alt="image" src="https://github.com/user-attachments/assets/09994e75-3029-4f61-aac8-b50f7c5fd2b1"> | <img width="550" alt="image" src="https://github.com/user-attachments/assets/2b25d187-85e2-48e7-9a97-e7549f28ed9c"> |
112123

124+
https://github.com/user-attachments/assets/56308a58-cf72-4f0f-bf3e-e9e1669fa0df
125+
113126
2. **Create Dataflows or Pipelines**: Use Data Factory to create dataflows or pipelines that ingest data into the Bronze lakehouse.
114127
- In Data Factory, create a new pipeline.
115128
- Add activities to the pipeline to ingest data from the identified sources.

0 commit comments

Comments
 (0)