Skip to content

Commit f9f5f02

Browse files
committed
add notebook to contributing
1 parent 5bc770b commit f9f5f02

1 file changed

Lines changed: 35 additions & 0 deletions

File tree

mkdocs/docs/contributing.md

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -211,6 +211,41 @@ export PYICEBERG_CATALOG__TEST_CATALOG__ACCESS_KEY_ID=username
211211
export PYICEBERG_CATALOG__TEST_CATALOG__SECRET_ACCESS_KEY=password
212212
```
213213

214+
## Notebooks for Experimentation
215+
216+
PyIceberg provides Jupyter notebooks for quick experimentation and learning. Two Make commands are available depending on your needs:
217+
218+
### PyIceberg Examples (`make notebook`)
219+
220+
For basic PyIceberg experimentation without additional infrastructure:
221+
222+
```bash
223+
make notebook
224+
```
225+
226+
This will install notebook dependencies and launch Jupyter Lab in the `notebooks/` directory.
227+
228+
**PyIceberg Example Notebook** (`notebooks/pyiceberg_example.ipynb`) is based on the [Getting Started with PyIceberg](https://py.iceberg.apache.org/#getting-started-with-pyiceberg) page. It demonstrates basic PyIceberg operations like creating catalogs, schemas, and querying tables without requiring any external services.
229+
230+
### Spark Integration Examples (`make notebook-infra`)
231+
232+
For working with PyIceberg alongside Spark, use the infrastructure-enabled notebook environment:
233+
234+
```bash
235+
make notebook-infra
236+
```
237+
238+
This command spins up the full integration test infrastructure via Docker Compose, including:
239+
240+
- **Spark** (with Spark Connect)
241+
- **Iceberg REST Catalog** (using the [`apache/iceberg-rest-fixture`](https://hub.docker.com/r/apache/iceberg-rest-fixture) image)
242+
- **Hive Metastore**
243+
- **S3-compatible object storage** (Minio)
244+
245+
**Spark Example Notebook** (`notebooks/spark_integration_example.ipynb`) is based on the [Spark Getting Started](https://iceberg.apache.org/docs/nightly/spark-getting-started/) guide. This notebook demonstrates how to work with PyIceberg alongside Spark, leveraging the Docker-based testing setup for a complete local development environment.
246+
247+
After running `make notebook-infra`, open `spark_integration_example.ipynb` in the Jupyter Lab interface to explore Spark integration capabilities.
248+
214249
## Code standards
215250

216251
Below are the formalized conventions that we adhere to in the PyIceberg project. The goal of this is to have a common agreement on how to evolve the codebase, but also using it as guidelines for newcomers to the project.

0 commit comments

Comments
 (0)