Generate standardized JSON data contracts directly from Dataiku datasets.
This plugin automatically extracts dataset schema, metadata, and governance information to create machine-readable JSON data contracts that can be shared across teams and integrated into governance workflows.
One of the biggest challenges in modern data platforms is that datasets rarely come with clear ownership, expectations, or documentation.
Data contracts help solve this by creating a shared agreement between data producers and consumers.
This plugin automates much of that process by generating standardized contracts directly from your Dataiku projects.
- Automatically extracts dataset schema
- Captures table and column descriptions
- Generates standardized JSON data contracts
- Supports optional column-level tags
- Supports optional classifications and governance categories
- Allows administrators to configure metadata values
- Optionally allows users to enter custom tag values
- Infers numeric precision (
multipleOf) where possible - Creates managed folder automatically
Download this repository as a zip file and upload it in your Dataiku instance under Administration & Settings -> Plugins.
Before using the Data Contract Generator macro, configure where generated contracts should be stored and which optional metadata fields your organization wants to support.
The macro writes generated contracts to a project managed folder named data_contracts. In the macro settings, choose a connection that allows managed folders, like Dataiku filesystem, Databricks/Snowflake, S3/ADLS/GCP, or another file-based connection.
Before running the macro, make sure an admin does one of the following with the chosen connection:
- Create a managed folder in the project named
data_contracts, or - Make sure the connection selected in the plugin settings allows new managed folders to be created.
To allow the macro to create the folder automatically, a Dataiku administrator should enable managed folder creation on the selected connection.
The setting is found in the Admin Settings -> Connections under the connection’s "Usage Params".
If Allow managed folders is not enabled for the selected connection, the macro will not be able to create the data_contracts folder automatically. In that case, create the managed folder manually or choose another connection that supports managed folder creation.
The plugin can optionally include column-level metadata in the generated data contract. Use the plugin settings to choose which metadata fields your organization wants to support:
- Tags
- Classifications
- Categories
If a metadata type is enabled, it will appear in the macro screen when users generate a data contract. If it is disabled, that field will be hidden from the macro and omitted from the generated JSON.
Tags are written as an array, so users can select more than one tag for the same column. Classifications and categories are single-select per column.
Add the tag values that users should be able to select for columns when generating data contracts.
Enable Allow custom tag values if users should be able to type their own tag values in addition to selecting from the configured list. Leave this unchecked if users should only use the approved tag values configured by the plugin administrator. Custom tags are added to the same tag array as configured dropdown tags.
If custom tag values are not enabled, users can only choose from the tag values configured in the plugin settings.
Add the classification values that users should be able to select for columns.
Enable classifications only if your organization uses a formal classification taxonomy. If classifications are not needed, disable this option and the generated contract will omit the classification field.
Add the category values that users should be able to select for columns.
Enable categories only if your organization groups sensitive or governed data into categories. If categories are not needed, disable this option and the generated contract will omit the category field.
At a minimum, select the filesystem for the managed folder and decide which metadata fields should be available to users. Then add the approved values for any enabled metadata fields. This ensures users can generate consistent data contracts while still allowing each organization to use its own terminology. It is recommended to allow managed folder creation on your filesystem connection, otherwise an admin will have to manually create the managed folder in each project.
After the plugin is configured, select the dataset you want to document and run the Generate Data Contract macro, located under "Other Actions".
In the macro screen:
- The selected dataset is prepopulated but you can change it to a different one.
- Add one row for each column that needs additional metadata.
- Select the column.
- Choose one or more tags, if tags are enabled.
- Choose a classification, if classifications are enabled.
- Choose a category, if categories are enabled.
- Run the macro.
The plugin will generate a JSON data contract and save it to the project’s data_contracts managed folder.
{
"type": "object",
"properties": {
"transaction_id": {
"order": 1,
"type": "string",
"title": "transaction_id",
"description": "Unique transaction identifier"
},
"customer_id": {
"order": 2,
"type": "string",
"title": "customer_id",
"description": "Unique customer identifier",
"tag": ["id", "sensitive"],
"category": "PII"
},
"amount": {
"order": 3,
"type": "number",
"title": "amount",
"description": "Transaction amount",
"multipleOf": 0.01
},
"transaction_date": {
"order": 4,
"type": "string",
"title": "transaction_date",
"description": "Date of the transaction",
"format": "date"
},
"created_at": {
"order": 5,
"type": "string",
"title": "created_at",
"description": "Timestamp when the record was created",
"format": "date-time"
}
},
"description": "Customer transaction history used for fraud detection."
}Contracts are written as
<dataset_name>_data_contract.json
inside a Dataiku managed folder
data_contracts/
If the data_contracts managed folder does not already exist, the macro attempts to create it on the storage connection selected in the plugin settings. A Dataiku administrator may need to enable managed folder creation on that connection.
- Developed for DSS 12.7, tested and compatible through 14.7
- YAML contract support
This project accompanies my blog series on data contracts.
- Part 1 – The Case for Data Contracts
- Part 2 – Operationalizing Data Contracts with Dataiku (coming soon)
Contributions are welcome. Please open an issue or pull request if you find a bug or want to improve the plugin.
Apache 2.0
This is an independent open-source project built using the Dataiku plugin framework and is not an officially supported Dataiku product. For issues, please open a pull request or contact julia.hofmeister@dataiku.com.