Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions core-spec/expression_language.md
Original file line number Diff line number Diff line change
Expand Up @@ -643,6 +643,8 @@ a IS DISTINCT FROM b -- TRUE if one is NULL and other isn't

Ossie implementations MAY support additional functions through dialect-specific extensions. When using dialect extensions, the expression must specify the dialect.

Standardized dialect identifiers in the core spec include `ANSI_SQL`, `SNOWFLAKE`, `MDX`, `DAX`, `TABLEAU`, `DATABRICKS`, `MAQL`, and `BIGQUERY`.

The Ossie dialect should always be supported. Other dialects MAY be ignored. There is no guarantee that all different dialects for an expression will act the same, so implementations should be consistent with their dialect handling. This means that if an Ossie model has an expression written in two dialects, the implementation should deterministically choose which dialect to use.

### Declaring Dialect-Specific Expressions
Expand All @@ -656,18 +658,20 @@ expression:
expression: DATE_TRUNC('month', order_date)
- dialect: BIGQUERY
expression: DATE_TRUNC(order_date, MONTH)
- dialect: DAX
expression: DATE(YEAR(order_date), MONTH(order_date), 1)
```

### Common Dialect Variations

| Function | ANSI\_SQL | Snowflake | BigQuery | Databricks | PostgreSQL |
| :---- | :---- | :---- | :---- | :---- | :---- |
| Date truncation | `DATE_TRUNC('month', d)` | `DATE_TRUNC('month', d)` | `DATE_TRUNC(d, MONTH)` | `DATE_TRUNC('month', d)` | `DATE_TRUNC('month', d)` |
| Date add | `DATEADD(day, 7, d)` | `DATEADD(day, 7, d)` | `DATE_ADD(d, INTERVAL 7 DAY)` | `DATE_ADD(d, 7)` | `d + INTERVAL '7 days'` |
| String concat | `CONCAT(a, b)` | `CONCAT(a, b)` | `CONCAT(a, b)` | `CONCAT(a, b)` | `a || b` |
| Null coalesce | `COALESCE(a, b)` | `COALESCE(a, b)` or `NVL(a, b)` | `COALESCE(a, b)` or `IFNULL(a, b)` | `COALESCE(a, b)` | `COALESCE(a, b)` |
| Current timestamp | `CURRENT_TIMESTAMP` | `CURRENT_TIMESTAMP()` | `CURRENT_TIMESTAMP()` | `CURRENT_TIMESTAMP()` | `CURRENT_TIMESTAMP` |
| Substring | `SUBSTRING(s, start, len)` | `SUBSTR(s, start, len)` | `SUBSTR(s, start, len)` | `SUBSTRING(s, start, len)` | `SUBSTRING(s, start, len)` |
| Function | ANSI\_SQL | Snowflake | BigQuery | Databricks | PostgreSQL | DAX |
| :---- | :---- | :---- | :---- | :---- | :---- | :---- |
| Date truncation | `DATE_TRUNC('month', d)` | `DATE_TRUNC('month', d)` | `DATE_TRUNC(d, MONTH)` | `DATE_TRUNC('month', d)` | `DATE_TRUNC('month', d)` | `DATE(YEAR(d), MONTH(d), 1)` |
| Date add | `DATEADD(day, 7, d)` | `DATEADD(day, 7, d)` | `DATE_ADD(d, INTERVAL 7 DAY)` | `DATE_ADD(d, 7)` | `d + INTERVAL '7 days'` | `DATEADD(d, 7, DAY)` |
| String concat | `CONCAT(a, b)` | `CONCAT(a, b)` | `CONCAT(a, b)` | `CONCAT(a, b)` | `CONCAT_WS('', a, b)` | `CONCATENATE(a, b)` or `a & b` |
| Null coalesce | `COALESCE(a, b)` | `COALESCE(a, b)` or `NVL(a, b)` | `COALESCE(a, b)` or `IFNULL(a, b)` | `COALESCE(a, b)` | `COALESCE(a, b)` | `COALESCE(a, b)` |
| Current timestamp | `CURRENT_TIMESTAMP` | `CURRENT_TIMESTAMP()` | `CURRENT_TIMESTAMP()` | `CURRENT_TIMESTAMP()` | `CURRENT_TIMESTAMP` | `NOW()` or `UTCNOW()` |
| Substring | `SUBSTRING(s, start, len)` | `SUBSTR(s, start, len)` | `SUBSTR(s, start, len)` | `SUBSTRING(s, start, len)` | `SUBSTRING(s, start, len)` | `MID(s, start, len)` |

###

Expand Down
4 changes: 2 additions & 2 deletions core-spec/osi-schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,12 +23,12 @@
"$defs": {
"Dialect": {
"type": "string",
"enum": ["ANSI_SQL", "SNOWFLAKE", "MDX", "TABLEAU", "DATABRICKS", "MAQL", "BIGQUERY"],
"enum": ["ANSI_SQL", "SNOWFLAKE", "MDX", "DAX", "TABLEAU", "DATABRICKS", "MAQL", "BIGQUERY"],
"description": "Supported SQL and expression language dialects"
},
"Vendor": {
"type": "string",
"examples": ["COMMON", "SNOWFLAKE", "SALESFORCE", "DBT", "DATABRICKS", "GOODDATA", "WISDOM"],
"examples": ["COMMON", "SNOWFLAKE", "SALESFORCE", "DBT", "DATABRICKS", "GOODDATA", "WISDOM", "MICROSOFT"],
"description": "Vendor name for custom extensions. Any string value is accepted."
},
"AIContext": {
Expand Down
1 change: 1 addition & 0 deletions core-spec/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ Supported SQL and expression language dialects for metrics and field definitions
| `ANSI_SQL` | Standard SQL dialect |
| `SNOWFLAKE` | Snowflake SQL |
| `MDX` | Multi-Dimensional Expressions |
| `DAX` | Data Analysis Expressions |
| `TABLEAU` | Tableau calculations |
| `DATABRICKS` | Databricks SQL |
| `MAQL` | GoodData MAQL (Metric Analysis and Query Language) |
Expand Down
1 change: 1 addition & 0 deletions core-spec/spec.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,7 @@ dialects:
- "ANSI_SQL" # Standard SQL dialect
- "SNOWFLAKE" # Snowflake
- "MDX" # Multi-Dimensional Expressions
- "DAX" # Data Analysis Expressions
- "TABLEAU" # Tableau
Comment thread
MikeCarlo marked this conversation as resolved.
- "DATABRICKS" # Databricks SQL
- "MAQL" # GoodData MAQL (Multi-Dimensional Analytical Query Language)
Expand Down
2 changes: 2 additions & 0 deletions examples/tpcds_semantic_model.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -517,6 +517,8 @@ semantic_model:
dialects:
- dialect: ANSI_SQL
expression: SUM(store_sales.ss_ext_sales_price)
- dialect: DAX
expression: SUM(store_sales.ss_ext_sales_price)
description: Total sales revenue across all transactions
ai_context:
synonyms:
Expand Down
1 change: 1 addition & 0 deletions python/src/ossie/models.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ class OSIDialect(str, Enum):
ANSI_SQL = "ANSI_SQL"
SNOWFLAKE = "SNOWFLAKE"
MDX = "MDX"
DAX = "DAX"
MAQL = "MAQL"
TABLEAU = "TABLEAU"
DATABRICKS = "DATABRICKS"
Expand Down
3 changes: 2 additions & 1 deletion validation/validate.py
Original file line number Diff line number Diff line change
Expand Up @@ -67,12 +67,13 @@
"DATABRICKS": "databricks",
"BIGQUERY": "bigquery",
"MDX": None, # Not supported by sqlglot, skip validation
"DAX": None, # Not supported by sqlglot, skip validation
"TABLEAU": None, # Not supported by sqlglot, skip validation
"MAQL": None, # Not supported by sqlglot, skip validation
}

# Dialects that sqlglot cannot parse
SKIP_SQL_VALIDATION = {"MDX", "TABLEAU", "MAQL"}
SKIP_SQL_VALIDATION = {"MDX", "DAX", "TABLEAU", "MAQL"}


def validate_schema(data: dict, schema: dict) -> list[str]:
Expand Down