Skip to content
6 changes: 3 additions & 3 deletions site/dolt/src/content/architecture/storage-engine.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ Databases are built on top of [Search Trees](https://en.wikipedia.org/wiki/Searc

Dolt is built on a novel Search Tree, closely related to a B-tree, called a Probabilistic B-Tree, or Prolly Tree for short. As far as we can tell, Prolly Trees were [invented by the Noms team specifically for database version control](https://github.com/attic-labs/noms/blob/master/doc/intro) and they also coined the term.

## B-Trees
### B-Trees

Most SQL databases you are familiar with, like [Postgres](https://www.postgresql.org/) or [MySQL](https://www.mysql.com/), are built on [B-tree](https://www.dolthub.com/blog/2020-04-01-how-dolt-stores-table-data/#b-tree-review) storage. Tables are represented as a map of primary keys to values and the keys of that map are stored in a B-tree. Values are stored in the leaf nodes.

Expand All @@ -66,7 +66,7 @@ However, finding the differences between two B-trees requires scanning both tree

Also, writes to B-trees are not history independent, the order of the writes internally changes the structure of the tree. Thus, storage cannot be easily shared between two versions of the same tree.

## Prolly Trees
### Prolly Trees

A Prolly Tree, or Probabilistic B-tree, is a content-addressed B-tree.

Expand All @@ -82,7 +82,7 @@ Moreover, sections of the tree that share the same root hash can share storage b

Prolly trees are described in more detail [here](/architecture/storage-engine/prolly-tree).

## Comparison
### Comparison

[The Noms documentation](https://github.com/attic-labs/noms/blob/master/doc/intro#some-properties-of-prolly-trees) provides the following useful algorithmic, big O() comparison of B-trees and Prolly Trees:

Expand Down
4 changes: 2 additions & 2 deletions site/dolt/src/content/introduction/use-cases/audit.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,11 +23,11 @@ If you're not ready to switch your primary database to Dolt to get its audit cap

## Dolt replaces...

## Soft Deletes
### Soft Deletes

A technique to add audit capability to an existing database is to add [soft deletes](https://www.dolthub.com/blog/2022-11-03-soft-deletes/). Soft delete is the use various techniques to mark data as inactive instead of deleting it. This is strictly worse than a version controlled database for audit purposes. With soft deletes, an operator can still modify data or the application can make mistakes. In Dolt, every write is part of the audit log. It is far more difficult for an operator to change Dolt history.

## Change Data Capture
### Change Data Capture

[Change Data Capture](https://www.dolthub.com/blog/2023-03-01-change-data-capture/) is another way to add audit capability to an existing database. Some change data capture techniques are similar to [soft delete](https://www.dolthub.com/blog/2022-11-03-soft-deletes/) strategies. Modern change data capture tools consume replication logs to audit database changes. Dolt can consume the same logs in the [versioned MySQL replica use case](/introduction/use-cases/versioned-replica) producing a simpler and thus, more audit-friendly, change data capture solution.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ This use case is particularly popular in video games where much of the game func

## Dolt replaces...

## Files in Git
### Files in Git

Most large configuration files are stored and versioned in Git. If the files get too large they are store in cloud storage and linked to Git using [git-lfs](https://git-lfs.com/). If the files are stored in git-lfs, you lose the ability to diff the contents of the files. Dolt improves the experience by adding query capabilities and large fine-grained diffs to the data stored in configuration files. The diff and merge experience will be greatly improved in Dolt for this type of data.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,11 @@ Lastly, [commits](/concepts/dolt/git/commits), [logs](/concepts/dolt/git/log), a

## Dolt replaces...

## Unstructured files in cloud storage
### Unstructured files in cloud storage

It is common practice to store copies of training data or database backups in cloud storage for model reproducibility. A full copy of the data is stored for every training run. This can become quite expensive and limit the amount of models you can reproduce. Dolt stores only the differences between stored versions decreasing the cost of data storage. Additionally, Dolt can produce diffs between versions of training data producing novel model insights.

## MySQL, Postgres, or other databases
### MySQL, Postgres, or other databases

Dolt can replace any database used to store and query data. Many of our customers switch from other OLTP databases like MySQL or Postgres to improve data and model quality through versioning. Customers have also switched to Dolt from document databases like MongoDB. Dolt's additional unique features like branches, diffs, and merges allow for human review of data changes and multiple parallel data projects.

Expand Down
4 changes: 2 additions & 2 deletions site/dolt/src/content/introduction/use-cases/data-sharing.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,11 +27,11 @@ Dolt and DoltHub are also great if vendors share data with you. When you receive

## Dolt replaces...

## Exchanging Files
### Exchanging Files

Dolt replaces exchanging flat data files like CSVs via email, FTP servers, or other file transfer techniques. Dolt allows data to maintain schema on exchange including constraints, triggers, and views. This more rich format of exchange reduces transfer errors. Dolt also allows you to change the data to fit your needs and still get updates from your source. Dolt will notify you if your changes [conflict](/concepts/dolt/git/conflicts) with the source.

## External APIs
### External APIs

Dolt is ideal for sharing data that does not have an API. But even for data with an API, Dolt is often more convenient. With Dolt, you get all the data and its history. With APIs you often have to assemble the data with multiple API calls. With APIs, the data can change out from under you, whereas with Dolt you can read a version of the data until you are ready to upgrade. DoltHub ships with a [SQL API](/products/dolthub/api/v1alpha1/sql) so you can choose the data sharing solution that is right for your use case.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ Dolt is a MySQL compatible database so exporting the manually created data to pr

## Dolt replaces...

## Spreadsheets
### Spreadsheets

Dolt replaces Excel or Google Sheets for manual data curation. Versioning features allow for more efficient asynchronous collaboration and human review of data changes. The DoltHub interface is still easy enough for non-technical users to contribute and review data changes.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Conflicting writes are surfaced quickly and an operator or software can take add

## Dolt replaces

## Custom syncing processes
### Custom syncing processes

Dolt replaces custom code to synchronize your client and server. This code is complicated and hard to get right. The Git [remote](/concepts/dolt/git/remotes) model of clone, fetch, push, and pull is a proven synchronization model. Dolt brings this model to the database allowing you to remove most of your synchronization code.

Expand Down
4 changes: 2 additions & 2 deletions site/dolt/src/content/introduction/use-cases/vc-your-app.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,11 @@ In the past applications that needed these features required [slowly changing di

## Dolt replaces

## Soft Deletes
### Soft Deletes

A common technique to version your database is to use [soft deletes](https://www.dolthub.com/blog/2022-11-03-soft-deletes/). When your application would make an update or a delete, you application instead makes an insert and marks the old row invalid. Dolt obviates the need for this technique. You can keep your existing database schema and Dolt ensures every write is non-destructive. Queries against soft deleted rows become Dolt history queries against [system tables](/sql-reference/version-control/dolt-system-tables).

## Slowly Changing Dimension
### Slowly Changing Dimension

A more advanced technique for versioning databases is [slowly changing dimension](https://www.dolthub.com/blog/2023-06-22-slowly-changing-dimension/). Slowly Changing Dimension is similar to soft deletes. Additional database columns are added to tables to manage versioning. Dolt is slowly changing dimension on every table by default. Queries involving the slowly changing dimension become Dolt history queries against [system tables](/sql-reference/version-control/dolt-system-tables). Moreover, complicated [merge](/concepts/dolt/git/merge) processes can happen at the database layer. Merges must handled by custom code at the application layer with slowly changing dimension.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,11 +25,11 @@ Additionally, a Dolt replica can be easily cloned (ie. copied) to a developer's

## Dolt replaces...

## Backups and Transaction Logs
### Backups and Transaction Logs

Dolt as a versioned replica becomes your first line of defense against a bad operator query, script, or deployment. Dolt is online and contains the full history of your database. In a disaster you can use diffs to find a bad query and roll it back. Then you can produce a database patch and apply it to production. You do not need to reinstall from a backup and play the transaction log back to the point of the failure, an extremely time consuming process.

## Change Data Capture
### Change Data Capture

[Change Data Capture](https://www.dolthub.com/blog/2023-03-01-change-data-capture/) is a way to add a history of data changes to an existing database. Modern change data capture tools consume replication logs to produce database changes in a consumable stream. Dolt can consume the same logs producing a simpler change data capture solution.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The following sections will introduce you to how CI works with Dolt, DoltHub and

CI configuration for a DoltHub or DoltLab database is stored in the database itself. At the time of this writing, in order to add CI configuration to a DoltHub or DoltLab database, you will need to have a local Dolt client version >= [v1.45.3](https://github.com/dolthub/dolt/releases/tag/v1.45.3) and will have to clone a copy of the the database. In order to configure CI on the database, you will use Dolt's CI CLI commands.

## Dolt CI Commands
### Dolt CI Commands

The primary interface for creating and editing CI configuration in a Dolt database is via the `dolt ci` CLI command. These commands aim to simplify CI configuration in Dolt, so that users do not need to manually interact with the underlying CI tables directly.

Expand Down Expand Up @@ -57,7 +57,7 @@ A workflow Job is made up of a series of Steps. A step, in its current form, is

For more information on Steps, please see the [workflow reference](/products/dolthub/continuous-integration/reference).

## Saved Query
### Saved Query

A [Saved Query](/sql-reference/version-control/saved-queries) is a SQL query that is stored and versioned in a Dolt database. For the purpose of DoltHub and DoltLab CI, this allows users to write a SQL query that will be executed on command at a later time, during a CI run.

Expand Down
10 changes: 5 additions & 5 deletions site/dolt/src/content/products/hosted/cloning.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ analytics query. [Clone](/cli-reference/cli#dolt-clone) makes it easy to get a l
copy of your Hosted database with one command. Do whatever potentially destructive or
performance-degrading operations on your laptop while respecting your production database.

## 1. Expose remotesapi endpoint
### 1. Expose remotesapi endpoint

In order to enable cloning from your Hosted database, first expose the [remotesapi
endpoint](/cli-reference/cli#dolt-sql-server). This will set the port for a server
Expand All @@ -48,7 +48,7 @@ deployment will also work.

![](../../.gitbook/assets/hosted-create-deployment-remotesapi.png)

## 2. Set remote password and run clone command with user flag
### 2. Set remote password and run clone command with user flag

To authenticate against it, you have to set a `DOLT_REMOTE_PASSWORD` environment variable
and pass along a `--user` flag to the `dolt clone` command. You can find these
Expand Down Expand Up @@ -92,7 +92,7 @@ cloning https://dolthub-us-housing.dbs.hosted.doltdb.com/us-housing-prices
+----------+
```

## 3. Sync your local copy with upstream changes
### 3. Sync your local copy with upstream changes

Now we can run whatever queries or schema migrations we want without affecting production.
If there are updates to the database, easily sync your local copy using [`dolt
Expand All @@ -107,7 +107,7 @@ commands to your remotesapi endpoint in the same way by passing the `--user` fla
% dolt fetch --user "[username]"
```

## 4. Sync your upstream with changes from local copy
### 4. Sync your upstream with changes from local copy

If you make any changes on your local copy, you can push them to your upstream using
[`dolt push`](/cli-reference/cli#dolt-push). Push to a feature branch rather than
Expand All @@ -127,7 +127,7 @@ You can also push straight to `main` if you'd rather skip review, subject to any
% dolt push origin --user "[username]" HEAD:main
```

## 5. Open a pull request
### 5. Open a pull request

Hosted Dolt doesn't have forks — that's a DoltHub and DoltLab feature. To contribute
a change for review, push a feature branch to the deployment (as in step 4 above) and
Expand Down
16 changes: 8 additions & 8 deletions site/dolt/src/content/products/hosted/dolthub-as-remote.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ pull, and fetch from public or private databases.

## Example

## 1. Find a database on DoltHub to clone
### 1. Find a database on DoltHub to clone

We have an example user metrics database on DoltHub named `dolthub/user_metrics`. Since we
don't want our metrics to be publicly accessible, the database is private.
Expand All @@ -49,7 +49,7 @@ We want to host this data on Hosted Dolt so that we can use [Google Looker
Studio](https://lookerstudio.google.com/) to visualize our data. Learn more about that
process [here](https://www.dolthub.com/blog/2023-02-13-dolt-looker/).

## 2. Create a new deployment on Hosted
### 2. Create a new deployment on Hosted

Next, I create a [new deployment](https://hosted.doltdb.com/create-deployment) on Hosted
and check the `Expose Dolt credentials` check box from the form.
Expand All @@ -65,14 +65,14 @@ will have access to this key.
If I accidentally expose my key or decide I want to remove it, I also have those options
there.

## 3. Add public key to DoltHub
### 3. Add public key to DoltHub

I click on "Add to DoltHub" to add my public key to DoltHub. This will open my DoltHub
[credentials settings page](https://www.dolthub.com/settings/credentials).

![](../../.gitbook/assets/dolthub-credentials-for-hosted.png)

## 4. Connect to Hosted instance and clone DoltHub database
### 4. Connect to Hosted instance and clone DoltHub database

Now that I have my credentials set up, I can connect to the Hosted instance using the
information in the Connectivity tab and clone my `dolthub/user_metrics` database.
Expand Down Expand Up @@ -128,7 +128,7 @@ Now I can do whatever I want with my metrics data, including following [these
steps](https://www.dolthub.com/blog/2023-02-13-dolt-looker/#create-a-data-source) to
connect my Hosted instance to Looker.

## 5. Make changes to database on DoltHub
### 5. Make changes to database on DoltHub

Finance reviews our metrics charts and finds a hole in the data suggesting an outage.

Expand All @@ -139,7 +139,7 @@ editor and create a pull request. They don't even need to know SQL!

![](../../.gitbook/assets/dolthub-spreadsheet-editor-user-metrics.png)

## 6. Pull new DoltHub branch to Hosted instance for testing
### 6. Pull new DoltHub branch to Hosted instance for testing

We want to review the new chart from the change before we merge it to `main`. We can pull
that branch to our Hosted instance and use it to [create a new branch in
Expand Down Expand Up @@ -167,7 +167,7 @@ mysql> select * from dolt_diff('main...outage-estimates', 'user_counts');

```

## 7. Make a change from Hosted and push it back to DoltHub
### 7. Make a change from Hosted and push it back to DoltHub

We are mostly satisfied with the new chart, but I want to make a small update to the
metrics for one of the days. I can make the change from Hosted on a branch and push it
Expand Down Expand Up @@ -207,7 +207,7 @@ I will see the new branch in my database on DoltHub.

![](../../.gitbook/assets/dolthub-outage-updates-branch.png)

## 8. Merge branches into `main` on DoltHub and pull again to Hosted
### 8. Merge branches into `main` on DoltHub and pull again to Hosted

From there, our changes are approved and merged into the `main` branch on DoltHub. One
more `dolt_pull` will update our `main` branch on Hosted.
Expand Down
18 changes: 9 additions & 9 deletions site/dolt/src/content/products/hosted/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ There are three ways to read or write from Hosted Dolt. You can:

In this blog, we will show off (1) and (2) but we're going to start with the Workbench because it is the easiest to use.

## Start the Workbench
### Start the Workbench

Click on the Workbench tab of your deployment.

Expand All @@ -67,7 +67,7 @@ The workbench has writes off by default. We will turn those on and create a data

This should feel like a standard SQL workbench like Tableplus or Datagrip but it's web-based and has some extra Dolt-specific features like a Commit Log and Pull Requests.

## Create Some Tables
### Create Some Tables

Now let's create some tables using SQL. We're going to enter the following SQL queries into the query box. You have to run them one at a time.

Expand Down Expand Up @@ -99,7 +99,7 @@ I finish up by subsequently running the last two create table queries.

![](../../.gitbook/assets/hosted-getting-started/multi-table-diff.png)

## Create a Dolt Commit
### Create a Dolt Commit

Now time to use my first Dolt feature! I'm going to create a [Dolt Commit](/concepts/dolt/git/commits). Make a Dolt commit when you want to preserve the state of the Dolt database permanently for future reference.

Expand Down Expand Up @@ -145,7 +145,7 @@ Taylor can now access the deployment page to get connectivity information. Her m

Dolt is a MySQL-compatible database. You can connect any client that can connect to MySQL to it. We're going to use the MySQl client that comes with MySQL in this section to connect to Dolt.

## Install
### Install

Head over to the [MySQL Getting Started documentation](https://dev.mysql.com/doc/mysql-getting-started/en/) and install MySQL on your machine. I used [Homebrew](https://brew.sh/) to install MySQL on my Mac.

Expand All @@ -156,7 +156,7 @@ MySQL comes with a MySQL server called `mysqld` and a MySQL client called `mysql
mysql Ver 8.0.29 for macos12.2 on x86_64 (Homebrew)
```

## Connect
### Connect

Now, to connect the mysql client to Dolt, you need the host, port, username, and password from the Connectivity tab.

Expand Down Expand Up @@ -186,7 +186,7 @@ mysql>

This MySQL client is connected to your Hosted Dolt instance. Any changes you make here will be visible to users of the workbench or any other client connected to the Hosted Dolt database.

## Create a branch
### Create a branch

So let's be safe and make our changes on a [branch](/concepts/dolt/git/branch). A branch in Dolt is a lightweight way of isolating your changes from the "main" copy of the database. Since our goal is to make a Pull Request in the SQL Workbench, making a branch is necessary as pull requests are done between two branches, in this case "main" and our new branch.

Expand Down Expand Up @@ -216,7 +216,7 @@ mysql> select active_branch();

Great, Taylor is now on a new branch and can safely make her changes.

## Insert some rows
### Insert some rows

This is easy if you know SQL. In the MySQL client on the `inserts` branch, Taylor ran the following SQL to add a few of the early employees here at DoltHub and assign them to teams.

Expand Down Expand Up @@ -273,7 +273,7 @@ mysql> call dolt_commit('-m', 'inserted early employees');

Let's head back over to the workbench to make a Pull Request.

## Make and Review a Pull Request
### Make and Review a Pull Request

The Hosted Dolt workbench supports pull requests and human review of your Hosted Dolt database. Pull requests on Hosted are always opened and merged from the workbench — there are no forks (forks are a [DoltHub](/concepts/dolthub/prs) / [DoltLab](/products/doltlab) feature) and no CLI/SQL surface for opening a PR. The branch you're proposing for review can come from anywhere (the workbench, a MySQL client, a [local clone](/products/hosted/cloning), automation), but the PR itself is created in the web UI.

Expand All @@ -299,7 +299,7 @@ Finally, she clicks "Create pull request". She now sends me this Pull Request li

![](../../.gitbook/assets/hosted-getting-started/workbench-pr.png)

## Review and Merge
### Review and Merge

On the Pull Request Page, I have access to a human readable diff of the changes under review.

Expand Down
Loading
Loading