PRDCT-486: Storage section rework — unified method + live-verify

_data/navigation.yml

@@ -659,6 +659,9 @@ items:
       - url: /storage/tables/
         title: Tables & Aliases
         items:
+          - url: /storage/tables/incremental-loading/
+            title: Primary Keys & Incremental Loading
+
           - url: /storage/tables/data-types/
             title: Data Types
 

src/content/docs/catalog/multi-project/index.md

@@ -33,7 +33,7 @@ This page explains the concept with a worked example. For a guided walkthrough o
 Let's say that you have an existing Keboola project that contains:
 
 - Oracle database data source connector with the following configurations:
-    - `ora-history` -- The source database is over 2TB; the largest table is `op_history`, which can be easily loaded [incrementally](/storage/tables/#incremental-loading) as records are only added; it is updated every 10 minutes.
+    - `ora-history` -- The source database is over 2TB; the largest table is `op_history`, which can be easily loaded [incrementally](/storage/tables/incremental-loading/) as records are only added; it is updated every 10 minutes.
     - `ora-crm` -- Tables from a CRM system. Major updates are being made to the CRM, so the structure of the tables changes often. Every two weeks, a column is renamed or a table is split into two.
     - `ora-common` -- Auxiliary tables with addresses and product names that are updated 4 times a year with data from the parent company.
     - `ora-is` -- Extracts `ORA_IS_XXX` bunch of tables which represent a port of a legacy information system, where column names had to fit into a 6 character limit.

src/content/docs/components/extractors/communication/slack/index.md

@@ -16,7 +16,7 @@ Then click **Authorize Account** to [authorize the configuration](/components/#a
 
 Select the template you wish to use:
 
-- Smart Mode -- using this mode you always get just missing data (recommended), loads data [incrementally](/storage/tables/#incremental-loading).
+- Smart Mode -- using this mode you always get just missing data (recommended), loads data [incrementally](/storage/tables/incremental-loading/).
 - Full Mode -- using this mode you always get everything.
 
 You can download:

src/content/docs/components/extractors/database/azure-storage-table/index.md

@@ -38,7 +38,7 @@ the [**Configuration Parameters**](#configuration-parameters). Then click **Save
 - **`table`**: string (required); the name of the input table in the Table storage
 - **`output`**: string (required); the name of the output CSV file
 - **`maxTries`**: integer (optional); the max number of retries if an error occurs; the default is `5`
-- **`incremental`**: boolean (optional); enables [Incremental Loading](https://help.keboola.com/storage/tables/#incremental-loading); the default is `false`
+- **`incremental`**: boolean (optional); enables [Incremental Loading](https://help.keboola.com/storage/tables/incremental-loading/); the default is `false`
 - **`incrementalFetchingKey`**: string (optional); the name of the key for [incremental fetching](https://help.keboola.com/components/extractors/database/#incremental-fetching)
 - **`mode`**: enum (optional)
   - `mapping` (default) 

src/content/docs/components/extractors/database/cosmosdb/index.md

@@ -33,7 +33,7 @@ the [**Configuration Parameters**](#configuration-parameters). Then click **Save
 
 - **`containerId`**: string (required); the ID of the Cosmos DB container
 - **`output`**: string (required); the name of the output table in your bucket
-- **`incremental`**: boolean (optional); enables [incremental loading](/storage/tables/#incremental-loading); the default is `false`
+- **`incremental`**: boolean (optional); enables [incremental loading](/storage/tables/incremental-loading/); the default is `false`
 - **`incrementalFetchingKey`**: string (optional); the name of the key for [incremental fetching](/components/extractors/database/#incremental-fetching), e.g., `c.id`
 - **`mode`**: enum (optional)
     - `mapping` (default) -- items are exported using specified `mapping`

src/content/docs/components/extractors/database/index.md

@@ -122,11 +122,11 @@ The rows are fetched from the source table, including the last fetched value. Th
 ideal to set the ordering column as a primary key so you don't receive duplicated rows in
 the Storage table. You can clear the stored value if you need to fetch the entire table.
 
-This incremental fetching feature is related to [**incremental loading**](/storage/tables/#incremental-loading).
+This incremental fetching feature is related to [**incremental loading**](/storage/tables/incremental-loading/).
 While not required, it is recommended to turn on incremental loading when fetching data incrementally; otherwise,
 the table in Storage will contain only newly added rows. This may sound like a good idea when you want to
 process only the newly added rows. In that case, however, you should do so using
-[**incremental processing**](/storage/tables/#incremental-processing). The advantage of using incremental processing
+[**incremental processing**](/storage/tables/incremental-loading/#incremental-processing). The advantage of using incremental processing
 over having only newly added rows in a Storage table is that the table contains all loaded data, and it is not necessary
 to synchronize extraction and processing.
 

src/content/docs/components/extractors/database/sqldb/index.md

@@ -50,10 +50,10 @@ If you want to modify the table extraction setup, click on the corresponding row
 ![Screenshot - Table Detail](/components/extractors/database/sqldb/sqldb-5.png)
 
 You can modify the source table, limit the extraction to specific columns, or change the destination table name in
-[Storage](/storage/). The table detail also allows you to define a [**primary key**](/storage/tables/#primary-keys)
-and [**incremental loading**](/storage/tables/#incremental-loading).
-We highly recommend you define a **primary key** where possible. [Primary keys](/storage/tables/#primary-keys) substantially
-speed up the data loads and further table processing. Also, [incremental loading](/storage/tables/#incremental-loading) should be used when possible, which considerably speeds up the data loads.
+[Storage](/storage/). The table detail also allows you to define a [**primary key**](/storage/tables/incremental-loading/#primary-keys)
+and [**incremental loading**](/storage/tables/incremental-loading/).
+We highly recommend you define a **primary key** where possible. [Primary keys](/storage/tables/incremental-loading/#primary-keys) substantially
+speed up the data loads and further table processing. Also, [incremental loading](/storage/tables/incremental-loading/) should be used when possible, which considerably speeds up the data loads.
 Both options require knowledge of the source table, so don't turn them on blindly.
 
 ### Advanced Mode

src/content/docs/components/extractors/generic-extractor/incremental/index.md

@@ -12,7 +12,7 @@ Extracting data incrementally is universally beneficial — it **speeds up the e
 
 ## Options
 After you have incrementally extracted data from an API, the data must be 
-[incrementally loaded](/storage/tables/#incremental-loading)
+[incrementally loaded](/storage/tables/incremental-loading/)
 into Storage. To do that, simply set `"incrementalOutput": true` in the `config` section. 
 
 There are, however, a number of implications in the incremental loads. It essentially boils downs to the following use cases, 

src/content/docs/components/extractors/marketing-sales/bigcommerce/index.md

@@ -44,7 +44,7 @@ If you are extracting *time-bound* data (i.e., anything **except** `Brands`), yo
 - **Date From**: only data modified after this date are downloaded. Use the `YYYY-MM-DD` format or a human readable description, e.g., `5 days ago`, `1 month ago`, `yesterday`, etc. You can also set this as `last run`, which will fetch data from the last run of the component; if no previous successful run exists, all data up to specified **Date To** will be downloaded.
 - **Date To**: only data modified before this date are downloaded. Use the `YYYY-MM-DD` format or a a human readable description, e.g., `5 days ago`, `1 week ago`, `today`, etc.
 
-Finally, in the **Destination** part of the row configuration, you must choose the **Load Type**; i. e., whether you want to use [incremental loading](/storage/tables/#incremental-loading) (by selecting `Incremental Load`) or full loading (by selecting `Full Load`).
+Finally, in the **Destination** part of the row configuration, you must choose the **Load Type**; i. e., whether you want to use [incremental loading](/storage/tables/incremental-loading/) (by selecting `Incremental Load`) or full loading (by selecting `Full Load`).
 
 ![Row configuration entry](/components/extractors/marketing-sales/bigcommerce/row_config.png)
 

src/content/docs/components/extractors/other/aws-cur-reports/index.md

@@ -55,7 +55,7 @@ and copy the path of the report folder.
 
 ![AWS configuration](/components/extractors/other/aws-cur-reports/report_config.png)
 
-If the [**Incremental Load**](/storage/tables/#incremental-loading) is set to true, the new data will be appended to the old ones. 
+If the [**Incremental Load**](/storage/tables/incremental-loading/) is set to true, the new data will be appended to the old ones. 
 This way you can import new data, e.g., from today, without deleting the data imported before.
 
 ## Output Table

src/content/docs/components/extractors/other/azure-cost/index.md

@@ -30,7 +30,7 @@ In the [Configuration Row](/components/#configuration-rows) fill in
 
 ![Screenshot - Configuration Row](/components/extractors/other/azure-cost/row.png)
 
-If the [**Incremental Load**](/storage/tables/#incremental-loading) is set to true, the new data will be appended to the old ones. 
+If the [**Incremental Load**](/storage/tables/incremental-loading/) is set to true, the new data will be appended to the old ones. 
 This way you can import new data, e.g., from today, without deleting the data imported before.
 
 ## Output Table

src/content/docs/components/extractors/other/github/index.md

@@ -15,7 +15,7 @@ to Keboola.
 Then click **Authorize Account** to [authorize the configuration](/components/#authorization), and
 select the template you wish to use. There are two configuration templates available:
 
-- `Smart Mode` -- always gets missing data only, loads data [incrementally](/storage/tables/#incremental-loading).
+- `Smart Mode` -- always gets missing data only, loads data [incrementally](/storage/tables/incremental-loading/).
 - `Full Mode` -- always gets everything.
 
 ![Screenshot - GitHub configuration](/components/extractors/other/github/github-1.png)

src/content/docs/components/extractors/storage/aws-s3/index.md

@@ -143,7 +143,7 @@ The **additional source settings** section allows you to set up the following:
 
 - The initial value in **Storage Table Name** is derived from the configuration table name. You can change it at any time; however,
 the [Storage bucket](/storage/buckets/) where the table will be saved cannot be changed.
-- **Incremental Load** will turn on [incremental loading to Storage](/storage/tables/#incremental-loading). The result of the
+- **Incremental Load** will turn on [incremental loading to Storage](/storage/tables/incremental-loading/). The result of the
 incremental load depends on other settings (mainly **Primary Key**).
 - **Primary Key** can be used to specify the primary key in Storage; it can be used with **Incremental Load**
 and **New Files Only** to create a configuration that incrementally loads all new files into a table in Storage.

src/content/docs/components/extractors/storage/azure-datalake-gen2/index.md

@@ -70,7 +70,7 @@ The **additional source settings** section allows you to set up the following:
 
 - The initial value in **Storage Table Name** is derived from the configuration table name. You can change it at any time; however,
 the [Storage bucket](/storage/buckets/) where the table will be saved cannot be changed.
-- **Incremental Load** will turn on [incremental loading to Storage](/storage/tables/#incremental-loading). The result of the
+- **Incremental Load** will turn on [incremental loading to Storage](/storage/tables/incremental-loading/). The result of the
 incremental load depends on other settings (mainly **Primary Key**).
 - **Primary Key** can be used to specify the primary key in Storage; it can be used with **Incremental Load**
 and **New Files Only** to create a configuration that incrementally loads all new files into a table in Storage.

src/content/docs/components/extractors/storage/ftp/index.md

@@ -49,7 +49,7 @@ Now determine how to save the data in Storage.
 
 - The initial value in **Table Name** is derived from the configuration table name. You can change it at any time; however,
 the [Storage bucket](/storage/buckets/) where the table will be saved cannot be changed.
-- **Incremental Load** will turn on [incremental loading to Storage](/storage/tables/#incremental-loading). The result of the
+- **Incremental Load** will turn on [incremental loading to Storage](/storage/tables/incremental-loading/). The result of the
 incremental load depends on other settings (mainly **Primary Key**).
 - **Delimiter** and **Enclosure** specify the CSV settings.
 

src/content/docs/components/extractors/storage/http/index.md

@@ -45,7 +45,7 @@ want to load one of our tutorial tables, enter its path, e.g., `/tutorial/opport
 
 - The initial value in **Table name** is derived from the row name. You can change it at any time; however,
 the [Storage bucket](/storage/buckets/) where the table will be saved to cannot be changed.
-- **Incremental load** will turn on [incremental loading to Storage](/storage/tables/#incremental-loading). The result of the
+- **Incremental load** will turn on [incremental loading to Storage](/storage/tables/incremental-loading/). The result of the
 incremental load depends on other settings (mainly **Primary Key**).
 - **Delimiter** and **Enclosure** specify the CSV settings.
 

src/content/docs/components/extractors/storage/storage-api/index.md

@@ -56,7 +56,7 @@ As the token has access to a single bucket only, you do not need to specify the
 
 ### Save Settings
 
-- **Incremental** -- enables [incremental loading](/storage/tables/#incremental-loading) in the current project. If the **Primary Key** is not set, the data is appended. 
+- **Incremental** -- enables [incremental loading](/storage/tables/incremental-loading/) in the current project. If the **Primary Key** is not set, the data is appended. 
 Otherwise the rows with an existing primary key are updated.
 - **Primary Key** -- sets the primary key of the table in the current project. The primary key does not have to be the same
 as in the *source project*.

src/content/docs/components/writers/bi-tools/gooddata/index.md

@@ -101,7 +101,7 @@ Incremental load will keep the existing data in the GoodData project.
 It can be much faster, but the source data needs to be correctly prepared. 
 The incremental load relies on the following two features:
 
-- [Incremental processing](/storage/tables/#incremental-processing) in Storage, and
+- [Incremental processing](/storage/tables/incremental-loading/#incremental-processing) in Storage, and
 - Identity in GoodData; this can be either a `CONNECTION_POINT` column (which acts as a database primary key), or a [Fact Grain](https://help.gooddata.com/doc/enterprise/en/data-integration/data-modeling-in-gooddata/logical-data-model-components-in-gooddata/facts-in-logical-data-models#FactsinLogicalDataModels-FactDatasets) (which acts as a compound unique key). Fact Grain can be used when there is no single identifying column (i.e. there is no `CONNECTION_POINT`) and there is a combination of columns which can be used to identify rows. If there is no such combination, then incremental loading cannot be used.
 
 From the table configuration page, you can also **Run** a load of a single table. However, if the table has relations to other

src/content/docs/components/writers/bi-tools/tableau/index.md

@@ -43,7 +43,7 @@ be part of the TDE file. When configuring the data types, use the *Preview* icon
 
 As optional last steps in table configuration, you can configure the name of the TDE file (useful for uploading to Dropbox or Google Drive)
 and *Table data filter*. The table data filter allows you to set a simple filter for one column or
-take advantage of [Incremental processing](/storage/tables/#incremental-processing) by writing only
+take advantage of [Incremental processing](/storage/tables/incremental-loading/#incremental-processing) by writing only
 recently modified data.
 
 ![Screenshot - Table Configuration Additional Settings](/components/writers/bi-tools/tableau/tableau-4.png)

src/content/docs/components/writers/bi-tools/thoughtspot/index.md

@@ -58,4 +58,4 @@ In the **Full Load** mode, the table is completely overwritten including the tab
 using the [`DROP`](https://docs.thoughtspot.com/software/latest/tql-cli-commands) command and it is recreated.
 
 Additionally, you can specify a **primary key** for the table, a simple column **data filter**, and a filter for
-[incremental processing](/storage/tables/#incremental-processing).
+[incremental processing](/storage/tables/incremental-loading/#incremental-processing).

src/content/docs/components/writers/database/mssql/index.md

@@ -69,7 +69,7 @@ This means that if the database is used by other applications which acquire tabl
 freeze waiting for the locks to be released.
 
 Additionally, you can specify a **Primary key** of the table, a simple column **Data filter**, and a filter for
-[incremental processing](/storage/tables/#incremental-processing).
+[incremental processing](/storage/tables/incremental-loading/#incremental-processing).
 
 ## Binary types
 

src/content/docs/components/writers/database/mysql/index.md

@@ -65,4 +65,4 @@ This means that if the database is used by other applications which acquire tabl
 freeze waiting for the locks to be released.
 
 Additionally, you can specify a **primary key** of the table, a simple column **data filter**, and a filter for
-[incremental processing](/storage/tables/#incremental-processing).
+[incremental processing](/storage/tables/incremental-loading/#incremental-processing).

src/content/docs/components/writers/database/oracle/index.md

@@ -67,4 +67,4 @@ fail with the following message:
     Query failed: 'ORA-00955: name is already used by an existing object
 
 Additionally, you can specify a **Primary key** of the table, a simple column **Data filter**, and a filter for
-[incremental processing](/storage/tables/#incremental-processing).
+[incremental processing](/storage/tables/incremental-loading/#incremental-processing).

src/content/docs/components/writers/database/postgresql/index.md

@@ -68,4 +68,4 @@ freeze waiting for the locks to be released. This will be recorded in the connec
     Table "account" is locked by 1 transactions, waiting for them to finish
 
 Additionally, you can specify a **Primary key** of the table, a simple column **Data filter**, and a filter for
-[incremental processing](/storage/tables/#incremental-processing).
+[incremental processing](/storage/tables/incremental-loading/#incremental-processing).

src/content/docs/components/writers/database/redshift/index.md

@@ -94,7 +94,7 @@ freeze waiting for the locks to be released. See the [Redshift docs](https://doc
 for more details.
 
 Additionally, you can specify a **Primary key** of the table, a simple column **Data filter**, and a filter for
-[incremental processing](/storage/tables/#incremental-processing).
+[incremental processing](/storage/tables/incremental-loading/#incremental-processing).
 
 ## Using Keboola Provisioned Database
 The connector offers the option to create a [Keboola Provisioned database](#keboola-redshift-database) for you. You can

src/content/docs/components/writers/database/snowflake/index.md

@@ -95,7 +95,7 @@ using the [`ALTER SWAP`](https://docs.snowflake.com/en/sql-reference/sql/alter-t
 the shortest unavailability of the target table. However, this operation still drops the table.
 
 Additionally, you can specify a **Primary key** of the table, a simple column **Data filter**, and a **Data changed in last** filter for
-[incremental processing](/storage/tables/#incremental-processing).
+[incremental processing](/storage/tables/incremental-loading/#incremental-processing).
 
 **Data changed in last** filter is not available when using **Automatic incremental load**, as the component will use the last run date to determine the data to be loaded.
 

src/content/docs/components/writers/database/synapse/index.md

@@ -90,4 +90,4 @@ This means that if the database is used by other applications which acquire tabl
 freeze waiting for the locks to be released.
 
 Additionally, you can specify a **Primary key** of the table, a simple column **Data filter**, and a filter for
-[incremental processing](/storage/tables/#incremental-processing).
+[incremental processing](/storage/tables/incremental-loading/#incremental-processing).

src/content/docs/components/writers/storage/google-drive/index.md

@@ -19,7 +19,7 @@ Then click the **New Table** button to add a new table:
 
 ![Screenshot - Add Table Step 1](/components/writers/storage/google-drive/google-drive-1.png)
 
-Select a table from Storage. You may also specify additional filters as well as [incremental processing](/storage/tables/#incremental-processing).
+Select a table from Storage. You may also specify additional filters as well as [incremental processing](/storage/tables/incremental-loading/#incremental-processing).
 All options may be modified later. Click **Next** to select how to load the table to Google Drive:
 
 ![Screenshot - Add Table Step 2](/components/writers/storage/google-drive/google-drive-2.png)