From b367515538c30b25f07b4f5912d86af0e053a4eb Mon Sep 17 00:00:00 2001 From: Nikita Date: Tue, 4 Aug 2026 01:45:13 +0200 Subject: [PATCH 1/2] fix(links): repoint the 73 stale anchors left by the Jekyll/dev era MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Anchor targets were never validated by CI — audit-phase2 checks link *paths* but not the `#fragment` — so every heading rename since the Jekyll days left a link pointing at nothing. A checker over the built site found 73; this fixes all of them. Grouped by cause: - **Heading renamed, article dropped** (~30): `#inviting-a-user` → `#inviting-user`, `#running-a-component` → `#running-component`, `#preparing-the-data-folder` → `#preparing-data-folder`, and so on. - **Section moved to another page** (11): the manifest-files hub became one page per manifest kind (`#dataouttables-manifests` → `out-tables-manifests/`); `/storage/file-uploads/#limits` → `/storage/files/#limits`; `/transformations/{python,r}/#development-tutorial` → the `-plain` pages — those three paths are redirect stubs, so the anchor never survived the hop. - **Section is gone** (10): `#new-transformations` (5 links) and `/management/jobs/#jobs` now link the page itself; `#main-header` was a Jekyll top-of-page artifact. - **Starlight has no `{#id}` heading syntax** (1 page): telemetry-dashboards wrote `### Organization Usage {#ac-organization-usage}` to disambiguate two same-named sections. Starlight rendered the braces *literally* on the page and slugged them into the id. Renamed to "Organization Usage (Activity Center)", which keeps the sections distinct and gives the link a real target. - **Nothing to point at** (3): the Phases/Dependencies cells in the transformations comparison table and "output buffering" in the implementation notes are now plain text — the sections they referenced don't exist anywhere. Flagged in the PR for someone who knows the original intent. - **A link that was never a link**: `/overview/` shipped `our Flows [/tutorial/automate/#main-header], obtaining` as literal text. Verified with `astro build`: **73 → 0** broken anchors out of 8476 checked across 500 pages. audit-phase2 unchanged on every category (45 broken internal links / 0 missing images / 3 multi-h1 / 0 unclosed fences / 0 malformed tables); its old-docs-smell count goes 99 → 101 for the two `developers.keboola.com/#development-project` links, the one target with no help equivalent yet — deliberate, and they match the sibling link already in `/tutorial/`. Co-Authored-By: Claude Opus 5 --- src/content/docs/ai/index.md | 2 +- src/content/docs/catalog/index.md | 4 ++-- .../components/extractors/database/bigquery/index.md | 2 +- .../docs/components/extractors/database/index.md | 2 +- .../components/extractors/database/ms-sql/index.md | 2 +- .../docs/components/extractors/database/mysql/index.md | 8 ++++---- .../components/extractors/database/oracle/index.md | 2 +- .../components/extractors/database/postgresql/index.md | 6 +++--- .../docs/components/running-jobs-in-parallel/index.md | 2 +- .../docs/components/writers/bi-tools/tableau/index.md | 2 +- .../docs/extend/common-interface/config-file/index.md | 6 +++--- .../docs/extend/common-interface/folders/index.md | 10 +++++----- .../docs/extend/component/code-patterns/index.md | 4 ++-- src/content/docs/extend/component/deployment/index.md | 10 +++++----- .../docs/extend/component/implementation/index.md | 10 +++++----- .../extend/component/implementation/python/index.md | 6 +++--- .../docs/extend/component/implementation/r/index.md | 2 +- src/content/docs/extend/component/index.md | 2 +- src/content/docs/extend/component/processors/index.md | 4 ++-- src/content/docs/extend/component/running/index.md | 4 ++-- src/content/docs/extend/component/tutorial/index.md | 4 ++-- .../extend/component/tutorial/output-mapping/index.md | 2 +- src/content/docs/extend/job-queue/index.md | 8 ++++---- src/content/docs/management/account/index.md | 6 +++--- src/content/docs/management/index.md | 2 +- src/content/docs/management/jobs/index.md | 2 +- src/content/docs/management/organization/index.md | 2 +- src/content/docs/management/project/limits/index.md | 4 ++-- src/content/docs/management/project/tokens/index.md | 6 +++--- src/content/docs/management/support/index.md | 4 ++-- .../management/telemetry/telemetry-dashboards/index.md | 4 ++-- src/content/docs/overview/index.md | 4 ++-- .../docs/transformations/code-patterns/index.md | 2 +- src/content/docs/transformations/index.md | 4 ++-- src/content/docs/transformations/mappings/index.md | 4 ++-- src/content/docs/tutorial/ad-hoc/index.md | 2 +- src/content/docs/tutorial/branches/project-diff.md | 2 +- src/content/docs/tutorial/manipulate/index.md | 4 ++-- src/content/docs/workspace/index.md | 2 +- .../snowflake-workspaces-access-changes/index.md | 2 +- src/content/docs/workspace/table-export.md | 2 +- 41 files changed, 81 insertions(+), 81 deletions(-) diff --git a/src/content/docs/ai/index.md b/src/content/docs/ai/index.md index 62207de86..8a7dc8954 100644 --- a/src/content/docs/ai/index.md +++ b/src/content/docs/ai/index.md @@ -39,7 +39,7 @@ Allows users to get AI suggestions when searching for a component. To activate t ### AI Rules The AI Rules section allows users to define specific instructions for AI functionality within Keboola platform. -[Learn more about AI Rules →](/management/project/ai-rules/#main-header) +[Learn more about AI Rules →](/management/project/ai-rules/) ### AI-Generated Descriptions Automatically generates a description using AI. diff --git a/src/content/docs/catalog/index.md b/src/content/docs/catalog/index.md index 06fc73188..7596cd5e4 100644 --- a/src/content/docs/catalog/index.md +++ b/src/content/docs/catalog/index.md @@ -38,7 +38,7 @@ Sharing data via the data catalog is useful in numerous ways -- for example: instead of distributing updates throughout your company, you can create a project with the product table shared to the rest of the company. Regardless of how and how often the table is updated, this ensures that everyone always looks at the same data. -- You might want to use [**Multi-project architecture**](#multi-project-architecture). +- You might want to use [**Multi-project architecture**](/catalog/multi-project/). The following terminology is used: @@ -118,7 +118,7 @@ Select the tables and/or buckets you want to share: ![Screenshot -- Select Tables](/catalog/catalog-7.png) -Enter the bucket name and a [sharing type](#sharing-type). Assign a **Datashare Owner**, and optionally customize the bucket color and enter a description: +Enter the bucket name and a [sharing type](#sharing-types). Assign a **Datashare Owner**, and optionally customize the bucket color and enter a description: ![Screenshot -- Select Tables](/catalog/catalog-8.png) diff --git a/src/content/docs/components/extractors/database/bigquery/index.md b/src/content/docs/components/extractors/database/bigquery/index.md index 6cf2eb6e9..7da0d9985 100644 --- a/src/content/docs/components/extractors/database/bigquery/index.md +++ b/src/content/docs/components/extractors/database/bigquery/index.md @@ -15,7 +15,7 @@ Running the connector creates a background job that - exports the results from Google Cloud Storage and stores them in specified tables in Keboola Storage. - removes the results from Google Cloud Storage. -***Note:** Using the Google BigQuery connector is also described in our [Getting Started Tutorial](/tutorial/ad-hoc/#using-bigquery-extractor).* +***Note:** Using the Google BigQuery connector is also described in our [Getting Started Tutorial](/tutorial/ad-hoc/#using-bigquery-connector).* ## Initial Setup diff --git a/src/content/docs/components/extractors/database/index.md b/src/content/docs/components/extractors/database/index.md index 7c873c636..0f5d8c11b 100644 --- a/src/content/docs/components/extractors/database/index.md +++ b/src/content/docs/components/extractors/database/index.md @@ -32,7 +32,7 @@ Several variants of connectors may exist for each database type, depending on th These connectors work on a relational level, performing queries against the source database to synchronize data. This straightforward approach suits most use cases and supports Timestamp-based CDC replication. -All are [configured](/components/extractors/database/sqldb/#create-new-configuration) similarly and offer an [advanced mode](/components/extractors/database/sqldb/). +All are [configured](/components/extractors/database/sqldb/#initial-setup) similarly and offer an [advanced mode](/components/extractors/database/sqldb/). Their basic configuration is also part of the [Tutorial - Loading Data from Database](/tutorial/load/database/). diff --git a/src/content/docs/components/extractors/database/ms-sql/index.md b/src/content/docs/components/extractors/database/ms-sql/index.md index 993ac4f87..edecfaeef 100644 --- a/src/content/docs/components/extractors/database/ms-sql/index.md +++ b/src/content/docs/components/extractors/database/ms-sql/index.md @@ -14,7 +14,7 @@ This connector supports the most recent versions of both SQL Server and Azure SQ This [standard SQL database connector](/components/extractors/database/sqldb) performs queries against the source database to synchronize data. It offers a straightforward approach suitable for most use cases, enabling [time-stamp based](/components/extractors/database/#incremental-fetching) CDC replication. -All SQL database connectors are [configured](/components/extractors/database/sqldb/#create-new-configuration) similarly and offer an [advanced mode](/components/extractors/database/sqldb/). +All SQL database connectors are [configured](/components/extractors/database/sqldb/#initial-setup) similarly and offer an [advanced mode](/components/extractors/database/sqldb/). For guidance on basic configuration, please refer to our tutorial: [Loading Data with Database data source connector](/tutorial/load/database/). diff --git a/src/content/docs/components/extractors/database/mysql/index.md b/src/content/docs/components/extractors/database/mysql/index.md index 353a0cf94..a79fc5c55 100644 --- a/src/content/docs/components/extractors/database/mysql/index.md +++ b/src/content/docs/components/extractors/database/mysql/index.md @@ -9,15 +9,15 @@ slug: 'components/extractors/database/mysql' Our connectors support the latest versions of MySQL and AWS Aurora. You can choose among different strategies to synchronize your data: -- [Query-based connector](/components/extractors/database/sqldb/#create-new-configuration) -- [Log-based CDC](/components/extractors/database/mysql#log-based-binlog-cdc) +- [Query-based connector](/components/extractors/database/sqldb/#initial-setup) +- [Log-based CDC](/components/extractors/database/mysql/#mysql-log-based-cdc) ## Query-Based Connector This [standard connector](https://components.keboola.com/components/keboola.ex-db-mysql) performs queries against the source database to sync data. It is a straightforward approach suitable for most use cases, allowing for [time-stamp based](/components/extractors/database/#incremental-fetching) CDC replication. -All connectors are [configured](/components/extractors/database/sqldb/#create-new-configuration) similarly and +All connectors are [configured](/components/extractors/database/sqldb/#initial-setup) similarly and offer an [advanced mode](/components/extractors/database/sqldb/). Basic configuration is covered in the [Tutorial - Loading Data from Database](/tutorial/load/database/). @@ -336,7 +336,7 @@ Descriptions of MySQL binlog configuration properties: | `log_bin` | The value of `log_bin` is the base name of the sequence of binlog files. | | `binlog_format` | The `binlog-format` must be set to `ROW` or `row`. | | `binlog_row_image` | The `binlog_row_image` must be set to `FULL` or `full`. | -| `binlog_expire_logs_seconds` | The `binlog_expire_logs_seconds` corresponds to the deprecated system variable `expire_logs_days`. This is the number of seconds for automatic binlog file removal. The default is `2592000`, which equals 30 days. Set the value to match the needs of your environment. See [MySQL purges binlog files](#mysql-purges-binlog-files-used-by-debezium). | +| `binlog_expire_logs_seconds` | The `binlog_expire_logs_seconds` corresponds to the deprecated system variable `expire_logs_days`. This is the number of seconds for automatic binlog file removal. The default is `2592000`, which equals 30 days. Set the value to match the needs of your environment. See [MySQL purges binlog files](#functionality). | #### Enabling GTIDs diff --git a/src/content/docs/components/extractors/database/oracle/index.md b/src/content/docs/components/extractors/database/oracle/index.md index 1af49a5ed..ebf9773f7 100644 --- a/src/content/docs/components/extractors/database/oracle/index.md +++ b/src/content/docs/components/extractors/database/oracle/index.md @@ -10,7 +10,7 @@ slug: 'components/extractors/database/oracle' This is a [standard connector](https://components.keboola.com/components/keboola.ex-db-mysql) that performs queries against the source database to sync data. It is the simplest approach suitable for most use cases and allows for [time-stamp based](/components/extractors/database/#incremental-fetching) CDC replication. -They are all [configured](/components/extractors/database/sqldb/#create-new-configuration) in the same way and +They are all [configured](/components/extractors/database/sqldb/#initial-setup) in the same way and have an [advanced mode](/components/extractors/database/sqldb/). Their basic configuration is also part of the [Tutorial - Loading Data with Database Extractor](/tutorial/load/database/). diff --git a/src/content/docs/components/extractors/database/postgresql/index.md b/src/content/docs/components/extractors/database/postgresql/index.md index 9ccd43c39..4e0a5a8f1 100644 --- a/src/content/docs/components/extractors/database/postgresql/index.md +++ b/src/content/docs/components/extractors/database/postgresql/index.md @@ -9,8 +9,8 @@ slug: 'components/extractors/database/postgresql' Our connectors support the most recent versions of PostgreSQL. You may choose different strategies to sync your data: -- [Query-based connector](/components/extractors/database/sqldb/#create-new-configuration) -- [Log-based CDC](/components/extractors/database/postgresql/#log-based-cdc) +- [Query-based connector](/components/extractors/database/sqldb/#initial-setup) +- [Log-based CDC](/components/extractors/database/postgresql/#postgresql-log-based-cdc) ## Query-Based Connector @@ -18,7 +18,7 @@ Our connectors support the most recent versions of PostgreSQL. You may choose di This is a [standard connector](https://components.keboola.com/components/keboola.ex-db-mysql) that performs queries against the source database to sync data. It is the simplest approach suitable for most use cases and allows for [time-stamp based](/components/extractors/database/#incremental-fetching) CDC replication. -They are all [configured](/components/extractors/database/sqldb/#create-new-configuration) in the same way and +They are all [configured](/components/extractors/database/sqldb/#initial-setup) in the same way and have an [advanced mode](/components/extractors/database/sqldb/). Their basic configuration is also part of the [Tutorial - Loading Data with Database Extractor](/tutorial/load/database/). diff --git a/src/content/docs/components/running-jobs-in-parallel/index.md b/src/content/docs/components/running-jobs-in-parallel/index.md index 1bb970231..5899dcf3b 100644 --- a/src/content/docs/components/running-jobs-in-parallel/index.md +++ b/src/content/docs/components/running-jobs-in-parallel/index.md @@ -42,7 +42,7 @@ Every job passes through predictable states: **waiting** → **processing** → **success** / **error** **How billing relates to job state:** -- Jobs in the **waiting** state are not billed at the job level. A job only consumes [credits](/management/project/limits/#project-power) once it starts **processing**. +- Jobs in the **waiting** state are not billed at the job level. A job only consumes [credits](/management/project/limits/#project-power--time-credits) once it starts **processing**. - Jobs in the **processing** state are billed based on compute resources consumed. **Important — container runtime billing:** Some components run inside a container that orchestrates multiple child jobs. In these cases, the parent container may continue running and accumulating runtime costs even while individual child jobs are in the waiting state. Setting very high parallelism in a container-based component does not pause the container while jobs queue — the container remains active throughout. diff --git a/src/content/docs/components/writers/bi-tools/tableau/index.md b/src/content/docs/components/writers/bi-tools/tableau/index.md index 331b95e56..1f9b2a527 100644 --- a/src/content/docs/components/writers/bi-tools/tableau/index.md +++ b/src/content/docs/components/writers/bi-tools/tableau/index.md @@ -14,7 +14,7 @@ files and optionally uploads them to a destination (Tableau Server, to be used together with [Tableau Desktop](https://www.tableau.com/products/desktop) or [Tableau Server](https://www.tableau.com/products/server). An alternative approach is to send data using the **[Snowflake data destination connector](/components/writers/database/snowflake/)** through a -[Keboola-provisioned database](/components/writers/database/snowflake/#using-keboola-provisioned-database). That is more suitable to be used +[Keboola-provisioned database](/components/writers/database/snowflake/#using-keboola-snowflake-database). That is more suitable to be used together with [Tableau Online](https://www.tableau.com/products/cloud-bi) and also for larger data sets. Both approaches are interchangeable though. diff --git a/src/content/docs/extend/common-interface/config-file/index.md b/src/content/docs/extend/common-interface/config-file/index.md index ad50d2227..a24daf4e2 100644 --- a/src/content/docs/extend/common-interface/config-file/index.md +++ b/src/content/docs/extend/common-interface/config-file/index.md @@ -26,7 +26,7 @@ way you wish. Your component should validate the contents of this section. For p data, use [encryption](/overview/encryption/). This section is not available in Transformations. - `image_parameters`: See [below](#image-parameters). - `authorization`: Contains Oauth2 [authorization contents](/extend/common-interface/oauth/) or -[Workspace credentials](/extend/common-interface/folders/#exchanging-data-via-workspace) . +[Workspace credentials](/extend/common-interface/folders/#exchanging-data-via-database-workspace) . - `action`: Name of the [action](/extend/common-interface/actions/) to execute; defaults to `run`. All actions except `run` have a strict execution time limit of 30 seconds. See [actions](/extend/common-interface/actions/) for more details. @@ -213,7 +213,7 @@ write the usage file regularly** during the component run, not only at the end. validated and a wrong format will cause a component failure.* ## Examples -To create an example configuration, use the [Run Job API call in debug mode](/extend/component/running/#preparing-the-data-folder). You will get a +To create an example configuration, use the [Run Job API call in debug mode](/extend/component/running/#preparing-data-folder). You will get a `stage_0.zip` archive in your **Storage** > **File Uploads**, which will contain the `config.json` file. You can also use these configuration structure to create an API request for actually [running a component](https://api.keboola.com/?service=job-queue#post-/jobs). @@ -387,7 +387,7 @@ Download 2 days of data from the `in.c-storage.StoredData` table to `/data/table ``` #### Input mapping — column types -This is applicable only to [workspace mapping](/extend/common-interface/folders/#exchanging-data-via-workspace), for CSV files this setting has no effect. The `column_types` setting maps to [Storage API load options](https://api.keboola.com/?service=storage#post-/v2/storage/branch/-branchId-/workspaces/-workspaceId-/load). It also acts the same way as `columns` setting allowing you to limit the table columns. +This is applicable only to [workspace mapping](/extend/common-interface/folders/#exchanging-data-via-database-workspace), for CSV files this setting has no effect. The `column_types` setting maps to [Storage API load options](https://api.keboola.com/?service=storage#post-/v2/storage/branch/-branchId-/workspaces/-workspaceId-/load). It also acts the same way as `columns` setting allowing you to limit the table columns. If both `column_types` and `columns` setting are used, then the listed columns must match. If you omit `columns` and use only `column_types` (recommended) then `columns` will be propagated automatically from `column_types`. ```json diff --git a/src/content/docs/extend/common-interface/folders/index.md b/src/content/docs/extend/common-interface/folders/index.md index ef398edd1..6e4b82278 100644 --- a/src/content/docs/extend/common-interface/folders/index.md +++ b/src/content/docs/extend/common-interface/folders/index.md @@ -20,7 +20,7 @@ depends fully on your component code (or Dockerfile). If you want to use a diffe **use the [`KBC_DATADIR` environment variable](/extend/common-interface/environment/#environment-variables)**. In production, this variable will always be set to `/data/`. During development, you can set it to your liking. -To create a data folder sample, use the [Debug mode](/extend/component/running/#preparing-the-data-folder) on the +To create a data folder sample, use the [Debug mode](/extend/component/running/#preparing-data-folder) on the [Create Job API](https://api.keboola.com/?service=job-queue#post-/jobs). All the resources you need in your component will be provided in a ZIP archive. @@ -114,7 +114,7 @@ The following is an example of specifying columns in the manifest file `/data/ou } All files from the folder are uploaded irrespective of their name or extension. They are uploaded -to Storage in parallel and in an undefined order. Use sliced tables in case you want to upload tables [larger than 5GB](/storage/file-uploads/#limits). The slices may be compressed by gzip. +to Storage in parallel and in an undefined order. Use sliced tables in case you want to upload tables [larger than 5GB](/storage/files/#limits). The slices may be compressed by gzip. A rule of thumb is that slices are [best around 10-100 MB](https://docs.snowflake.net/manuals/user-guide/data-load-considerations-prepare.html#splitting-large-data-files-before-loading) in size **compressed**. ### Folder /data/in/files/ @@ -135,7 +135,7 @@ In this case, the data folders contain only [manifest files](/extend/common-inte not the actual data. This mode of operation can be enabled by setting the **Staging storage input** option to **AWS S3** in [component settings](https://components.keboola.com/). If this option is enabled, all the data folders will contain only manifest files, extended with an additional -[`s3` section](/extend/common-interface/manifest-files/#s3-staging). +[`s3` section](/extend/common-interface/manifest-files/in-files-s3-staging/). **Note**: Exchanging data via S3 is currently only available for input mapping. @@ -145,7 +145,7 @@ In this case, the data folders contain only [manifest files](/extend/common-inte not the actual data. This mode of operation can be enabled by setting the **Staging storage input** option to **ABS** in [component settings](https://components.keboola.com/). If this option is enabled, all the data folders will contain only manifest files, extended with an additional -[`abs` section](/extend/common-interface/manifest-files/#abs-staging). +[`abs` section](/extend/common-interface/manifest-files/in-files-abs-staging/). **Note**: Exchanging data via ABS is currently only available for input mapping. @@ -198,7 +198,7 @@ When exchanging data via workspace, there are couple of differences to loading d are always saved to the directory structure. - The `days` attribute is not supported for filtering table, use `changed_since` instead. - [Automatic Incremental Processing](/storage/tables/#automatic-incremental-processing) (also known as Adaptive Input Mapping) is not supported. -- When used for output mapping, the `columns` of the output table **must be** specified, this can be done either in the [output manifest](/extend/common-interface/manifest-files/#dataouttables-manifests) or in the [output mapping](/extend/common-interface/config-file/#output-mapping--headless-csv). +- When used for output mapping, the `columns` of the output table **must be** specified, this can be done either in the [output manifest](/extend/common-interface/manifest-files/out-tables-manifests/) or in the [output mapping](/extend/common-interface/config-file/#output-mapping--headless-csv). **Note**: Currently only some combinations of input/output staging storage settings are supported: `local<->local`, `local<->s3`, `workspace-snowflake<->workspace-snowflake`, `workspace-redshift<->workspace-redshift`. diff --git a/src/content/docs/extend/component/code-patterns/index.md b/src/content/docs/extend/component/code-patterns/index.md index 58dcd43f4..932a592d9 100644 --- a/src/content/docs/extend/component/code-patterns/index.md +++ b/src/content/docs/extend/component/code-patterns/index.md @@ -6,9 +6,9 @@ slug: 'extend/component/code-patterns' Code Patterns are a special type of [component](/extend/component/). They -- **generate code** for [transformations](/transformations/#new-transformations), +- **generate code** for [transformations](/transformations/), - implement the [generate action](/extend/component/code-patterns/interface#generate-action), and -- use the [configuration format](/extend/component/code-patterns/interface#configuration-format). +- use the [configuration format](/extend/component/code-patterns/interface/#configuration). The generated code is written in a specific [output format](/extend/component/code-patterns/interface#output-format). diff --git a/src/content/docs/extend/component/deployment/index.md b/src/content/docs/extend/component/deployment/index.md index 2da695059..ace601177 100644 --- a/src/content/docs/extend/component/deployment/index.md +++ b/src/content/docs/extend/component/deployment/index.md @@ -64,7 +64,7 @@ deploy: The `.travis.yml` file offers a vast number of [configuration options](https://docs.travis-ci.com/user/customizing-the-build/). We only need a few of them though. The options `sudo`, `language` and `services` define that all we need is Docker. The `before_script` section executes a single shell command which -[builds the image](/extend/component/tutorial/debugging/#step-2--build-the-image) and tags it `keboola-component`. The +[builds the image](/extend/component/tutorial/debugging/#step-2--build-image) and tags it `keboola-component`. The tag is completely arbitrary at this moment, but we'll need it later. The `after_success` section simply lists the built images in the log. @@ -120,7 +120,7 @@ image `quay.io/keboola/developer-portal-cli-v2`. The entire script uses the foll - `KBC_DEVELOPERPORTAL_VENDOR` -- Vendor ID - `KBC_DEVELOPERPORTAL_APP` -- Component ID -You can read more about using the Developer Portal CLI in the [chapter about running components](/extend/component/running/#running-a-component). +You can read more about using the Developer Portal CLI in the [chapter about running components](/extend/component/running/#running-component). The deploy script first pulls the image and then calls the `ecr:get-repository` command (while passing in the `KBC_DEVELOPERPORTAL_USERNAME` and `KBC_DEVELOPERPORTAL_PASSWORD` variables). The result of that command is stored in the `REPOSITORY` variable. After that the `ecr:get-login` command is called; it returns @@ -150,8 +150,8 @@ or The above deploy script requires four environment variables to be set. Set the following environment variables in the repository configuration: - `KBC_DEVELOPERPORTAL_APP` the component ID -- e.g.: `keboola-test.ex-docs-tutorial` - - `KBC_DEVELOPERPORTAL_PASSWORD` with the [**Service Account**](/extend/component/tutorial/#creating-a-deployment-account) password - - `KBC_DEVELOPERPORTAL_USERNAME` with the [**Service Account**](/extend/component/tutorial/#creating-a-deployment-account) login + - `KBC_DEVELOPERPORTAL_PASSWORD` with the [**Service Account**](/extend/component/tutorial/#creating-deployment-account) password + - `KBC_DEVELOPERPORTAL_USERNAME` with the [**Service Account**](/extend/component/tutorial/#creating-deployment-account) login - `KBC_DEVELOPERPORTAL_VENDOR` with the vendor of the component -- e.g.: `keboola-test` ![Screenshot -- Repository Configuration](/extend/component/deployment/deploy-config-3.png) @@ -282,7 +282,7 @@ If you want to use another continuous integration setting or deploy to the repos As in the [above script](/extend/component/deployment/#deploy-script), we recommend using the [Developer Portal CLI client](https://github.com/keboola/developer-portal-cli-v2). This CLI tool (runnable in Docker or PHP) allows you to obtain the repository for a component and push credentials to that repository. See the chapter about -[running components](/extend/component/running/#running-a-component), for example, how to obtain the AWS registry credentials. +[running components](/extend/component/running/#running-component), for example, how to obtain the AWS registry credentials. If you want to get even more low level, you can use the [Developer Portal API](https://api.keboola.com/?service=developer-portal) directly. It also allows you to [generate credentials for a service account](https://api.keboola.com/?service=developer-portal#post-/vendors/-vendor-/credentials) programmatically. We use our AWS ECR registry for hosting all component images. diff --git a/src/content/docs/extend/component/implementation/index.md b/src/content/docs/extend/component/implementation/index.md index 5f80475c1..431372b69 100644 --- a/src/content/docs/extend/component/implementation/index.md +++ b/src/content/docs/extend/component/implementation/index.md @@ -31,9 +31,9 @@ Before you create any complex components, be sure to read about [configurations](/storage/api/configurations/) and [processors](/extend/component/processors/) as they can substantially simplify your component code. We also recommend that you use our [common interface](/extend/common-interface/) library, which is available for -[Python](/extend/component/implementation/python/#using-the-kbc-package), -[R](/extend/component/implementation/r/#using-the-kbc-package), -and [PHP](/extend/component/implementation/php/#using-the-kbc-package). +[Python](/extend/component/implementation/python/#using-keboola-python-package), +[R](/extend/component/implementation/r/#using-keboola-package), +and [PHP](/extend/component/implementation/php/#using-keboola-package). You may use any Docker image you see fit. We recommend to base your images on those from an [official repository](https://hub.docker.com/search?q=&type=image) because they are the most stable ones. @@ -71,7 +71,7 @@ the component exit code is correct. On the other hand, the user error is suppose Also keep in mind that the output of the components (job events) serve to pass only informational and error messages; **no data** can be passed through. The event message size is limited (about 64KB). If the limit is exceeded, the message will be trimmed. If the component produces obscene amount (dozens of MBs) of output in a very short time, it may be terminated with an internal error. -Also make sure your component does not use any [output buffering](#language-specific-notes), otherwise all events will be cached after the application finishes. +Also make sure your component does not use any output buffering, otherwise all events will be cached after the application finishes. ## Implementing Processors [Processors](/extend/component/processors/) @@ -120,7 +120,7 @@ you can, for example, rely on - the CSV file being orthogonal. If the above conditions are not met, then another processor should be added before yours. I.e. you should keep the -processor simple and delegate the assumptions to other processors (and [**document** them](#publishing-a-processor)). If possible the +processor simple and delegate the assumptions to other processors (and [**document** them](#publishing-processor)). If possible the processor should also assume that the CSV files are headless and stored in arbitrary sub-folders. When implemented with this assumption the processor will support [sliced tables](/extend/common-interface/folders/#sliced-tables). diff --git a/src/content/docs/extend/component/implementation/python/index.md b/src/content/docs/extend/component/implementation/python/index.md index 3003834c2..2235e7e23 100644 --- a/src/content/docs/extend/component/implementation/python/index.md +++ b/src/content/docs/extend/component/implementation/python/index.md @@ -11,7 +11,7 @@ Use the [official images](https://hub.docker.com/_/python/) if possible. Usually smallest and fastest. We recommend using [our templates](https://github.com/keboola/component-generator/tree/master/templates). ## Working with CSV Files -We advise you to follow the guidelines for the [Python transformation](/transformations/python/#development-tutorial). +We advise you to follow the guidelines for the [Python transformation](/transformations/python-plain/#development-tutorial). The build-in CSV functions for Python work well except when the data in the CSV file contain a null character. This is [usually fixed](https://stackoverflow.com/questions/4166070/python-csv-error-line-contains-null-byte) by @@ -163,7 +163,7 @@ Apart from that, all input tables provided by user also include manifest file wi Tables and their manifest files are represented by the `keboola.component.dao.TableDefinition` object and may be loaded using the convenience method `get_input_tables_definitions()`. The result object contains all metadata about the table, -such as [manifest file](/extend/common-interface/manifest-files/#dataintables-manifests) representations (if present), system path and name. +such as [manifest file](/extend/common-interface/manifest-files/in-tables-manifests/) representations (if present), system path and name. #### Manifest & input folder content @@ -223,7 +223,7 @@ for table in tables: ### Output tables - manifest files and processing results -The component may define output [manifest files](/extend/common-interface/manifest-files/#dataouttables-manifests) +The component may define output [manifest files](/extend/common-interface/manifest-files/out-tables-manifests/) that define options on storing the results back to the Keboola Storage. This library provides methods that simplifies the manifest file creation and allows defining the export options. diff --git a/src/content/docs/extend/component/implementation/r/index.md b/src/content/docs/extend/component/implementation/r/index.md index 98c23f543..c05030e0a 100644 --- a/src/content/docs/extend/component/implementation/r/index.md +++ b/src/content/docs/extend/component/implementation/r/index.md @@ -12,7 +12,7 @@ The [R base image](https://hub.docker.com/r/rocker/r-base/) does not keep older If you want to use the same environment as in transformations, use [our image](#docker). ## Working with CSV Files -We recommend that you follow the guidelines for the [R transformation](/transformations/r/#development-tutorial). +We recommend that you follow the guidelines for the [R transformation](/transformations/r-plain/#development-tutorial). The standard R functions for CSV files work without problems: ```R diff --git a/src/content/docs/extend/component/index.md b/src/content/docs/extend/component/index.md index e4b6a9302..9ec427d2b 100644 --- a/src/content/docs/extend/component/index.md +++ b/src/content/docs/extend/component/index.md @@ -45,7 +45,7 @@ from our side. It also takes care of executing your component in its own [isolat ## Requirements Before you start developing a new component, you should -- have a [Keboola project](/#development-project) where you can test your code. +- have a [Keboola project](https://developers.keboola.com/#development-project) where you can test your code. - get yourself acquainted with Docker. You should be able to run `docker` commands. Strictly speaking, you can get away with not using them, but it will certainly speed things up for you. diff --git a/src/content/docs/extend/component/processors/index.md b/src/content/docs/extend/component/processors/index.md index 17efa713a..9fb7ed2d5 100644 --- a/src/content/docs/extend/component/processors/index.md +++ b/src/content/docs/extend/component/processors/index.md @@ -25,7 +25,7 @@ examples of working with the [Component Configuration API](/storage/api/configur If you want to implement your own processor, see our [implementation notes](/extend/component/implementation/#implementing-processors). If the component does not contain the [respective configuration field](/extend/component/ui-options/#genericdockerui-processors) or -an [advanced configuration mode](/extractors/other/aws-s3/#advanced), processors are +an [advanced configuration mode](/components/extractors/storage/aws-s3/), processors are completely **invisible in the UI**. In such case, modifying the configuration through the UI may delete the processor configuration (though you can always [rollback](https://api.keboola.com/?service=storage#post-/v2/storage/branch/-branchId-/components/-componentId-/configs/-configurationId-/versions/-versionId-/rollback)). Therefore be sure to add an **appropriate warning** to the configuration description. @@ -187,7 +187,7 @@ you have to use the [Update Configuration Row](https://api.keboola.com/?service= API call to set the processors. Provide `componentId`, `configurationId`, `rowId` and the contents of the configuration in -the same manner as when [adding a processor to configuration](#adding-a-processor). +the same manner as when [adding a processor to configuration](#adding-processor). See an example *Add processor to S3 Extractor configuration Row* in [our collection](https://documenter.getpostman.com/view/3086797/kbc-samples/77h845D?version=latest#9b9f3e7b-de3b-4c90-bad6-a8760e3852eb). diff --git a/src/content/docs/extend/component/running/index.md b/src/content/docs/extend/component/running/index.md index d60c02cec..e23140fa6 100644 --- a/src/content/docs/extend/component/running/index.md +++ b/src/content/docs/extend/component/running/index.md @@ -197,7 +197,7 @@ the component locally. You should now be able to run the component with it: If you want to run a component during development, it is the easiest to build it locally and [run the built version](/extend/component/tutorial/debugging/). If you want to run a production code component, you need to do a couple of things. Let's assume you want to run the `keboola-test.ex-docs-tutorial` component and you have -already [prepared the data directory](#preparing-the-data-folder). +already [prepared the data directory](#preparing-data-folder). The next step is to obtain the repository settings and credentials from the [Developer Portal](https://components.keboola.com/). You can either use the [API](https://api.keboola.com/?service=developer-portal) or @@ -262,7 +262,7 @@ This is a known [bug in Docker](https://github.com/docker/for-win/issues/1306), ## Running Transformations Both R and Python transformations are implemented as Docker components. They can be run -locally as well. Use the [Run Job API call in debug mode](/extend/component/running/#preparing-the-data-folder) to obtain the data directory. +locally as well. Use the [Run Job API call in debug mode](/extend/component/running/#preparing-data-folder) to obtain the data directory. In the [API call](https://api.keboola.com/?service=job-queue#post-/jobs), specify the full configuration (using the `configData` node). See [examples](https://documenter.getpostman.com/view/3086797/kbc-samples/77h845D?version=latest#9b9f3e7b-de3b-4c90-bad6-a8760e3852eb) for both R and Python transformations. diff --git a/src/content/docs/extend/component/tutorial/index.md b/src/content/docs/extend/component/tutorial/index.md index 0da9c7750..0acfdb469 100644 --- a/src/content/docs/extend/component/tutorial/index.md +++ b/src/content/docs/extend/component/tutorial/index.md @@ -32,7 +32,7 @@ you should also receive access to a development Keboola project. ![Screenshot -- Join a vendor](/extend/component/tutorial/join-vendor.png) In order to create a **new vendor**, a Keboola administrator has to approve your request, and you will -receive a [development project](/#development-project) in Keboola. In addition to that, you need to provide us +receive a [development project](https://developers.keboola.com/#development-project) in Keboola. In addition to that, you need to provide us with a channel for receiving internal errors from your components. Anything supported by [Papertrail notifications](https://help.papertrailapp.com/kb/how-it-works/alerts#supported-services) is available, though e-mail or a Slack channel is most commonly used. @@ -61,7 +61,7 @@ Choose the appropriate [component type](/extend/component/#component-types): - `extractor` -- brings data into Keboola - `writer` -- sends data out of Keboola -- `transformation` -- does some transformation of the data, [read more](/transformations/#new-transformations) +- `transformation` -- does some transformation of the data, [read more](/transformations/) - `code pattern` -- generates code for transformation's component, [read more](/extend/component/code-patterns) - `application` -- another arbitrary component diff --git a/src/content/docs/extend/component/tutorial/output-mapping/index.md b/src/content/docs/extend/component/tutorial/output-mapping/index.md index ba6a25dd0..8922bdb46 100644 --- a/src/content/docs/extend/component/tutorial/output-mapping/index.md +++ b/src/content/docs/extend/component/tutorial/output-mapping/index.md @@ -75,7 +75,7 @@ This script reads a CSV file line by line and checks whether it is odd or even. Finally, the result is written to either `odd.csv` or `even.csv`. Commit and push the code in your repository and tag it with a [normal version tag](https://semver.org/#spec-item-2). -This will trigger a [build on Travis CI](/extend/component/tutorial/#building-the-component) and automatically +This will trigger a [build on Travis CI](/extend/component/tutorial/#building-component) and automatically deploy the new version into Keboola. Keep in mind that after the deployment, it may take up to 5 minutes for the update to propagate to all Keboola instances. ## Verifying diff --git a/src/content/docs/extend/job-queue/index.md b/src/content/docs/extend/job-queue/index.md index d073cef4a..958b06a16 100644 --- a/src/content/docs/extend/job-queue/index.md +++ b/src/content/docs/extend/job-queue/index.md @@ -23,13 +23,13 @@ All [components](/extend/component/), including our internal R and Python Transf The Job Queue functionality can be described in the following steps: - Download and build the specified Docker image. -- Download all [tables](/extend/common-interface/folders/#dataintables-folder) and [files](/extend/common-interface/folders/#datainfiles-folder) specified in the input mapping from Storage. +- Download all [tables](/extend/common-interface/folders/#folder-dataintables) and [files](/extend/common-interface/folders/#folder-datainfiles) specified in the input mapping from Storage. - Create a [configuration file](/extend/common-interface/config-file/). - Run [before processors](/extend/component/processors/) if there are any. - Run the Docker image (create a Docker container). - Run [after processors](/extend/component/processors/) if there are any. -- Upload all [tables](/extend/common-interface/folders/#dataouttables-folder) and -[files](/extend/common-interface/folders/#dataoutfiles-folder) in the output mapping to Storage. +- Upload all [tables](/extend/common-interface/folders/#folder-dataouttables) and +[files](/extend/common-interface/folders/#folder-dataoutfiles) in the output mapping to Storage. - Delete the container and all temporary files. When the component execution is finished, Job Queue automatically collects the exit code and the content of STDOUT and STDERR. @@ -65,7 +65,7 @@ The [Job Queue API](https://api.keboola.com/?service=job-queue) has API calls to - run a [component](/extend/component/). - [encrypt values](/overview/encryption/). -- [prepare the data folder](/extend/component/running/#preparing-the-data-folder). +- [prepare the data folder](/extend/component/running/#preparing-data-folder). - run [component actions](/extend/common-interface/actions/). - run a [component](/extend/component/) with a [specified Docker image tag](https://api.keboola.com/?service=job-queue#post-/jobs), usable for [testing images](/extend/component/deployment/#test-live-configurations). diff --git a/src/content/docs/management/account/index.md b/src/content/docs/management/account/index.md index ea4f2703a..9e4a1aad8 100644 --- a/src/content/docs/management/account/index.md +++ b/src/content/docs/management/account/index.md @@ -43,12 +43,12 @@ We support two types of MFA: 1. **Time-based one-time password** (TOTP) — a **software**-based authentication technique. When logging into a site supporting TOTP, the authenticator app generates a six-digit one-time password that you must enter in addition to your usual login details. You can use a phone or another device -as a virtual multi-factor authentication (TOTP) device. Follow the [instructions](#totp-multi-factor-authentication). +as a virtual multi-factor authentication (TOTP) device. Follow the [instructions](#totp). 2. **Universal 2nd Factor** (U2F) — a **hardware** device. When signing in, activate your security key as suggested in its documentation (e.g., by pressing a button) rather than typing in a verification code. For using security keys, use the [FIDO U2F](https://fidoalliance.org/) standard. -Follow the [instructions](#u2f-multi-factor-authentication). +Follow the [instructions](#u2f). ### TOTP [TOTP](https://en.wikipedia.org/wiki/Time-based_One-time_Password_algorithm) @@ -136,7 +136,7 @@ To do that, follow the key's documentation (e.g., press the button): ![Screenshot -- Enable U2F -- Step 7](/management/account/enable-u2f-6.png) ## Invitations -The **Invitations** page shows a project or a list of projects you have been [invited](/management/project/users/#inviting-a-user) to. +The **Invitations** page shows a project or a list of projects you have been [invited](/management/project/users/#inviting-user) to. When you accept an invitation, you'll become a user of the project and will be redirected there. When you decline the invitation, you'll lose the opportunity to enter the project. diff --git a/src/content/docs/management/index.md b/src/content/docs/management/index.md index 2633c04db..9a36d898e 100644 --- a/src/content/docs/management/index.md +++ b/src/content/docs/management/index.md @@ -15,7 +15,7 @@ Everything that happens with your data in Keboola at any stage of your project i It enables you to use your uploaded data for other tables, access previous versions of your tables, and extract all your project data. Everything done in Keboola is traceable, and if possible, also recoverable. -- [**Jobs**](/management/jobs/#jobs) — give you an overview of all your jobs, +- [**Jobs**](/management/jobs/) — give you an overview of all your jobs, running and finished (what tables were modified, how much data was transferred and what events occurred during the job execution). - [**Storage Jobs**](/management/jobs/#storage-jobs) — track all data diff --git a/src/content/docs/management/jobs/index.md b/src/content/docs/management/jobs/index.md index 01a5904b3..f12e9ba87 100644 --- a/src/content/docs/management/jobs/index.md +++ b/src/content/docs/management/jobs/index.md @@ -14,7 +14,7 @@ All jobs are logged and their tracked history is virtually unlimited. Click on a - what tables were imported (created by the job and imported into your Storage). - what tables were exported (read from your Storage by the job). -- how many [credits](/management/project/limits/#project-power) were used by running the job. +- how many [credits](/management/project/limits/#project-power--time-credits) were used by running the job. - what events occurred during the job execution. - what exact parameters were used for the job (this might be useful when working with the [API](https://developers.keboola.com/integrate/jobs/#apis-for-working-with-jobs)). diff --git a/src/content/docs/management/organization/index.md b/src/content/docs/management/organization/index.md index c77c28e9b..a5cd1c8de 100644 --- a/src/content/docs/management/organization/index.md +++ b/src/content/docs/management/organization/index.md @@ -90,7 +90,7 @@ Project templates differ mainly in the project expiration time. Some may be subj ## Organization Settings ### Auto Join -Some organizations may prefer to explicitly approve [access](/management/project/users/#who-can-access-a-project) +Some organizations may prefer to explicitly approve [access](/management/project/users/#who-can-access-project) to their project by the [support team](/management/support/#require-approval-for-support-access). To disable Auto Join and require approval for support accounts, click the **Change** link to modify the settings: diff --git a/src/content/docs/management/project/limits/index.md b/src/content/docs/management/project/limits/index.md index 6059dbbfa..e3281d49a 100644 --- a/src/content/docs/management/project/limits/index.md +++ b/src/content/docs/management/project/limits/index.md @@ -103,10 +103,10 @@ The storage size is the sum of the sizes of the tables in your [table Storage](/ [Aliases](/storage/tables/#aliases) and [linked buckets](/catalog/) do not count towards this number, and neither do [files](/storage/files/). -The table storage size is measured as it is reported by the underlying [backend](/storage/#backend-properties). +The table storage size is measured as it is reported by the underlying [backend](/storage/#storage-backend-types-and-features). This means that the reported size is substantially smaller than the size of imported raw CSV files, thanks to compression used by the database backend. This also means that reported sizes of the same data may differ slightly -across projects with different [backends](/storage/#backend-properties) (or between buckets in a project +across projects with different [backends](/storage/#storage-backend-types-and-features) (or between buckets in a project with mixed backends). ## Platform Limits diff --git a/src/content/docs/management/project/tokens/index.md b/src/content/docs/management/project/tokens/index.md index 9cf83af1e..d249e65e4 100644 --- a/src/content/docs/management/project/tokens/index.md +++ b/src/content/docs/management/project/tokens/index.md @@ -30,7 +30,7 @@ Tokens can be managed from the **Project Settings > API Tokens** page. ## Master Tokens Tokens that belong to project administrators are called **master tokens**. Their description is the email of the user they belong to. Master tokens cannot be modified, shared or deleted. -The only way to delete a master token is by [removing the user](/management/project/users/#removing-a-user) +The only way to delete a master token is by [removing the user](/management/project/users/#removing-user) from the project on the **Project Settings > Users** page. A single user has only a single master token. In addition, master tokens are the only ones which can be @@ -55,7 +55,7 @@ These are the typical reasons to manually create a new API token: Although tokens cannot be used to directly log in to the Keboola user interface, they do allow executing almost all operations in a Keboola project. As such, they must be treated as secret. Therefore the token string is shown only when the token is created and it is not accessible later. You should -immediately [refresh a token](#refreshing-a-token) in case there is a suspicion that the +immediately [refresh a token](#refreshing-token) in case there is a suspicion that the token string was revealed to unauthorized persons. When creating a new token, the following rules apply: @@ -117,7 +117,7 @@ You would then create a token that is authorized for running the **MySQL databas *Note: For historical reasons, specifying the Orchestrator component in component permissions is optional. This means that the token will also work if it has access to no components.* -You can then [share the token](#sharing-a-token) to the person responsible for the database process and be +You can then [share the token](#sharing-token) to the person responsible for the database process and be sure that they can use only that particular component in that particular bucket. They will be even able to reconfigure it — e.g., update the extraction queries (but only via the API). Also, writing to a limited set of buckets is a good way of preventing accidentally overwriting data. diff --git a/src/content/docs/management/support/index.md b/src/content/docs/management/support/index.md index 1843f79c9..256e0d776 100644 --- a/src/content/docs/management/support/index.md +++ b/src/content/docs/management/support/index.md @@ -18,7 +18,7 @@ a button to contact our support in the error alert message. ## Keboola Support Users In order to solve an issue or gain context regarding an application error, a technical support member -may require access to your project. By default, Keboola support staff [may join](/management/project/users/#who-can-access-a-project) your project when +may require access to your project. By default, Keboola support staff [may join](/management/project/users/#who-can-access-project) your project when requested, and they will appear in the member's list marked by the Keboola badge: ![Screenshot -- User joined](/management/support/users.png) @@ -55,7 +55,7 @@ reason stated in the request and also which project user approved the request. ### Organization Changes When [Auto Join](#require-approval-for-support-access) is disabled, it limits -[who can access the projects](/management/project/users/#who-can-access-a-project) in the organization. It also means +[who can access the projects](/management/project/users/#who-can-access-project) in the organization. It also means that neither [the maintainer](/management/organization/), nor Keboola Support can join the organization itself. In other words, when Auto Join is turned off for an organization, only the organization users and project users may diff --git a/src/content/docs/management/telemetry/telemetry-dashboards/index.md b/src/content/docs/management/telemetry/telemetry-dashboards/index.md index 647c86161..60006cb94 100644 --- a/src/content/docs/management/telemetry/telemetry-dashboards/index.md +++ b/src/content/docs/management/telemetry/telemetry-dashboards/index.md @@ -47,7 +47,7 @@ Four summary cards are displayed at the top: ![Project Consumption](/management/telemetry/telemetry-dashboards/project-consumption.png) ## Organization Usage -The Organization Usage dashboard shows consumption across all projects in your organization. It is available to organizations with an active contract. This content is also available as the [Organization Usage tab](#ac-organization-usage) within the Activity Center. +The Organization Usage dashboard shows consumption across all projects in your organization. It is available to organizations with an active contract. This content is also available as the [Organization Usage tab](#organization-usage-activity-center) within the Activity Center. ### KPIs Four KPI cards compare current usage against contract limits: @@ -117,7 +117,7 @@ All five tabs share the following controls: ![Activity Center Header](/management/telemetry/telemetry-dashboards/ac-header.png) -### Organization Usage {#ac-organization-usage} +### Organization Usage (Activity Center) This tab is identical in content and functionality to the standalone [Organization Usage](#organization-usage) dashboard described above — KPIs with contract limits, Active Contract Consumption, Consumption charts, Top Projects and Components, and Configuration Health. Please refer to that section for a detailed description of each visual. ![Activity Center Organization Usage](/management/telemetry/telemetry-dashboards/ac-org-usage.png) diff --git a/src/content/docs/overview/index.md b/src/content/docs/overview/index.md index 31fc780a8..349d623a9 100644 --- a/src/content/docs/overview/index.md +++ b/src/content/docs/overview/index.md @@ -119,7 +119,7 @@ The platform automates infrastructure, user, and data management, offering servi and reverse billing. Components can be private or shared with Keboola users via our marketplace featuring applications mainly from 3rd parties to enhance workflows and support a composable enterprise. -Components can be run as standard pieces of our Flows [/tutorial/automate/#main-header], obtaining the full support and services (a link to your +Components can be run as standard pieces of our [Flows](/tutorial/automate/), obtaining the full support and services (a link to your [components](https://components.keboola.com/components), [logs, etc.](https://developers.keboola.com/extend/common-interface/)). ### Keboola CLI @@ -175,6 +175,6 @@ This token system enables easy [sharing of specific resources](/management/proje ### Input and Output Mapping To make sure your transformation does not harm data in Storage, [mapping](/transformations/mappings) separates source data from your script. -A secure [workspace](/workspace/#main-header) is created with data copied from the tables specified +A secure [workspace](/workspace/) is created with data copied from the tables specified in the [input mapping](/transformations/mappings/#input-mapping). After the transformation is executed successfully, only tables and files defined in the [output mapping](/transformations/mappings/#output-mapping) are returned to Storage. diff --git a/src/content/docs/transformations/code-patterns/index.md b/src/content/docs/transformations/code-patterns/index.md index a4b6f6216..20dd7e609 100644 --- a/src/content/docs/transformations/code-patterns/index.md +++ b/src/content/docs/transformations/code-patterns/index.md @@ -8,7 +8,7 @@ slug: 'transformations/code-patterns' Code Pattern is a special type of [component](/components/) that - generates code based on [parameters](#parameters-form), and -- can be used in the user interface of [New Transformations](/transformations/#new-transformations). +- can be used in the user interface of [New Transformations](/transformations/). ## List of Code Patterns diff --git a/src/content/docs/transformations/index.md b/src/content/docs/transformations/index.md index 11f4c6631..2c582a53c 100644 --- a/src/content/docs/transformations/index.md +++ b/src/content/docs/transformations/index.md @@ -185,11 +185,11 @@ Python and R transformations. ✓ - Phases + Phases Not available - Dependencies + Dependencies Not available diff --git a/src/content/docs/transformations/mappings/index.md b/src/content/docs/transformations/mappings/index.md index 86cc33abc..bdb2cc6a6 100644 --- a/src/content/docs/transformations/mappings/index.md +++ b/src/content/docs/transformations/mappings/index.md @@ -54,7 +54,7 @@ Depending on the transformation backend, the table input mapping process can do - **Database Staging**: Copy the selected tables to *tables* in a newly created *database* schema. If you have not selected any tables and [read-only input mappings](/transformations/mappings/#read-only-input-mapping) are enabled, you can access them automatically in the workspace. In this case the tables are not copied. - **File Staging**: Export the selected tables to *CSV files* and copy them to a designated staging *Storage* -(not to be confused with a [file mapping](/transformations/mappings/#file-mapping), as we're still working with tables). +(not to be confused with a [file mapping](/transformations/mappings/#file-input-mapping), as we're still working with tables). Depending on the transformation types, you can either build your transformations working with database tables or with CSV files. Furthermore, the CSV files can be placed locally with the transformation @@ -181,7 +181,7 @@ This function is automatically enabled in transformations. ##### Read-only input mapping -*Note: You must be using [new transformations](/transformations/#new-transformations) to see this feature.* +*Note: You must be using [new transformations](/transformations/) to see this feature.* When **read-only input mappings** are enabled, you automatically have read access to all buckets and tables in the project (this also applies to linked buckets). Alias tables are materialized as database VIEWs and are fully accessible via read-only input mappings — including filtered aliases and aliases from linked buckets. diff --git a/src/content/docs/tutorial/ad-hoc/index.md b/src/content/docs/tutorial/ad-hoc/index.md index f48296401..08983340a 100644 --- a/src/content/docs/tutorial/ad-hoc/index.md +++ b/src/content/docs/tutorial/ad-hoc/index.md @@ -36,7 +36,7 @@ Then create a [service account](https://cloud.google.com/bigquery/docs/authentic of the Google BigQuery data source connector, and create a Google Storage bucket as a temporary storage for off-loading the data from BigQuery. ***Note:** If setting up the Google BigQuery connector seems too complicated to you, export the query results to Google Sheets and -[load them from Google Sheets](/tutorial/load/googlesheets/). Or, export them to a CSV file and [load them from local files](/tutorial/load/#manually-loading-data).* +[load them from Google Sheets](/tutorial/load/googlesheets/). Or, export them to a CSV file and [load them from local files](/tutorial/load/#manual-data-loading).* ### Prepare Before you start, have a Google service account and a Google Storage bucket ready. diff --git a/src/content/docs/tutorial/branches/project-diff.md b/src/content/docs/tutorial/branches/project-diff.md index bd36b43ce..a4db83f97 100644 --- a/src/content/docs/tutorial/branches/project-diff.md +++ b/src/content/docs/tutorial/branches/project-diff.md @@ -33,7 +33,7 @@ production storage will not be affected by the merge. This means that no tables unless you run the configurations that created them, and no data will be transferred from the branch to production. For example, the table `bitcoin_transactions` which you -[created in branch](/tutorial/branches/tables-in-branch/#extend-the-transformation) will not be transferred to production, +[created in branch](/tutorial/branches/tables-in-branch/#extend-transformation) will not be transferred to production, and if the branch is deleted after the merge, the branch version of the table will be discarded as well. The table `bitcoin_transactions` will be created by running the HTTP data source connector in production after you merge it. diff --git a/src/content/docs/tutorial/manipulate/index.md b/src/content/docs/tutorial/manipulate/index.md index 7ccb48d4e..be149222e 100644 --- a/src/content/docs/tutorial/manipulate/index.md +++ b/src/content/docs/tutorial/manipulate/index.md @@ -31,8 +31,8 @@ meaning queries are not executed directly against your Storage tables. Instead, executes queries, and finally unloads created/modified objects back to the Storage. 1. [**Input Mapping**](/transformations/mappings/#input-mapping): This is where you specify the tables to be used in your transformation. In the default setup, tables not mentioned in Input Mapping cannot be used in the transformation. -2. [**Output Mapping**](/transformations/#output-mapping): This section deals with tables created or modified within your transformation. Here, you specify the tables that will be written into Storage after the successful execution of the transformation. Tables not mentioned in Output Mapping will neither be modified nor permanently stored; they are considered temporary. -3. [**Queries**](/tutorial/manipulate/#transformation-script): SQL queries define what will happen with the data. These queries take the tables from Input Mapping, modify them, and produce the tables referenced in Output Mapping. To enhance clarity, queries can be further organized into blocks. +2. [**Output Mapping**](/transformations/mappings/#output-mapping): This section deals with tables created or modified within your transformation. Here, you specify the tables that will be written into Storage after the successful execution of the transformation. Tables not mentioned in Output Mapping will neither be modified nor permanently stored; they are considered temporary. +3. [**Queries**](/tutorial/manipulate/#transformation-queries): SQL queries define what will happen with the data. These queries take the tables from Input Mapping, modify them, and produce the tables referenced in Output Mapping. To enhance clarity, queries can be further organized into blocks. The mapping concept serves as a crucial safeguard when manipulating your data. It ensures that there is no accidental modification of the wrong tables. The only tables modified by your transformation are those explicitly specified in the Output Mapping. Additionally, this concept plays a vital role in maintaining a diff --git a/src/content/docs/workspace/index.md b/src/content/docs/workspace/index.md index bb2e45756..db990fe0e 100644 --- a/src/content/docs/workspace/index.md +++ b/src/content/docs/workspace/index.md @@ -249,7 +249,7 @@ advantage of [alias tables](/storage/tables/#aliases) and prepare buckets with t ### Read-Only Input Mapping -*Note: You must be using [new transformations](/transformations/#new-transformations) to see this feature.* +*Note: You must be using [new transformations](/transformations/) to see this feature.* The workspace also supports **read-only input mappings**, as described in the [mapping section](/transformations/mappings/#read-only-input-mapping). For each **workspace** or **Snowflake writer** (data destination) configuration, users can choose whether to use a **read-only input mapping**. diff --git a/src/content/docs/workspace/snowflake-workspaces-access-changes/index.md b/src/content/docs/workspace/snowflake-workspaces-access-changes/index.md index e96a95564..e7045da1a 100644 --- a/src/content/docs/workspace/snowflake-workspaces-access-changes/index.md +++ b/src/content/docs/workspace/snowflake-workspaces-access-changes/index.md @@ -81,7 +81,7 @@ With the upcoming workspace access changes, this editor will serve as the go-to ✅ MT/PAYG -Projects using Keboola's shared Snowflake backend (MT/PAYG) will no longer be able to use the [Keboola-provisioned Snowflake database](/components/writers/database/snowflake/#using-keboola-provisioned-database) as a data destination for Snowflake Writer. +Projects using Keboola's shared Snowflake backend (MT/PAYG) will no longer be able to use the [Keboola-provisioned Snowflake database](/components/writers/database/snowflake/#using-keboola-snowflake-database) as a data destination for Snowflake Writer. The option to use the Keboola Snowflake database has been used mainly for connecting 3rd party BI tools. This use case will be covered by the recently released [Data Gateway](/components/applications/data-gateway/) component. diff --git a/src/content/docs/workspace/table-export.md b/src/content/docs/workspace/table-export.md index e589e6d1f..0a728f544 100644 --- a/src/content/docs/workspace/table-export.md +++ b/src/content/docs/workspace/table-export.md @@ -44,7 +44,7 @@ Content-Type: application/json ### Response -The endpoint returns a standard asynchronous [storage job](/overview/#storage-jobs) with HTTP 202. When the job +The endpoint returns a standard asynchronous [storage job](/management/jobs/#storage-jobs) with HTTP 202. When the job finishes, its `results` contain the ID of the exported file: ```json From c66a9650cf27e4212a42a388c3a84df97a236c83 Mon Sep 17 00:00:00 2001 From: Nikita Date: Tue, 4 Aug 2026 02:09:37 +0200 Subject: [PATCH 2/2] fix(links): sharpen five anchor targets after the fact-check MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The sweep resolved every anchor, but the fact-check found better homes for a few. All verified against the build: - `/workspace/table-export/`: "returns a standard asynchronous storage job" → `/storage/jobs/`, the page that defines the concept, instead of `/management/jobs/#storage-jobs`, which is a UI walkthrough. - `/management/project/limits/` (2 links): how table size is measured → `/storage/#storage-data`, where "backend" is defined. The section I had used, Storage Backend Types and Features, is about tenancy models (MT/KBDB/BYODB). - `/overview/`: the Flows mention now goes to `/flows/`, matching the same page's other two Flows links, rather than to the Part 4 tutorial. - MySQL binlog note: the original `#mysql-purges-binlog-files-used-by-debezium` was copied verbatim from Debezium's own docs and refers to the *failure mode* (connector stops too long → server purges the binlog → new snapshot needed). Our `### Functionality` only says snapshots exist because MySQL purges binlogs, and nothing in-repo covers the failure mode, so the link goes upstream to Debezium. - Five `[advanced mode](…/sqldb/)` links pick up `#advanced-mode`, the anchor sqldb's own prose already uses. Anchors still 0 broken (8480 checked); audit-phase2 unchanged. Co-Authored-By: Claude Opus 5 --- src/content/docs/components/extractors/database/index.md | 4 ++-- .../docs/components/extractors/database/ms-sql/index.md | 2 +- .../docs/components/extractors/database/mysql/index.md | 4 ++-- .../docs/components/extractors/database/oracle/index.md | 2 +- .../docs/components/extractors/database/postgresql/index.md | 2 +- src/content/docs/management/project/limits/index.md | 4 ++-- src/content/docs/overview/index.md | 2 +- src/content/docs/workspace/table-export.md | 2 +- 8 files changed, 11 insertions(+), 11 deletions(-) diff --git a/src/content/docs/components/extractors/database/index.md b/src/content/docs/components/extractors/database/index.md index 0f5d8c11b..06af4ab09 100644 --- a/src/content/docs/components/extractors/database/index.md +++ b/src/content/docs/components/extractors/database/index.md @@ -32,7 +32,7 @@ Several variants of connectors may exist for each database type, depending on th These connectors work on a relational level, performing queries against the source database to synchronize data. This straightforward approach suits most use cases and supports Timestamp-based CDC replication. -All are [configured](/components/extractors/database/sqldb/#initial-setup) similarly and offer an [advanced mode](/components/extractors/database/sqldb/). +All are [configured](/components/extractors/database/sqldb/#initial-setup) similarly and offer an [advanced mode](/components/extractors/database/sqldb/#advanced-mode). Their basic configuration is also part of the [Tutorial - Loading Data from Database](/tutorial/load/database/). @@ -51,7 +51,7 @@ Typically, these connectors are useful in the following scenarios: Unlike the connectors for SQL databases, connectors for **NoSQL databases** require a different configuration (except the [BigQuery data source connector](/components/extractors/database/bigquery/) for the [BigQuery](https://cloud.google.com/bigquery/) -database, which is quite similar to SQL databases and also supports the [advanced mode](/components/extractors/database/sqldb/)): +database, which is quite similar to SQL databases and also supports the [advanced mode](/components/extractors/database/sqldb/#advanced-mode)): - [MongoDB connector](/components/extractors/database/mongodb/) for the [MongoDB](https://www.mongodb.com/) database and the [CosmosDB for MongoDB API](https://docs.microsoft.com/en-us/azure/cosmos-db/mongodb-introduction). - [CosmosDB connector](/components/extractors/database/cosmosdb/) for the [CosmosDB SQL API](https://docs.microsoft.com/en-us/azure/cosmos-db/tutorial-query-sql-api). diff --git a/src/content/docs/components/extractors/database/ms-sql/index.md b/src/content/docs/components/extractors/database/ms-sql/index.md index edecfaeef..3894be250 100644 --- a/src/content/docs/components/extractors/database/ms-sql/index.md +++ b/src/content/docs/components/extractors/database/ms-sql/index.md @@ -14,7 +14,7 @@ This connector supports the most recent versions of both SQL Server and Azure SQ This [standard SQL database connector](/components/extractors/database/sqldb) performs queries against the source database to synchronize data. It offers a straightforward approach suitable for most use cases, enabling [time-stamp based](/components/extractors/database/#incremental-fetching) CDC replication. -All SQL database connectors are [configured](/components/extractors/database/sqldb/#initial-setup) similarly and offer an [advanced mode](/components/extractors/database/sqldb/). +All SQL database connectors are [configured](/components/extractors/database/sqldb/#initial-setup) similarly and offer an [advanced mode](/components/extractors/database/sqldb/#advanced-mode). For guidance on basic configuration, please refer to our tutorial: [Loading Data with Database data source connector](/tutorial/load/database/). diff --git a/src/content/docs/components/extractors/database/mysql/index.md b/src/content/docs/components/extractors/database/mysql/index.md index a79fc5c55..797a4304e 100644 --- a/src/content/docs/components/extractors/database/mysql/index.md +++ b/src/content/docs/components/extractors/database/mysql/index.md @@ -18,7 +18,7 @@ This [standard connector](https://components.keboola.com/components/keboola.ex-d It is a straightforward approach suitable for most use cases, allowing for [time-stamp based](/components/extractors/database/#incremental-fetching) CDC replication. All connectors are [configured](/components/extractors/database/sqldb/#initial-setup) similarly and -offer an [advanced mode](/components/extractors/database/sqldb/). +offer an [advanced mode](/components/extractors/database/sqldb/#advanced-mode). Basic configuration is covered in the [Tutorial - Loading Data from Database](/tutorial/load/database/). @@ -336,7 +336,7 @@ Descriptions of MySQL binlog configuration properties: | `log_bin` | The value of `log_bin` is the base name of the sequence of binlog files. | | `binlog_format` | The `binlog-format` must be set to `ROW` or `row`. | | `binlog_row_image` | The `binlog_row_image` must be set to `FULL` or `full`. | -| `binlog_expire_logs_seconds` | The `binlog_expire_logs_seconds` corresponds to the deprecated system variable `expire_logs_days`. This is the number of seconds for automatic binlog file removal. The default is `2592000`, which equals 30 days. Set the value to match the needs of your environment. See [MySQL purges binlog files](#functionality). | +| `binlog_expire_logs_seconds` | The `binlog_expire_logs_seconds` corresponds to the deprecated system variable `expire_logs_days`. This is the number of seconds for automatic binlog file removal. The default is `2592000`, which equals 30 days. Set the value to match the needs of your environment. See [MySQL purges binlog files](https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-purges-binlog-files-used-by-debezium). | #### Enabling GTIDs diff --git a/src/content/docs/components/extractors/database/oracle/index.md b/src/content/docs/components/extractors/database/oracle/index.md index ebf9773f7..048d19b0b 100644 --- a/src/content/docs/components/extractors/database/oracle/index.md +++ b/src/content/docs/components/extractors/database/oracle/index.md @@ -11,7 +11,7 @@ This is a [standard connector](https://components.keboola.com/components/keboola It is the simplest approach suitable for most use cases and allows for [time-stamp based](/components/extractors/database/#incremental-fetching) CDC replication. They are all [configured](/components/extractors/database/sqldb/#initial-setup) in the same way and -have an [advanced mode](/components/extractors/database/sqldb/). +have an [advanced mode](/components/extractors/database/sqldb/#advanced-mode). Their basic configuration is also part of the [Tutorial - Loading Data with Database Extractor](/tutorial/load/database/). diff --git a/src/content/docs/components/extractors/database/postgresql/index.md b/src/content/docs/components/extractors/database/postgresql/index.md index 4e0a5a8f1..17171fa15 100644 --- a/src/content/docs/components/extractors/database/postgresql/index.md +++ b/src/content/docs/components/extractors/database/postgresql/index.md @@ -19,7 +19,7 @@ This is a [standard connector](https://components.keboola.com/components/keboola It is the simplest approach suitable for most use cases and allows for [time-stamp based](/components/extractors/database/#incremental-fetching) CDC replication. They are all [configured](/components/extractors/database/sqldb/#initial-setup) in the same way and -have an [advanced mode](/components/extractors/database/sqldb/). +have an [advanced mode](/components/extractors/database/sqldb/#advanced-mode). Their basic configuration is also part of the [Tutorial - Loading Data with Database Extractor](/tutorial/load/database/). diff --git a/src/content/docs/management/project/limits/index.md b/src/content/docs/management/project/limits/index.md index e3281d49a..dbbf5e18e 100644 --- a/src/content/docs/management/project/limits/index.md +++ b/src/content/docs/management/project/limits/index.md @@ -103,10 +103,10 @@ The storage size is the sum of the sizes of the tables in your [table Storage](/ [Aliases](/storage/tables/#aliases) and [linked buckets](/catalog/) do not count towards this number, and neither do [files](/storage/files/). -The table storage size is measured as it is reported by the underlying [backend](/storage/#storage-backend-types-and-features). +The table storage size is measured as it is reported by the underlying [backend](/storage/#storage-data). This means that the reported size is substantially smaller than the size of imported raw CSV files, thanks to compression used by the database backend. This also means that reported sizes of the same data may differ slightly -across projects with different [backends](/storage/#storage-backend-types-and-features) (or between buckets in a project +across projects with different [backends](/storage/#storage-data) (or between buckets in a project with mixed backends). ## Platform Limits diff --git a/src/content/docs/overview/index.md b/src/content/docs/overview/index.md index 349d623a9..5691b0318 100644 --- a/src/content/docs/overview/index.md +++ b/src/content/docs/overview/index.md @@ -119,7 +119,7 @@ The platform automates infrastructure, user, and data management, offering servi and reverse billing. Components can be private or shared with Keboola users via our marketplace featuring applications mainly from 3rd parties to enhance workflows and support a composable enterprise. -Components can be run as standard pieces of our [Flows](/tutorial/automate/), obtaining the full support and services (a link to your +Components can be run as standard pieces of our [Flows](/flows/), obtaining the full support and services (a link to your [components](https://components.keboola.com/components), [logs, etc.](https://developers.keboola.com/extend/common-interface/)). ### Keboola CLI diff --git a/src/content/docs/workspace/table-export.md b/src/content/docs/workspace/table-export.md index 0a728f544..12906096b 100644 --- a/src/content/docs/workspace/table-export.md +++ b/src/content/docs/workspace/table-export.md @@ -44,7 +44,7 @@ Content-Type: application/json ### Response -The endpoint returns a standard asynchronous [storage job](/management/jobs/#storage-jobs) with HTTP 202. When the job +The endpoint returns a standard asynchronous [storage job](/storage/jobs/) with HTTP 202. When the job finishes, its `results` contain the ID of the exported file: ```json