From 2cdc887bcd590a9e4c9a3a03e1378d87eb8bcfbf Mon Sep 17 00:00:00 2001 From: zhangjinpeng1987 Date: Thu, 13 Jan 2022 21:08:39 +0800 Subject: [PATCH 1/2] PD: supports multiple level meta data space Signed-off-by: zhangjinpeng1987 --- text/0083-multi-level-meta-data-space.md | 52 ++++++++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 text/0083-multi-level-meta-data-space.md diff --git a/text/0083-multi-level-meta-data-space.md b/text/0083-multi-level-meta-data-space.md new file mode 100644 index 00000000..414dff41 --- /dev/null +++ b/text/0083-multi-level-meta-data-space.md @@ -0,0 +1,52 @@ +# Multi-level Meta Data Space + +## Summary + +PD can handle multiple users' meta data, each user only see its own meta data. + +## Motivation + +PD is the mata data center of TiKV Cluster, the meta data contains TiKV instances' addr, user data range(region) +information, the number of replicas of data, etc. + +But PD currently only can handle the key space [min-key, max-key], it means that PD only can handle one user's +meta data, it is not possible when multiple users want share the same PD cluster as their meta system. There are +two different scenarios that require PD can handle multiple user' meta data: + +1. Multiple TiKV Cluster share the same PD cluster. Because the minimal demplyment of a TiKV Cluster is 3 TiKV 3 PD, +but it is not cost-effect if every small cluster has 3 dedicated meta data node. +2. There are Multiple tenant in the same TiKV Cluster, each tenant has it own meta data, each tenant's key range can +contains any key in the range of [min-key, max-key]. + +## Detailed design + +Change the meta data from +``` +{meta data} +``` +to +``` +user1 {meta data} +user2 {meta data} +user3 {meta data} +... +``` + +### Compatibility + +When upgrade from old version, all legacy meta data belongs to the default meta data space. +``` +{meta data} +``` +``` +default {meta data} +user1 {meta data} +user2 {meta data} +... +``` + +## Alternatives + +In the multi-tenant scenario, tenant can add a {tenant-id} prefix for each data key, but tenant-id +is a meta data esstionally, each data key has a tenant-id prefix may cost more disk space & memory +in TiKV. From 8ea74e248ca37205995216516a0d997977386a5e Mon Sep 17 00:00:00 2001 From: zhangjinpeng1987 Date: Fri, 14 Jan 2022 17:26:01 +0800 Subject: [PATCH 2/2] address comments Signed-off-by: zhangjinpeng1987 --- text/0083-multi-level-meta-space.md | 47 +++++++++++++++++++++++++++++ 1 file changed, 47 insertions(+) create mode 100644 text/0083-multi-level-meta-space.md diff --git a/text/0083-multi-level-meta-space.md b/text/0083-multi-level-meta-space.md new file mode 100644 index 00000000..243b9979 --- /dev/null +++ b/text/0083-multi-level-meta-space.md @@ -0,0 +1,47 @@ +# Multi-level Meta Data Space + +## Summary + +PD can handle multiple users' meta data, each user only see its own meta data. + +## Motivation + +PD is the mata data center of TiKV Cluster, the meta data contains TiKV instances' addr, user data range(region) +information, the number of replicas of data, etc. + +But PD currently only can handle the key space [min-key, max-key], it means that PD only can handle one user's +meta data, it is not possible when multiple users want share the same PD cluster as their meta system. There are +two different scenarios that require PD can handle multiple user' meta data: + +1. Multiple TiKV Cluster share the same PD cluster. Because the minimal demplyment of a TiKV Cluster is 3 TiKV 3 PD, +but it is not cost-effect if every small cluster has 3 dedicated meta data node. +2. There are multiple tenants in the same TiKV Cluster, each tenant has it own meta data, each tenant has its own +meta data. For example, there are multiple TiDB(tenants), each tanent has its own `gc-safepoint`, `ddl-owner-key` and +other meta data. + +## Detailed design + +Change the meta data from +``` +{meta data} +``` +to +``` +user1 {meta data} +user2 {meta data} +user3 {meta data} +... +``` + +### Compatibility + +When upgrade from old version, all legacy meta data belongs to the default meta data space. +``` +{meta data} +``` +``` +default {meta data} +user1 {meta data} +user2 {meta data} +... +```