feat: support tablet auth and flag to disable auth (#3885)

* add brpc authenticator

* fix: add auth library

* fix: make authenticator lifetime same as channel

* refactor: use global variables to pass auth tokens

* feat: added basic server auth backwards compatible with user = root and all internal services

* chore: remove debug log

* chore: apply lint style fixes

* feat: encrypt password before sending

* fix: add namespace for Authenticator class

* chore: add newline

* feat: add user verification logic

* feat: add nameserver server side auth

* chore: remove unneccessary logs

* chore: lint

* chore: make refreshable map header only

* chore: remove old refreshable map file

* chore: lint

* chore: lint

* chore: lint

* sync with db in user access manager constructor

* chore: add license

* chore: add license

* feat: default g_auth_token user to root if not provided

* fix: user table start in cluster mode

* feat: sync user data every 100ms

* fix: add auth library for linking

* fix: refreshable map test

* fix: add main method to refreshable map test

* fix: add auth to sdk

* eaat: treat omitted host as localhost

* feat: treat % as all hosts

* fix: minicluster auth

* fix: remove api server auth

* feat: add minicluster tablet auth

* fix: change scala test password

* fix: default root as user in python sdk

* fix: wait for nameservers to pick up auth data before starting api server

* fix: wait 5 seconds after nameserver launch for auth data to be picked up

* fix: minicluster options scope

* test: cat host file

* fix: remove cat

* fix: Init should finish after user table is available

* fix: always create user table

* fix: create user table after internal db create

* fix: add partition info setting for user table create

* fix: auth related tests

* fix: support multiple nameservers

* fix: recover db and table only for cluster mode

* fix: remote test ns tablet start order

* fix: set root as default user in python sdk

* fix: SyncTableReplicaCluster and AddAndRemoveReplicaCluster tests

* fix: remove user root

* fix: add erver stops to name_server_test cases and spilt sql availability tests

* fix: nameserver test scope

* test: add bad tablet server option

* fix: brpc authenticator destruction before nameserver

* feat: add user auth_str to tablet client

* feat: add tablet system table iterator

* fix: refactor to better system table iterator

* fix: remove unneccessary test

* test: sql_cmd_test should expect 2 user table replicas in cluster mode

* chore: lint

* feat: add skip auth flag

* fix: skip while wait in nameserver start if auth disabled

* test: enable auth for system table test

* fix: tablets read user from local

* fix: authenticator scope

* chore: add auth docs

* chore: more docs

* chore: add 0.8.5 clarification

---------

Co-authored-by: 张子恒 <zhangziheng00001@163.com>

Author: oh2024

docs/en/deploy/auth.md

@@ -12,7 +12,11 @@ OpenMLDB 0.8.4 and later versions support ZooKeeper with passwords. You need to
 
 ## OpenMLDB Authentication
 
-OpenMLDB 0.8.5 and later versions support username and password authentication in the cluster. Custom user and password creation is performed after the cluster is started. The initial cluster has only the root user with an empty password. By default, all servers and clients connect to the cluster using the root username and an empty password.
+### Alpha Feature in 0.9.0
+OpenMLDB introduces server-side authentication as an alpha feature in version 0.9.0. As of this release, server-side authentication is not enabled by default. This approach ensures backward compatibility and minimizes the risk of disrupting existing deployments. It can be enablled by including `--noskip_grant_tables` when starting clients and servers. Statements that alter the authentication settings or modify user credentials, such as `CREATE USER`, `ALTER USER`, and `DELETE USER` are not supported when authentication is disabled.
+
+### Authentication in 0.8.5
+OpenMLDB 0.8.5 supports username and password authentication in the cluster. Custom user and password creation is performed after the cluster is started. The initial cluster has only the root user with an empty password. By default, all servers and clients connect to the cluster using the root username and an empty password.
 
 For security reasons, we usually need the root user to use a password and then create ordinary users. Due to the architecture, some servers also connect to the cluster as clients. If you need to change the root password, pay attention to these servers.
 

docs/zh/deploy/auth.md

@@ -12,7 +12,11 @@ OpenMLDB 0.8.4版本及以后，可以使用有密码的ZooKeeper。需要在启
 
 ## OpenMLDB身份认证
 
-OpenMLDB 0.8.5版本及以后，集群内可以使用用户名和密码进行身份认证。自定义用户和密码的创建都在集群启动之后，初始集群只有root用户，密码为空。默认情况下，任何的server和client都是以root用户名和空密码连接到集群。
+### 0.9.0 中的 Alpha 功能
+OpenMLDB 在 0.9.0 版本中作为 alpha 功能引入了服务器端身份验证。 从此版本开始，默认情况下不启用服务器端身份验证。 这种方法可确保向后兼容性并最大限度地降低破坏现有部署的风险。 用户可以通过在启动客户端和服务器时配置“--noskip_grant_tables”来启用服务端认证。 禁用身份验证时，不支持更改身份验证设置或修改用户凭据的语句，例如“CREATE USER”、“ALTER USER”和“DELETE USER”。
+
+### 0.8.5 中的认证
+OpenMLDB 0.8.5版本集群内可以使用用户名和密码进行身份认证。自定义用户和密码的创建都在集群启动之后，初始集群只有root用户，密码为空。默认情况下，任何的server和client都是以root用户名和空密码连接到集群。
 
 为了安全，我们通常需要root用户使用密码，再创建普通用户。而由于架构上部分server也会作为client连接集群，如果修改root密码需要注意。
 

src/auth/refreshable_map.h

@@ -29,6 +29,8 @@ namespace openmldb::auth {
 template <typename Key, typename Value>
 class RefreshableMap {
  public:
+    RefreshableMap() : map_(std::make_shared<std::unordered_map<Key, Value>>()) {}
+
     std::optional<Value> Get(const Key& key) const {
         std::shared_lock<std::shared_mutex> lock(mutex_);
         if (auto it = map_->find(key); it != map_->end()) {

src/auth/user_access_manager.cc

@@ -23,9 +23,9 @@
 #include "nameserver/system_table.h"
 
 namespace openmldb::auth {
-UserAccessManager::UserAccessManager(IteratorFactory iterator_factory,
-                                     std::shared_ptr<nameserver::TableInfo> user_table_info)
-    : user_table_iterator_factory_(std::move(iterator_factory)), user_table_info_(user_table_info) {
+
+UserAccessManager::UserAccessManager(IteratorFactory iterator_factory)
+    : user_table_iterator_factory_(std::move(iterator_factory)) {
     StartSyncTask();
 }
 
@@ -49,26 +49,28 @@ void UserAccessManager::StopSyncTask() {
 }
 
 void UserAccessManager::SyncWithDB() {
-    auto new_user_map = std::make_unique<std::unordered_map<std::string, std::string>>();
-    auto it = user_table_iterator_factory_(::openmldb::nameserver::USER_INFO_NAME);
-    it->SeekToFirst();
-    while (it->Valid()) {
-        auto row = it->GetValue();
-        auto buf = it->GetValue().buf();
-        auto size = it->GetValue().size();
-        codec::RowView row_view(user_table_info_->column_desc(), buf, size);
-        std::string host, user, password;
-        row_view.GetStrValue(0, &host);
-        row_view.GetStrValue(1, &user);
-        row_view.GetStrValue(2, &password);
-        if (host == "%") {
-            new_user_map->emplace(user, password);
-        } else {
-            new_user_map->emplace(FormUserHost(user, host), password);
+    if (auto it_pair = user_table_iterator_factory_(::openmldb::nameserver::USER_INFO_NAME); it_pair) {
+        auto new_user_map = std::make_unique<std::unordered_map<std::string, std::string>>();
+        auto it = it_pair->first.get();
+        it->SeekToFirst();
+        while (it->Valid()) {
+            auto row = it->GetValue();
+            auto buf = it->GetValue().buf();
+            auto size = it->GetValue().size();
+            codec::RowView row_view(*it_pair->second.get(), buf, size);
+            std::string host, user, password;
+            row_view.GetStrValue(0, &host);
+            row_view.GetStrValue(1, &user);
+            row_view.GetStrValue(2, &password);
+            if (host == "%") {
+                new_user_map->emplace(user, password);
+            } else {
+                new_user_map->emplace(FormUserHost(user, host), password);
+            }
+            it->Next();
         }
-        it->Next();
+        user_map_.Refresh(std::move(new_user_map));
     }
-    user_map_.Refresh(std::move(new_user_map));
 }
 
 bool UserAccessManager::IsAuthenticated(const std::string& host, const std::string& user, const std::string& password) {

src/auth/user_access_manager.h

@@ -22,23 +22,25 @@
 #include <memory>
 #include <string>
 #include <thread>
+#include <utility>
 
 #include "catalog/distribute_iterator.h"
 #include "refreshable_map.h"
 
 namespace openmldb::auth {
 class UserAccessManager {
  public:
-    using IteratorFactory =
-        std::function<std::unique_ptr<::openmldb::catalog::FullTableIterator>(const std::string& table_name)>;
+    using IteratorFactory = std::function<std::optional<
+        std::pair<std::unique_ptr<::openmldb::catalog::FullTableIterator>, std::unique_ptr<openmldb::codec::Schema>>>(
+        const std::string& table_name)>;
+
+    explicit UserAccessManager(IteratorFactory iterator_factory);
 
-    UserAccessManager(IteratorFactory iterator_factory, std::shared_ptr<nameserver::TableInfo> user_table_info);
     ~UserAccessManager();
     bool IsAuthenticated(const std::string& host, const std::string& username, const std::string& password);
 
  private:
     IteratorFactory user_table_iterator_factory_;
-    std::shared_ptr<nameserver::TableInfo> user_table_info_;
     RefreshableMap<std::string, std::string> user_map_;
     std::atomic<bool> sync_task_running_{false};
     std::thread sync_task_thread_;

src/catalog/client_manager.cc

@@ -554,7 +554,7 @@ bool ClientManager::UpdateClient(const std::map<std::string, std::string>& endpo
     for (const auto& kv : endpoint_map) {
         auto it = real_endpoint_map_.find(kv.first);
         if (it == real_endpoint_map_.end()) {
-            auto wrapper = std::make_shared<TabletAccessor>(kv.first);
+            auto wrapper = std::make_shared<TabletAccessor>(kv.first, auth_token_);
             if (!wrapper->UpdateClient(kv.second)) {
                 LOG(WARNING) << "add client failed. name " << kv.first << ", endpoint " << kv.second;
                 continue;
@@ -583,7 +583,7 @@ bool ClientManager::UpdateClient(
     for (const auto& kv : tablet_clients) {
         auto it = real_endpoint_map_.find(kv.first);
         if (it == real_endpoint_map_.end()) {
-            auto wrapper = std::make_shared<TabletAccessor>(kv.first, kv.second);
+            auto wrapper = std::make_shared<TabletAccessor>(kv.first, kv.second, auth_token_);
             DLOG(INFO) << "add client. name " << kv.first << ", endpoint " << kv.second->GetRealEndpoint();
             clients_.emplace(kv.first, wrapper);
             real_endpoint_map_.emplace(kv.first, kv.second->GetRealEndpoint());

src/catalog/client_manager.h

@@ -132,17 +132,20 @@ class AsyncTablesHandler : public ::hybridse::vm::MemTableHandler {
 
 class TabletAccessor : public ::hybridse::vm::Tablet {
  public:
-    explicit TabletAccessor(const std::string& name) : name_(name), tablet_client_() {}
+    explicit TabletAccessor(const std::string& name,
+                            const openmldb::authn::AuthToken auth_token = openmldb::authn::ServiceToken{"default"})
+        : name_(name), tablet_client_(), auth_token_(auth_token) {}
 
-    TabletAccessor(const std::string& name, const std::shared_ptr<::openmldb::client::TabletClient>& client)
-        : name_(name), tablet_client_(client) {}
+    TabletAccessor(const std::string& name, const std::shared_ptr<::openmldb::client::TabletClient>& client,
+                   const openmldb::authn::AuthToken auth_token = openmldb::authn::ServiceToken{"default"})
+        : name_(name), tablet_client_(client), auth_token_(auth_token) {}
 
     std::shared_ptr<::openmldb::client::TabletClient> GetClient() {
         return std::atomic_load_explicit(&tablet_client_, std::memory_order_relaxed);
     }
 
     bool UpdateClient(const std::string& endpoint) {
-        auto client = std::make_shared<::openmldb::client::TabletClient>(name_, endpoint);
+        auto client = std::make_shared<::openmldb::client::TabletClient>(name_, endpoint, auth_token_);
         if (client->Init() != 0) {
             return false;
         }
@@ -170,6 +173,7 @@ class TabletAccessor : public ::hybridse::vm::Tablet {
  private:
     std::string name_;
     std::shared_ptr<::openmldb::client::TabletClient> tablet_client_;
+    const openmldb::authn::AuthToken auth_token_;
 };
 
 class TabletsAccessor : public ::hybridse::vm::Tablet {
@@ -243,7 +247,8 @@ class TableClientManager {
 
 class ClientManager {
  public:
-    ClientManager() : real_endpoint_map_(), clients_(), mu_(), rand_(0xdeadbeef) {}
+    explicit ClientManager(const openmldb::authn::AuthToken auth_token = openmldb::authn::ServiceToken{"default"})
+        : real_endpoint_map_(), clients_(), mu_(), rand_(0xdeadbeef), auth_token_(auth_token) {}
     std::shared_ptr<TabletAccessor> GetTablet(const std::string& name) const;
     std::shared_ptr<TabletAccessor> GetTablet() const;
     std::vector<std::shared_ptr<TabletAccessor>> GetAllTablet() const;
@@ -257,6 +262,7 @@ class ClientManager {
     std::unordered_map<std::string, std::shared_ptr<TabletAccessor>> clients_;
     mutable ::openmldb::base::SpinMutex mu_;
     mutable ::openmldb::base::Random rand_;
+    const openmldb::authn::AuthToken auth_token_;
 };
 
 }  // namespace catalog

src/client/tablet_client.cc

@@ -35,11 +35,14 @@ DECLARE_uint32(absolute_ttl_max);
 namespace openmldb {
 namespace client {
 
-TabletClient::TabletClient(const std::string& endpoint, const std::string& real_endpoint)
-    : Client(endpoint, real_endpoint), client_(real_endpoint.empty() ? endpoint : real_endpoint) {}
-
-TabletClient::TabletClient(const std::string& endpoint, const std::string& real_endpoint, bool use_sleep_policy)
-    : Client(endpoint, real_endpoint), client_(real_endpoint.empty() ? endpoint : real_endpoint, use_sleep_policy) {}
+TabletClient::TabletClient(const std::string& endpoint, const std::string& real_endpoint,
+                           const openmldb::authn::AuthToken auth_token)
+    : Client(endpoint, real_endpoint), client_(real_endpoint.empty() ? endpoint : real_endpoint, auth_token) {}
+
+TabletClient::TabletClient(const std::string& endpoint, const std::string& real_endpoint, bool use_sleep_policy,
+                           const openmldb::authn::AuthToken auth_token)
+    : Client(endpoint, real_endpoint),
+      client_(real_endpoint.empty() ? endpoint : real_endpoint, use_sleep_policy, auth_token) {}
 
 TabletClient::~TabletClient() {}
 

src/client/tablet_client.h

@@ -46,9 +46,11 @@ const uint32_t INVALID_REMOTE_TID = UINT32_MAX;
 
 class TabletClient : public Client {
  public:
-    TabletClient(const std::string& endpoint, const std::string& real_endpoint);
+    TabletClient(const std::string& endpoint, const std::string& real_endpoint,
+                 const openmldb::authn::AuthToken auth_token = openmldb::authn::ServiceToken{"default"});
 
-    TabletClient(const std::string& endpoint, const std::string& real_endpoint, bool use_sleep_policy);
+    TabletClient(const std::string& endpoint, const std::string& real_endpoint, bool use_sleep_policy,
+                 const openmldb::authn::AuthToken auth_token = openmldb::authn::ServiceToken{"default"});
 
     ~TabletClient();
 

src/cmd/openmldb.cc

@@ -145,19 +145,19 @@ void StartNameServer() {
         PDLOG(WARNING, "Fail to register name");
         exit(1);
     }
-    std::shared_ptr<::openmldb::nameserver::TableInfo> table_info;
-    if (!name_server->GetTableInfo(::openmldb::nameserver::USER_INFO_NAME, ::openmldb::nameserver::INTERNAL_DB,
-                                   &table_info)) {
-        PDLOG(WARNING, "Failed to get table info for user table");
-        exit(1);
-    }
-    openmldb::auth::UserAccessManager user_access_manager(name_server->GetSystemTableIterator(), table_info);
+
     brpc::ServerOptions options;
-    openmldb::authn::BRPCAuthenticator server_authenticator(
-        [&user_access_manager](const std::string& host, const std::string& username, const std::string& password) {
-            return user_access_manager.IsAuthenticated(host, username, password);
-        });
-    options.auth = &server_authenticator;
+    std::unique_ptr<openmldb::auth::UserAccessManager> user_access_manager;
+    std::unique_ptr<openmldb::authn::BRPCAuthenticator> server_authenticator;
+    if (!FLAGS_skip_grant_tables) {
+        user_access_manager =
+            std::make_unique<openmldb::auth::UserAccessManager>(name_server->GetSystemTableIterator());
+        server_authenticator = std::make_unique<openmldb::authn::BRPCAuthenticator>(
+            [&user_access_manager](const std::string& host, const std::string& username, const std::string& password) {
+                return user_access_manager->IsAuthenticated(host, username, password);
+            });
+        options.auth = server_authenticator.get();
+    }
 
     options.num_threads = FLAGS_thread_pool_size;
     brpc::Server server;
@@ -256,8 +256,17 @@ void StartTablet() {
         exit(1);
     }
     brpc::ServerOptions options;
-    openmldb::authn::BRPCAuthenticator server_authenticator;
-    options.auth = &server_authenticator;
+    std::unique_ptr<openmldb::auth::UserAccessManager> user_access_manager;
+    std::unique_ptr<openmldb::authn::BRPCAuthenticator> server_authenticator;
+
+    if (!FLAGS_skip_grant_tables) {
+        user_access_manager = std::make_unique<openmldb::auth::UserAccessManager>(tablet->GetSystemTableIterator());
+        server_authenticator = std::make_unique<openmldb::authn::BRPCAuthenticator>(
+            [&user_access_manager](const std::string& host, const std::string& username, const std::string& password) {
+                return user_access_manager->IsAuthenticated(host, username, password);
+            });
+        options.auth = server_authenticator.get();
+    }
     options.num_threads = FLAGS_thread_pool_size;
     brpc::Server server;
     if (server.AddService(tablet, brpc::SERVER_DOESNT_OWN_SERVICE) != 0) {