博客oceanbase on k8s 部署篇
oceanbase on k8s 部署篇-c7电子娱乐
一、ob-operator简介
ob-operator是满足 kubernetes operator 扩展范式的自动化工具，可以极大简化在 kubernetes 上部署和管理 oceanbase 集群及相关资源的过程。
ob-operator 支持 oceanbase 集群的管理、租户管理、备份恢复、故障恢复等功能，具体如下：
- 集群管理：集群自举、调整集群拓扑、支持 k8s 拓扑配置、扩缩容、集群升级、修改参数
- 租户管理：创建租户、调整租户拓扑、管理资源单元、修改用户密码
- 备份恢复：向 oss 或 nfs 目的地周期性备份数据、从 oss 或 nfs 中恢复数据
- 物理备库：从备份中恢复出备租户、创建空备租户、备租户升主、主备切换
- 故障恢复：单节点故障恢复，ip 保持情况下的集群故障恢复
- dashboard(gui)：基于 ob-operator 的图形化 oceanbase 集群管理工具
二、部署oceanbase前置条件
在开始之前，请确保您已满足以下要求：
- kubernetes版本：>1.18版本
- oceanbase版本：根据需求选择4.2.5/4.3.3
- kubernetes 资源：基于local-path-provisioner 来管理 pvc，并且nodes上创建目录生成的待绑定的local pv。
- ob-operator 依赖 cert-manager，请确保您已安装 cert-manager。cert-manager 的安装方法请参考对应的 安装文档。
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.1/cert-manager.yaml
namespace/cert-manager created
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
.................
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
serviceaccount/cert-manager-cainjector created
serviceaccount/cert-manager created
serviceaccount/cert-manager-webhook created
clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created
..................
rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:dynamic-serving created
service/cert-manager-cainjector created
service/cert-manager created
service/cert-manager-webhook created
deployment.apps/cert-manager-cainjector created
deployment.apps/cert-manager created
deployment.apps/cert-manager-webhook created
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created三、部署 ob-operator
下面以部署version=2.2.0版本的ob-operator，operator最新应该已经到2.3.0版本(建议使用该版本)
/opt/ob-operator# helm repo add ob-operator https://oceanbase.github.io/ob-operator/
warning: kubernetes configuration file is group-readable. this is insecure. location: /root/.kube/config
warning: kubernetes configuration file is world-readable. this is insecure. location: /root/.kube/config
"ob-operator" has been added to your repositories
/opt/ob-operator# helm install ob-operator ob-operator/ob-operator --namespace=oceanbase-system --create-namespace --version=2.2.0
warning: kubernetes configuration file is group-readable. this is insecure. location: /root/.kube/config
warning: kubernetes configuration file is world-readable. this is insecure. location: /root/.kube/config
name: ob-operator
last deployed: mon oct 28 10:53:12 2024
namespace: oceanbase-system
status: deployed
revision: 1
test suite: none部署过程依赖镜像，需要提前下载到自己公司的镜像中心，否则报错镜像拉取失败报错：
events:
type reason age from message
---- ------ ---- ---- -------
warning failed 31m (x60 over 165m) kubelet (combined from similar events): failed to pull image "oceanbase/ob-operator:2.2.0": rpc error: code = unknown desc = error response from daemon: get https://registry-1.docker.io/v2/: dial tcp 199.16.158.190:443: connect: connection timed out
normal pulling 26m (x33 over 171m) kubelet pulling image "oceanbase/ob-operator:2.2.0"
normal backoff 91s (x728 over 171m) kubelet back-off pulling image "oceanbase/ob-operator:2.2.0"查看ob-operator安装是否ok（ready：2/2且status：running）
/opt/ob-operator# kubectl get pods -n oceanbase-system
name ready status restarts age
oceanbase-controller-manager-85f4b9d58c-cl9rq 2/2 running 0 3m27s四、部署 oceanbase 集群
按照以下步骤在 kubernetes 集群上部署 oceanbase 集群：
（1）新建namespace：kubectl create ns oceanbase-test
（2）创建 secrets 在创建 oceanbase 集群之前，您必须创建特定用户所需的 secrets。创建的目的就是ob集群初始化密码，使用以下命令创建 secrets：
kubectl create secret generic ob-root-password --from-literal=password='yourpassword' -n oceanbase-test
kubectl create secret generic ob-proxyro-password --from-literal=password='yourpassword' -n oceanbase-testps：注意在创建ob-proxyro-password时建议不要带特殊字符，否则可能出现部署obproxy时因权限链接不上后端observer的问题
（3）编辑集群配置文件oceanbase-test.yaml
apiversion: oceanbase.oceanbase.com/v1alpha1
kind: obcluster
metadata:
name: obcluste-test #ob的kubectl get obcluster名
namespace: oceanbase-test
spec:
clustername: obcluster-test #ob集群名
clusterid: 1
usersecrets:
root: ob-root-password
proxyro: ob-proxyro-password
topology:
- zone: zone1 #可以自定义zone名称
replica: 1 #一个zone的observer数量
# nodeselector:
# k1: v1
# affinity: 亲和性设置，生产环境必备
# nodeaffinity:
# podaffinity:
# podantiaffinity:
# tolerations: 污点设定，生产环境必备
# - key: "obtopo"
# value: "zone"
# effect: "noschedule"
- zone: zone2
replica: 1
# nodeselector:
# k1: v1
# affinity:
# nodeaffinity:
# podaffinity:
# podantiaffinity:
# tolerations:
# - key: "obtopo"
# value: "zone"
# effect: "noschedule"
- zone: zone3
replica: 1
# nodeselector:
# k1: v1
# affinity:
# nodeaffinity:
# podaffinity:
# podantiaffinity:
# tolerations:
# - key: "obtopo"
# value: "zone"
# effect: "noschedule"
observer:
image: oceanbase/oceanbase-cloud-native:4.3.3.0-100000142024101215
resource:
cpu: 12
memory: 40gi
storage:
datastorage:
storageclass: local-ssd-storage
size: 500gi
redologstorage:
storageclass: local-ssd-storage
size: 150gi
logstorage:
storageclass: local-ssd-storage
size: 20gi
monitor:
image: oceanbase/obagent:4.2.1-100000092023101717
resource:
cpu: 8
memory: 10gi（4）应用yaml，需要等待oceanbase集群创建，大概2分钟
kubectl apply -f oceanbase-test.yaml -n oceanbase-test注：如果需要镜像问题记得下载镜像到自己公司仓库
oceanbase/oceanbase-cloud-native:4.3.3.0-100000142024101215
oceanbase/obagent:4.2.1-100000092023101717（5）查看ob集群
/opt/ob-operator# kubectl get obcluster -n oceanbase-test1
name status age
obcluster running 3d2h
/opt/ob-operator# kubectl get pods -n oceanbase-test1 -o wide
name ready status restarts age ip node nominated node readiness gates
obcluster-test-1-zone1-6a036c0d84bf 2/2 running 0 3d2h 10.xxx.xxx.143 db02
obcluster-test-1-zone2-384f9bf55992 2/2 running 0 3d2h 10.xxx.xxx.142 db08
obcluster-test-1-zone3-6f375db39867 2/2 running 0 3d2h 10.xxx.xxx.242 db07
obproxy-test-6479ccb497-6r7xv 1/1 running 0 2d6h 10.xxx.xxx.182 kube610
obproxy-test-6479ccb497-bnj48 1/1 running 0 2d6h 10.xxx.xxx.227 kube600 （6）访问集群，因为没有部署obproxy，所以先直通过ob pods ip访问。
/opt/ob-operator# mysql -h10.xxx.xxx.143 -p2881 -uroot@sys -p oceanbase -a -c enter password: welcome to the mysql monitor. commands end with ; or \g. your mysql connection id is 3221489701 server version: 5.7.25 oceanbase_ce 4.3.3.0 (r100000142024101215-0723f9774e5f77a1253149261ba75a94414625d0) (built oct 12 2024 16:04:08) c7电子娱乐 copyright (c) 2009-2023 percona llc and/or its affiliates c7电子娱乐 copyright (c) 2000, 2023, oracle and/or its affiliates. oracle is a registered trademark of oracle corporation and/or its affiliates. other names may be trademarks of their respective owners. type 'help;' or '\h' for help. type '\c' to clear the current input statement.mysql> show databases; -------------------- | database | -------------------- | information_schema | | lbacsys | | mysql | | oceanbase | | oraauditor | | sys | | sys_external_tbs | | test | -------------------- 8 rows in set (0.01 sec) mysql> exit bye
五、总结
ob-operator 是基于 kubernetes operator 框架构建的工具，实现了声明式管理、自动化调谐和统一管理等优势，它提供简单可靠的方式实现 oceanbase 集群的容器化部署，简化oceanbase集群运维。
如果想更深入的了解ob-operator，可以参考以下资料
ob-operator文档：https://www.oceanbase.com/docs/community-ob-operator-doc-1000000000659335
ob-operator github文档：https://oceanbase.github.io/ob-operator/
集群yaml的详细配置：https://www.oceanbase.com/docs/community-ob-operator-doc-1000000000659352
点赞5
收藏
