面向 K8s 设计误区
Kirito的技术分享
共 14805字,需浏览 30分钟
·
2021-04-14 13:25
K8s 设计模式
自定义资源
Operator
通俗的理解
使用 SDK 创建一个新的 Operator 项目; 通过添加自定义资源(CRD)定义新的资源 API; 指定使用 SDK API 来 watch 的资源; 自定义 Controller 实现 K8s 协调(reconcile)逻辑;
有了锤子,看到的只有钉子
一切设计皆 YAML; 一切皆合一; 一切皆终态; 一切交互皆 cr。
误区1:一切设计皆 YAML
案例
先根据已知的数据初步整理一个大而全的 YAML,做一下初步的分类,例如应用大概包含基础信息,依赖服务,运维逻辑,监控采集等,每个分类做一个子部分。 开会讨论具体的内容是否能满足要求,结果每次开会都难以形成共识。
因为总是有新的需求满足不了,在讨论A时,就有人提到 B、C、D,不断有新的需求; 每个部分的属性非常难统一,因为不同的实现属性差异较大; 理解不一致,相同名字但使用时每个人的理解也不同;
由于工期很紧,只能临时妥协,做一个中间态,后面再进一步优化。 后续优化升级,相同的流程再来一遍,还是很难形成共识。
apiVersion: apps.mwops.alibaba-inc.com/v1alpha1
kind: AppDefinition
metadata:
labels:
app: "A"
name: A-1.0 //chart-name+chart-version
namespace: kubeone
spec:
appName: A //chart-name
version: 1.0 //chart-version
type: apps.mwops.alibaba-inc.com/v1alpha1.argo-helm
workloadSettings: //注 workloadSettings 标识type应该使用的属性
- name: "deployToK8SName"
value: ""
- name: "deployToNamespace"
value: ${resources:namespace-resource.name}
parameterValues: //注 parameterValues标识业务属性
- name: "enableTenant"
value: "1"
- name: "CPU"
value: "1"
- name: "MEM"
value: "2Gi"
- name: "jvm"
value: "flag;gc"
- name: vip.fileserver-edas.ip
value: ${resources:fileserver_edas.ip}
- name: DB_NAME
valueFromConfigMap:
name: ${resources:rds-resource.cm-name}
expr: ${database}
- name: DB_PASSWORD
valueFromSecret:
name: ${instancename}-rds-secret
expr: ${password}
- name: object-storage-endpoint
value: ${resources:object-storage.endpoint}
- name: object-storage-username
valueFromSecret:
name: ${resources:object-storage.secret-name}
expr: ${username}
- name: object-storage-password
valueFromSecret:
name: ${resources:object-storage.secret-name}
expr: ${password}
- name: redis-endpoint
value: ${resources:redis.endpoint}
- name: redis-password
value: ${resources:redis.password}
resources:
- name: tolerations
type: apps.mwops.alibaba-inc.com/tolerations
parameterValues:
- name: key
value: "sigma.ali/is-ecs"
- name: key
value: "sigma.ali/resource-pool"
- name: namespace-resource
type: apps.mwops.alibaba-inc.com/v1alpha1.namespace
parameterValues:
- name: name
value: edas
- name: fileserver-edas
type: apps.mwops.alibaba-inc.com/v1alpha1.database.vip
parameterValues:
- name: port
value: 21,80,8080,5000
- name: src_port
value: 21,80,8080,5000
- name: type
value: ClusterIP
- name: check_type
value: ""
- name: uri
value: ""
- name: ip
value: ""
- name: test-db
type: apps.mwops.alibaba-inc.com/v1alpha1.database.mysqlha
parameterValues:
- name: name
value: test-db
- name: user
value: test-user
- name: password
value: test-passwd
- name: secret
value: test-db-mysqlha-secret
- name: service-slb
type: apps.mwops.alibaba-inc.com/v1alpha1.slb
mode: post-create
parameterValues:
- name: service
value: "serviceA"
- name: annotations
value: "app:a,version:1.0"
- name: external-ip
value:
- name: service-resource2
type: apps.mwops.alibaba-inc.com/v1alpha1.service
parameterValues:
- name: second-domain
value: edas.console
- name: ports
value: "80:80"
- name: selectors
value: "app:a,version:1.0"
- name: type
value: "loadbalance"
- name: service-dns
type: apps.mwops.alibaba-inc.com/v1alpha1.dns
parameterValues:
- name: domain
value: edas.server.${global:domain}
- name: vip
value: ${resources:service-resource2.EXTERNAL-IP}
- name: dns-resource
type: apps.mwops.alibaba-inc.com/v1alpha1.dns
parameterValues:
- name: domain
value: edas.console.${global:domain}
- name: vip
value: “127.0.0.1”
- name: cni-resource
type: apps.mwops.alibaba-inc.com/v1alpha1.cni
parameterValues:
- name: count
value: 4
- name: ip_list
value:
- name: object-storage
type: apps.mwops.alibaba-inc.com/v1alpha1.objectStorage.minio
parameterValues:
- name: namespace
value: test-ns
- name: username
value: test-user
- name: password
value: test-password
- name: storage-capacity
value: 20Gi
- name: secret-name
value: minio-my-store-access-keys
- name: endpoint
value: minio-instance-external-service
- name: redis
type: apps.mwops.alibaba-inc.com/v1alpha1.database.redis
parameterValues:
- name: cpu
value: 500m
- name: memory
value: 128Mi
- name: password
value: i_am_a_password
- name: storage-capacity
value: 20Gi
- name: endpoint
value: redis-redis-cluster
- name: accesskey
type: apps.mwops.alibaba-inc.com/v1alpha1.accesskey
parameterValues:
- name: name
value: default
- name: userName
value: ecs_test@aliyun.com
exposes:
- name: dns
value: ${resources:dns-resource.domain}
- name: db-endpoint
valueFromConfigmap:
name: ${resources:rds-resource.cm-name}
expr: ${endpoint}:3306/${database}
- name: ip_list
value: ${resources:cni-resource.ip_list}
- name: object-storage-endpoint
value: ${resources:object-storage.endpoint}.${resource:namespace-resource.name}
- name: object-storage-username
valueFromSecret:
name: ${resources:object-storage.secret-name}
expr: ${username}
- name: object-storage-password
valueFromSecret:
name: ${resources:object-storage.secret-name}
expr: ${password}
- name: redis-endpoint
value: ${resources:redis.endpoint}.${resource:namespace-resource.name}
- name: redis-password
value: ${resources:redis.password}
反思
适用范围
1、适用场景
2、不适用场景
UML 用例图; 梳理用户故事; 基于用户故事对齐 Domain Object,确定关键的业务对象以及对象间关系; 需要 Operator 化的对象,每个对象描述为一个 CRD,当然 CRD 缺乏接口、继承等面向对象的能力,可以通过其他方式曲线表达; 不需要 Operator 化的对象,直接编写 Controller;
误区2:一切皆合一
案例
apiVersion: apps.mwops.alibaba-inc.com/v1alpha1
kind: AppDefinition
metadata:
labels:
app: "WordPress"
name: WordPress-1.0 //chart-name+chart-version
namespace: kubeone
spec:
appName: WordPress //chart-name
version: 1.0 //chart-version
type: apps.mwops.alibaba-inc.com/v1alpha1.argo-helm
parameterValues: //注 parameterValues标识业务属性
- name: "enableTenant"
value: "1"
- name: "CPU"
value: "1"
- name: "MEM"
value: "2Gi"
- name: "jvm"
value: "flag;gc"
- name: replicas
value: 3
- name: connectstring
valueFromConfigMap:
name: ${resources:test-db.exposes.connectstring}
expr: ${connectstring}
- name: db_user_name
valueFromSecret:
....
resources:
- name: test-db //创建一个新的DB
type: apps.mwops.alibaba-inc.com/v1alpha1.database.mysqlha
parameterValues:
- name: cpu
value: 2
- name: memory
value: 4G
- name: storage
value: 20Gi
- name: username
value: myusername
- name: password
value: i_am_a_password
- name: dbname
value: wordPress
exposes:
- name: connectstring
- name: username
- name: password
exposes:
- name: dns
value: ...
反思
apiVersion: apps.mwops.alibaba-inc.com/v1alpha1
kind: AppDefinition
metadata:
labels:
app: "WordPress"
name: WordPress-1.0 //chart-name+chart-version
namespace: kubeone
spec:
appName: WordPress //chart-name
version: 1.0 //chart-version
name: WordPress-test
type: apps.mwops.alibaba-inc.com/v1alpha1.argo-helm
parameterValues: //注 parameterValues标识业务属性
- ....
resources:
- name: test-db-secret
value: "wordPress1Secret" //引用已有的secret
exposes:
- name: dns
value: ...
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: wordPress-
spec:
templates:
- name: wordPress
steps:
# 创建db
- - name: wordpress-db
template: wordpress-db
arguments:
parameters: [{name: wordpress-db1}]
# 创建应用
- - name:
template: wordpress
arguments:
parameters: [{db-sercet: wordpress-db1}]
编排是一次性的配置,编排文件下发一次之后,后续操作都是操作单个对象,例如:变更时,只会单独变更 wordPress,或单独变更 wordPressDB,而不会一次性同时变更 2 个对象。 单独变更应用时,是否需要下发整个终态 YAML,这个要根据实际情况进行设计,值得大家思考。后面会提出针对整个应用生命周期状态机的设计,里面有详细的解释。
适用范围
1、适用场景
2、不适用场景
误区3:一切皆终态
案例
反思
1、命令式编程
第一步,创建一个存储结果的集合变量 results;
第二步,遍历这个数字集合 collection;
第三步,一个一个地判断每个数字是不是大于 5,如果是就将这个数字添加到结果集合变量 results 中。
List results = new List();
foreach(var num in collection)
{
if (num > 5)
results.Add(num);
}
2、声明式编程
命令式编程:命令“机器”如何去做事情(how),这样不管你想要的是什么(what),它都会按照你的命令实现。
声明式编程:告诉“机器”你想要的是什么(what),让机器想出如何去做(how)。
适用范围
1、适用场景
2、不适用场景
误区4:一切交互皆 cr
案例
调用一个 http 接口或 function,需要下发一个 cr;
应用 crud 都下发完整 cr;
反思
案例1:是否所有的逻辑都需要下发一个 cr?
通过 API 传入 cr,cr 保存到 etcd;
触发 informer;
controller 接收到对应的事件,触发逻辑;
更新 cr 状态;
清理 cr,否则会占用 etcd 存储;
案例2:
适用范围
1、适用场景
2、不适用场景
高频的服务调用,无需持久化的数据。
复杂状态机的驱动。
总结
揭秘 Kubernetes Operator: http://www.dockone.io/article/8769 声明式编程和命令式编程有什么区别 : https://www.zhihu.com/question/22285830 如何在 Kubernetes 中编写自定义控制器: https://www.sohu.com/a/363619791_198222
- END -
「技术分享」某种程度上,是让作者和读者,不那么孤独的东西。欢迎关注我的微信公众号:「Kirito的技术分享」
评论