vault k8s integration

本文记录了 vault 与 kubernetes 集成实验

参考文档

实验环境

  • vault cluster version: v1.7.2
  • kubernetes cluster version: v1.21.1

kubernetes config

Below is an example configuration of a working k8s auth. In this below example:

  • vault-auth is the service account with which k8s auth is configured.
  • test-cloud is the service account used to login to vault using the k8s auth method
  • test is the namespace of the service account test-cloud
  • role-tokenreview-binding is the name of the cluster role binding the service accounts (vault-auth and test-cloud) need to be associated with

这里是准备了 2 个 service account, 一个 vault-auth 是专门用来给 vault 回调 call k8s 验证来自 k8s 的request 的真实性。另外一个 test-cloud 是我们服务运行时, 用这个 test-cloud 的 secrets 向 vault 发起请求。 role-tokenreview-binding 是一个权限增加配置, 运行两个 service account 去 call rbac.authorization.k8s.io 验证身份。
这里我的理解是,有点像 AWS 的 role, 互相可以 assume ,验证 role 的身份。如果想更简单,就一个 service account 也是可以的。
这里让我联想到了 AWS STS, 每次当 AWS call vault ,vault 都需要验证来访的真实性。

1
2
3
4
5
6
7
8
9
10
11
12
13
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1639532537842",
"Action": [
"sts:GetCallerIdentity"
],
"Effect": "Allow",
"Resource": "*"
}
]
}

以下命令,都在 kubernetes 那台机器上执行。 我已经把返回值加#注释

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
kubectl create namespace test

# namespace/test created

cat <<EOF | kubectl create -f -
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: test-cloud
namespace: test
---
apiVersion: v1
kind: Secret
metadata:
name: test-cloud
namespace: test
annotations:
kubernetes.io/service-account.name: test-cloud
type: kubernetes.io/service-account-token
EOF

# serviceaccount/test-cloud created
# secret/test-cloud created

cat <<EOF | kubectl create -f -
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: vault-auth
---
apiVersion: v1
kind: Secret
metadata:
name: vault-auth
annotations:
kubernetes.io/service-account.name: vault-auth
type: kubernetes.io/service-account-token
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: role-tokenreview-binding
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: system:auth-delegator
subjects:
- kind: ServiceAccount
name: vault-auth
namespace: default
- kind: ServiceAccount
name: test-cloud
namespace: test
EOF

# serviceaccount/vault-auth created
# secret/vault-auth created
# clusterrolebinding.rbac.authorization.k8s.io/role-tokenreview-binding created

接下来是得到一些 config 用来给 vault 设置 kubernetes auth integration

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
TOKEN_REVIEW_JWT=$(kubectl get secret vault-auth -o go-template='{{ .data.token }}' | base64 --decode) 
echo $TOKEN_REVIEW_JWT
eyJhbGciOiJSUzI1NiIsImtpZCI6IlZNU3F3QURqYThpVndNdXQ0Nmlsc19LanFDM1Z4NzRwVXNxSTY0X3M5VW8ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZWZhdWx0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InZhdWx0LWF1dGgiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoidmF1bHQtYXV0aCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjAxNWUzNjU4LWM4MjEtNDQzNi1iMTcwLWNmZTUwMGRmZDMwZCIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpkZWZhdWx0OnZhdWx0LWF1dGgifQ.R86ZHaSIYK4evpUhT9ayQp5XgJS-Lj_GIsIRT2n1M3Q8cFiO8A3HKx44VI0XToPHKpkyrRvzAX3n-GodR2L7AEcsesr1vQ-qlb2qreb07uh9ZxMo8t8r9bJ91eRaoV-xygW6bu3_dJupKDFL7peuCJSB16VYkPoSONQA_v1gZdXK7hmPe-JSrKtyz31FpsoI5mEhqxCwMTON_7Ew41S2wN089lMR--Rvn1ZtEaLRy1fpzEvJmwiUrN28byGrszZXMnmHTA26UUAHs59fKo-My_jM7EpV029aJGZEytX8czlndwiijPCRnhiuSb7oCJNI8yFLXyTr7LVbe6q7ip6Qqw
echo $TOKEN_REVIEW_JWT >> TOKEN_REVIEW_JWT

KUBE_CA_CERT=$(kubectl config view --raw --minify --flatten --output='jsonpath={.clusters[].cluster.certificate-authority-data}' | base64 --decode)
echo $KUBE_CA_CERT
-----BEGIN CERTIFICATE----- MIIC5zCCAc+gAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl cm5ldGVzMB4XDTIxMDUxMzAxMzAxOFoXDTMxMDUxMTAxMzAxOFowFTETMBEGA1UE AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKjq 3U72Z95kACf7ZbWYN2ktCxQVzcgJiVsD9UP2vm9c0hsI+eQWxodEf3+T6XVMCJce YgrYpLFyM1Mv7W/ZJ5F6UAGaPG4jOrF4CUgnYRV3j39wRjuvguOaf3lSMVFkAkOF GAN2eEO+OVaIeH7IxIbQ+vzC72V3WNTD0RC3OQoJIIQ2nLebcxkbksSTpDgARg+O j34IbHyUAUw3+msaCsKtqF+Tg6cLWGOu+iNekfL27rGJotXD3NCInE2aFX9Ipbmd BjnsvauwcCuBDKkSxxr46kcd7NZ2SFcvokkkuGl/c8AVdlL0TeDOrl8AOz/eUiQj 6ihQ7s07SUdNU3MefIkCAwEAAaNCMEAwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB /wQFMAMBAf8wHQYDVR0OBBYEFFIG5EyHEyWO9GE2Gm1D7nRY7iXFMA0GCSqGSIb3 DQEBCwUAA4IBAQCFcFHMgDAQ3GMfrLX0EudpyeDqJ/+Mj6C5xHet/bxQopMyCbmC ZVVXx2jpRaMSo81U/eDE0hfUSD3JGVUeM6aajivsGrGjqjuF0t/5hUxPJT74lIbc 5OQNXyYYuPeRyRw7Cnx8YpdJh4NJvWI+LBWSOSaDyFjf5yMpWQvT3bfWV4PGvi/0 YHxglWM3Jfpn7JBF6OJPs4P+6E0FPmS88d5ry6znmrAg9/PCOefZ1r93EAIwP1P4 EhWsyF9xemSyTIeDA740X7ZFcCEugkS6mOoswWtXPsL0JURlkx0kRR5HAYcQ01N1 Cv0epL7mgWJpxcwnil9pWXVDRgWTXvFP5aaD -----END CERTIFICATE-----
echo $KUBE_CA_CERT >> KUBE_CA_CERT

KUBE_HOST=$(kubectl config view --raw --minify --flatten --output='jsonpath={.clusters[].cluster.server}')
echo $KUBE_HOST
https://192.168.1.119:6443
echo $KUBE_HOST >> KUBE_HOST

kubectl get secret test-cloud -n test -o go-template='{{ .data.token }}' | base64 --decode >> test-cloud
cat test-cloud
eyJhbGciOiJSUzI1NiIsImtpZCI6IlZNU3F3QURqYThpVndNdXQ0Nmlsc19LanFDM1Z4NzRwVXNxSTY0X3M5VW8ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJ0ZXN0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InRlc3QtY2xvdWQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoidGVzdC1jbG91ZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjlkNDE0NjNkLWEyNTctNGRjNy1iZGQwLTVlYmRhODliOGE5NiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDp0ZXN0OnRlc3QtY2xvdWQifQ.MaPfB0w-rj9ymn1nr7Y1IJPfTM1-HpCjRhRF-qqJ8qFbREVeYgMwqg6FCYTL_bH9thLW4oNSFTw80IN8H9-0D-TVlA6UsRQGbp4v6tvb4YZ_rPJ7DRC5iaHproGPPJ0Nr-1cPLbfJVfWZwqvrrkz_pNmF23qM9WliGNGHgxV0n58KOD06lb03wk-IVBAyTKBwRTKgA3yub7SzoR0HeCS_4fvB4T73vGeNUBoaS2kIC1MF7Hlunmr0rzkGv7zs6HseFSXnGg8wZ1jUqOrP0AKuOXpE8eUXoqxd3-t0Y9zAj4pjZMCsJJZQlEVrKxAhuyeTvh_SYy5SCxjIKhpjT8j8A

vault config

这里我们需要把上面从 kubernetes 得到的 TOKEN_REVIEW_JWT KUBE_CA_CERT KUBE_HOST test-cloud 拷贝到 vault 这台机器上

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
vault auth enable kubernetes

# Success! Enabled kubernetes auth method at: kubernetes/

# 这一步很容易出错,因为 cert 的格式不对, 后面我会讲
vault write auth/kubernetes/config token_reviewer_jwt="$TOKEN_REVIEW_JWT" kubernetes_host="$KUBE_HOST" kubernetes_ca_cert="$KUBE_CA_CERT" disable_local_ca_jwt="true”
# Success! Data written to: auth/kubernetes/config

# 这里 kubernetes_ca_cert 格式不对
vault read auth/kubernetes/config
Key Value
--- -----
disable_iss_validation true
disable_local_ca_jwt true
issuer n/a
kubernetes_ca_cert -----BEGIN CERTIFICATE----- MIIC5zCCAc+gAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl cm5ldGVzMB4XDTIxMDUxMzAxMzAxOFoXDTMxMDUxMTAxMzAxOFowFTETMBEGA1UE AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKjq 3U72Z95kACf7ZbWYN2ktCxQVzcgJiVsD9UP2vm9c0hsI+eQWxodEf3+T6XVMCJce YgrYpLFyM1Mv7W/ZJ5F6UAGaPG4jOrF4CUgnYRV3j39wRjuvguOaf3lSMVFkAkOF GAN2eEO+OVaIeH7IxIbQ+vzC72V3WNTD0RC3OQoJIIQ2nLebcxkbksSTpDgARg+O j34IbHyUAUw3+msaCsKtqF+Tg6cLWGOu+iNekfL27rGJotXD3NCInE2aFX9Ipbmd BjnsvauwcCuBDKkSxxr46kcd7NZ2SFcvokkkuGl/c8AVdlL0TeDOrl8AOz/eUiQj 6ihQ7s07SUdNU3MefIkCAwEAAaNCMEAwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB /wQFMAMBAf8wHQYDVR0OBBYEFFIG5EyHEyWO9GE2Gm1D7nRY7iXFMA0GCSqGSIb3 DQEBCwUAA4IBAQCFcFHMgDAQ3GMfrLX0EudpyeDqJ/+Mj6C5xHet/bxQopMyCbmC ZVVXx2jpRaMSo81U/eDE0hfUSD3JGVUeM6aajivsGrGjqjuF0t/5hUxPJT74lIbc 5OQNXyYYuPeRyRw7Cnx8YpdJh4NJvWI+LBWSOSaDyFjf5yMpWQvT3bfWV4PGvi/0 YHxglWM3Jfpn7JBF6OJPs4P+6E0FPmS88d5ry6znmrAg9/PCOefZ1r93EAIwP1P4 EhWsyF9xemSyTIeDA740X7ZFcCEugkS6mOoswWtXPsL0JURlkx0kRR5HAYcQ01N1 Cv0epL7mgWJpxcwnil9pWXVDRgWTXvFP5aaD -----END CERTIFICATE-----
kubernetes_host https://192.168.1.119:6443
pem_keys []

vault policy write devwebapp - <<EOF
path "secret/data/devwebapp/config" {
capabilities = ["read”]
}
EOF

# Success! Uploaded policy: devwebapp

vault write auth/kubernetes/role/devweb-app \
bound_service_account_names=test-cloud \
bound_service_account_namespaces=test \
policies=devwebapp \
ttl=24h
# Success! Data written to: auth/kubernetes/role/devweb-app

最后就是测试 login 了, 在 vault 那台机器上

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
root@ubuntu18-108:~# curl     --request POST   \
--data '{"jwt": "eyJhbGciOiJSUzI1NiIsImtpZCI6IlZNU3F3QURqYThpVndNdXQ0Nmlsc19LanFDM1Z4NzRwVXNxSTY0X3M5VW8ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJ0ZXN0Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZWNyZXQubmFtZSI6InRlc3QtY2xvdWQiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoidGVzdC1jbG91ZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjlkNDE0NjNkLWEyNTctNGRjNy1iZGQwLTVlYmRhODliOGE5NiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDp0ZXN0OnRlc3QtY2xvdWQifQ.MaPfB0w-rj9ymn1nr7Y1IJPfTM1-HpCjRhRF-qqJ8qFbREVeYgMwqg6FCYTL_bH9thLW4oNSFTw80IN8H9-0D-TVlA6UsRQGbp4v6tvb4YZ_rPJ7DRC5iaHproGPPJ0Nr-1cPLbfJVfWZwqvrrkz_pNmF23qM9WliGNGHgxV0n58KOD06lb03wk-IVBAyTKBwRTKgA3yub7SzoR0HeCS_4fvB4T73vGeNUBoaS2kIC1MF7Hlunmr0rzkGv7zs6HseFSXnGg8wZ1jUqOrP0AKuOXpE8eUXoqxd3-t0Y9zAj4pjZMCsJJZQlEVrKxAhuyeTvh_SYy5SCxjIKhpjT8j8A", "role": "devweb-app"}' \
http://192.168.64.3:8200/v1/auth/kubernetes/login |jq

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1574 100 653 100 921 19787 27909 --:--:-- --:--:-- --:--:-- 47696
{
"request_id": "2b9978ef-4010-cb5a-38b5-01fdfb4ea5d5",
"lease_id": "",
"renewable": false,
"lease_duration": 0,
"data": null,
"wrap_info": null,
"warnings": null,
"auth": {
"client_token": "s.ElCkHSzi20xOlZj0CfoDuGyv",
"accessor": "mCzG5MoxhCUSVdSNRNfEhdBS",
"policies": [
"default",
"devwebapp"
],
"token_policies": [
"default",
"devwebapp"
],
"metadata": {
"role": "devweb-app",
"service_account_name": "test-cloud",
"service_account_namespace": "test",
"service_account_secret_name": "test-cloud",
"service_account_uid": "9d41463d-a257-4dc7-bdd0-5ebda89b8a96"
},
"lease_duration": 86400,
"renewable": true,
"entity_id": "7ab08e2f-f956-d2c6-c13e-3ec06e4a2cf3",
"token_type": "service",
"orphan": true
}
}

可以看到测试成功,结果返回了我们需要的 vault client_token ,kubernetes 用这个 token 去读取存在 vault 里面的 credentials
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
root@ubuntu18-108:~# curl --header "X-Vault-Token: s.ElCkHSzi20xOlZj0CfoDuGyv" http://192.168.64.3:8200/v1/secret/devwebapp/config | jq

% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 205 100 205 0 0 200k 0 --:--:-- --:--:-- --:--:-- 200k
{
"request_id": "c465525a-4af5-50d2-6a66-67ffb8a48629",
"lease_id": "",
"renewable": false,
"lease_duration": 2764800,
"data": {
"password": "salsa",
"username": "giraffe"
},
"wrap_info": null,
"warnings": null,
"auth": null
}

踩过的坑

  • cert 格式不对
    这个时候, login 返回error 是 {“errors”:[“permission denied”]},实际上可能是证书格式不对

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    13
    14
    15
    16
    17
    18
    19
    20
    21
    22
    23
    24
    25
    26
    27
    28
    29
    30
    31
    32
    33
    34
    35
    36
    37
    38
    39
    40
    41
    42
    43
    44
    45
    46
    47
    48
    49
    50
    51
    52
    53
    54
    55
    56
    57
    58
    59
    60
    vault read auth/kubernetes/config

    Key Value
    --- -----
    disable_iss_validation true
    disable_local_ca_jwt true
    issuer n/a
    kubernetes_ca_cert -----BEGIN CERTIFICATE----- MIIC5zCCAc+gAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl cm5ldGVzMB4XDTIxMDUxMzAxMzAxOFoXDTMxMDUxMTAxMzAxOFowFTETMBEGA1UE AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKjq 3U72Z95kACf7ZbWYN2ktCxQVzcgJiVsD9UP2vm9c0hsI+eQWxodEf3+T6XVMCJce YgrYpLFyM1Mv7W/ZJ5F6UAGaPG4jOrF4CUgnYRV3j39wRjuvguOaf3lSMVFkAkOF GAN2eEO+OVaIeH7IxIbQ+vzC72V3WNTD0RC3OQoJIIQ2nLebcxkbksSTpDgARg+O j34IbHyUAUw3+msaCsKtqF+Tg6cLWGOu+iNekfL27rGJotXD3NCInE2aFX9Ipbmd BjnsvauwcCuBDKkSxxr46kcd7NZ2SFcvokkkuGl/c8AVdlL0TeDOrl8AOz/eUiQj 6ihQ7s07SUdNU3MefIkCAwEAAaNCMEAwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB /wQFMAMBAf8wHQYDVR0OBBYEFFIG5EyHEyWO9GE2Gm1D7nRY7iXFMA0GCSqGSIb3 DQEBCwUAA4IBAQCFcFHMgDAQ3GMfrLX0EudpyeDqJ/+Mj6C5xHet/bxQopMyCbmC ZVVXx2jpRaMSo81U/eDE0hfUSD3JGVUeM6aajivsGrGjqjuF0t/5hUxPJT74lIbc 5OQNXyYYuPeRyRw7Cnx8YpdJh4NJvWI+LBWSOSaDyFjf5yMpWQvT3bfWV4PGvi/0 YHxglWM3Jfpn7JBF6OJPs4P+6E0FPmS88d5ry6znmrAg9/PCOefZ1r93EAIwP1P4 EhWsyF9xemSyTIeDA740X7ZFcCEugkS6mOoswWtXPsL0JURlkx0kRR5HAYcQ01N1 Cv0epL7mgWJpxcwnil9pWXVDRgWTXvFP5aaD -----END CERTIFICATE-----
    kubernetes_host https://192.168.1.119:6443
    pem_keys []

    # 证书应该是多行
    kubectl config view --raw --minify codeatten --output='jsonpath={.clusters[].cluster.certificate-authority-data}' | base64 --dec
    -----BEGIN CERTIFICATE-----
    MIIC5zCCAc+gAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
    cm5ldGVzMB4XDTIxMDUxMzAxMzAxOFoXDTMxMDUxMTAxMzAxOFowFTETMBEGA1UE
    AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKjq
    3U72Z95kACf7ZbWYN2ktCxQVzcgJiVsD9UP2vm9c0hsI+eQWxodEf3+T6XVMCJce
    YgrYpLFyM1Mv7W/ZJ5F6UAGaPG4jOrF4CUgnYRV3j39wRjuvguOaf3lSMVFkAkOF
    GAN2eEO+OVaIeH7IxIbQ+vzC72V3WNTD0RC3OQoJIIQ2nLebcxkbksSTpDgARg+O
    j34IbHyUAUw3+msaCsKtqF+Tg6cLWGOu+iNekfL27rGJotXD3NCInE2aFX9Ipbmd
    BjnsvauwcCuBDKkSxxr46kcd7NZ2SFcvokkkuGl/c8AVdlL0TeDOrl8AOz/eUiQj
    6ihQ7s07SUdNU3MefIkCAwEAAaNCMEAwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB
    /wQFMAMBAf8wHQYDVR0OBBYEFFIG5EyHEyWO9GE2Gm1D7nRY7iXFMA0GCSqGSIb3
    DQEBCwUAA4IBAQCFcFHMgDAQ3GMfrLX0EudpyeDqJ/+Mj6C5xHet/bxQopMyCbmC
    ZVVXx2jpRaMSo81U/eDE0hfUSD3JGVUeM6aajivsGrGjqjuF0t/5hUxPJT74lIbc
    5OQNXyYYuPeRyRw7Cnx8YpdJh4NJvWI+LBWSOSaDyFjf5yMpWQvT3bfWV4PGvi/0
    YHxglWM3Jfpn7JBF6OJPs4P+6E0FPmS88d5ry6znmrAg9/PCOefZ1r93EAIwP1P4
    EhWsyF9xemSyTIeDA740X7ZFcCEugkS6mOoswWtXPsL0JURlkx0kRR5HAYcQ01N1
    Cv0epL7mgWJpxcwnil9pWXVDRgWTXvFP5aaD
    -----END CERTIFICATE-----

    # 重新配置后
    vault read auth/kubernetes/config

    Key Value
    --- -----
    disable_iss_validation true
    disable_local_ca_jwt true
    issuer n/a
    kubernetes_ca_cert -----BEGIN CERTIFICATE-----
    MIIC5zCCAc+gAwIBAgIBADANBgkqhkiG9w0BAQsFADAVMRMwEQYDVQQDEwprdWJl
    cm5ldGVzMB4XDTIxMDUxMzAxMzAxOFoXDTMxMDUxMTAxMzAxOFowFTETMBEGA1UE
    AxMKa3ViZXJuZXRlczCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKjq
    3U72Z95kACf7ZbWYN2ktCxQVzcgJiVsD9UP2vm9c0hsI+eQWxodEf3+T6XVMCJce
    YgrYpLFyM1Mv7W/ZJ5F6UAGaPG4jOrF4CUgnYRV3j39wRjuvguOaf3lSMVFkAkOF
    GAN2eEO+OVaIeH7IxIbQ+vzC72V3WNTD0RC3OQoJIIQ2nLebcxkbksSTpDgARg+O
    j34IbHyUAUw3+msaCsKtqF+Tg6cLWGOu+iNekfL27rGJotXD3NCInE2aFX9Ipbmd
    BjnsvauwcCuBDKkSxxr46kcd7NZ2SFcvokkkuGl/c8AVdlL0TeDOrl8AOz/eUiQj
    6ihQ7s07SUdNU3MefIkCAwEAAaNCMEAwDgYDVR0PAQH/BAQDAgKkMA8GA1UdEwEB
    /wQFMAMBAf8wHQYDVR0OBBYEFFIG5EyHEyWO9GE2Gm1D7nRY7iXFMA0GCSqGSIb3
    DQEBCwUAA4IBAQCFcFHMgDAQ3GMfrLX0EudpyeDqJ/+Mj6C5xHet/bxQopMyCbmC
    ZVVXx2jpRaMSo81U/eDE0hfUSD3JGVUeM6aajivsGrGjqjuF0t/5hUxPJT74lIbc
    5OQNXyYYuPeRyRw7Cnx8YpdJh4NJvWI+LBWSOSaDyFjf5yMpWQvT3bfWV4PGvi/0
    YHxglWM3Jfpn7JBF6OJPs4P+6E0FPmS88d5ry6znmrAg9/PCOefZ1r93EAIwP1P4
    EhWsyF9xemSyTIeDA740X7ZFcCEugkS6mOoswWtXPsL0JURlkx0kRR5HAYcQ01N1
    Cv0epL7mgWJpxcwnil9pWXVDRgWTXvFP5aaD
    -----END CERTIFICATE-----
    kubernetes_host https://192.168.1.119:6443
    pem_keys []
  • kubernetes 里面的 service account 缺少权限, 导致 {“errors”:[“permission denied”]}

    1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
    12
    apiVersion: rbac.authorization.k8s.io/v1
    kind: ClusterRoleBinding
    metadata:
    name: role-tokenreview-binding
    roleRef:
    apiGroup: rbac.authorization.k8s.io
    kind: ClusterRole
    name: system:auth-delegator
    subjects:
    - kind: ServiceAccount
    name: vault-auth
    namespace: default
  • {“errors”:[“claim \”iss\” is invalid”]} 参考问题
    这个是因为没有配置 issuer, 如果你的 kubernetes 高于 1.21.1 你需要设置 issuer 在 vault auth with kubernetes 的时候, 如何得到 issuer 地址,请参考这个
    Discovering the service account issuer