Sep 9, 2019
Pod状态异常，挂载configmap显示no such file or directory $ kubectl get pod -o wide test-7695dc7fb9-rjxlq 0/1 CrashLoopBackOff 13 17h 192.168.100.20 node1 <none> CrashLoopBackOff状态需要查看pod的日志，通过kubectl logs可以查看到具体的错误信息。
$ kubectl logs -n test-7695dc7fb9-rjxlq container_linux.go:345: starting container process caused "process_linux.go:430: container init caused \"rootfs_linux.go:58: mounting \\\"/var/lib/kubelet/pods/ed1b799a-d3a5-11e9-add6-0894ef725f6e/volume-subpaths/test-cm/app-conf/1\\\" to rootfs \\\"/var/lib/docker/overlay/23a69e27f79f84a14b50fbee1e4840836487f8c740423291c2f785da1e7a6820/merged\\\" at \\\"/var/lib/docker/overlay/23a69e27f79f84a14b50fbee1e4840836487f8c740423291c2f785da1e7a6820/merged/etc/app/app.conf\\\" caused \\\"no such file or directory\\\"\"" 查看日志发现是挂载的configmap有问题。
猜测原因：该pod采用subPath的方式挂载configmap，怀疑是configmap修改后，未重建pod导致，因此会出现上述no such file or directory的错误。
排查问题 查看异常pod的uid，通过下述的命令我们可以看到当前pod在2019-09-10T08:35:21Z的时间创建，并且uid为ed1b799a-d3a5-11e9-add6-0894ef725f6e。
$ kubectl get pod test-7695dc7fb9-rjxlq -o json | jq .metadata { "creationTimestamp": "2019-09-10T08:35:21Z", "generateName": "test-7695dc7fb9-", "labels": { "application": "test", "pod-template-hash": "3251873965" }, "name": "test-7695dc7fb9-rjxlq", "namespace": "openstack", "ownerReferences": [ { "apiVersion": "apps/v1", "blockOwnerDeletion": true, "controller": true, "kind": "ReplicaSet", "name": "test-7695dc7fb9", "uid": "ece58508-d3a5-11e9-add6-0894ef725f6e" } ], "resourceVersion": "1762165", "selfLink": "/api/v1/namespaces/default/pods/test-7695dc7fb9-rjxlq", "uid": "ed1b799a-d3a5-11e9-add6-0894ef725f6e" } 接下来就需要登录到pod所在的节点进行查看。
