BoundServiceAccountTokenVolume refresh token with EKS 1.21

[amalendur]

Describe the bug

Hi,

Kubernetes version 1.21 graduated BoundServiceAccountTokenVolume feature [1] to beta and enabled it by default. This feature improves security of service account tokens by requiring a one hour expiry time, over the previous default of no expiration. This means that applications that do not refetch service account tokens periodically will receive an HTTP 401 unauthorized error response on requests to Kubernetes API server with expired tokens.

In our kubernetes audit logs we see that fluentd still using stale tokens

annotations.authentication.k8s.io/stale-token | subject: system:serviceaccount:logging:fluentd-forwarder, seconds after warning threshold: 53577
-- | --

What I understand, that should be fixed by upgrading kubernetes client SDK to latest release as following

* Go v0.15.7 and later
* Python v12.0.0 and later
* Java v9.0.0 and later
* Javascript v0.10.3 and later
* Ruby master branch
* Haskell v0.3.0.0

What did you expect to happen?

Fluentd to support BoundServiceAccountTokenVolume refresh token after upgrading to k8s 1.21

To Reproduce

install fluentd in kubernetes/eks cluster with version >= 1.21 and check the kubernetes audit logs for stale-token

Expected behavior

should have fixed the stale-token issue.

Your Environment

- Fluentd version: v0.3.7 (image : 1.14.6-debian-10-r49)
- Kubermetes/EKS :  v1.21

Your Configuration

Using image: 1.14.6-debian-10-r49

Your Error Log

annotations.authentication.k8s.io/stale-token | subject: system:serviceaccount:logging:fluentd-forwarder, seconds after warning threshold: 53577

Additional context

No response

[DevAndrewGeorge]

+1 to this.

[lcohen-11]

Hi,
I face the same issue.

[Mrunali0721]

Hello, Facing the same issue.

[pnuccioiqvia]

Hello,
We are also this issue, we need a fix ASAP.
thank you

[ashie]

To Reproduce

install fluentd in kubernetes/eks cluster with version >= 1.21 and check the kubernetes audit logs

Please describe the detail of the steps to reproduce what you did.

Fluentd core itself doesn't aware k8s. k8s integration of Fluentd is completely done by third-party plugins, and they aren't controlled by our organization. So probably you need to forward your report to somewhere (fluent-plugin-kubernetes_metadata_filter?).

Your Environment

- Fluentd version: v0.3.7 (image : 1.14.6-debian-10-r49)
- Kubermetes/EKS :  v1.21

We don't aware such version of Fluend or Docker container.
Probably you use a docker container which is maintained by other organization (here?).

[ashie]

We'll continue this issue at fluent/fluentd-kubernetes-daemonset#1361

[ashie]

This issue has been addressed by fluent-plugin-kubernetes_metadata_filter: fabric8io/fluent-plugin-kubernetes_metadata_filter#337
Please use fluent-plugin-kubernetes_metadata_filter v2.11.1 or later.