You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I'm not sure if this is the right repository to open this issue, but the problem seems to be in the live-reloading feature in the kubesphere/fluent-bit image. It seems the configuration is not updated until I forcibly restart the pod.
To Reproduce
We updated the host value in the cluster output:
Before:
apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterOutput
metadata:
labels:
fluentbit.fluent.io/component: logging
fluentbit.fluent.io/enabled: "true"
name: my-output
spec:
alias: my_pattern
customPlugin:
config: |
Name http
Match my_pattern
Host ingress.coralogix.com
Port 443
URI /logs/v1/singles
Format json_lines
TLS On
Header Authorization Bearer <redacted>
compress gzip
Retry_Limit 10
match: my_pattern
After:
apiVersion: fluentbit.fluent.io/v1alpha2
kind: ClusterOutput
metadata:
labels:
fluentbit.fluent.io/component: logging
fluentbit.fluent.io/enabled: "true"
name: myoutput
spec:
alias: mypattern
customPlugin:
config: |
Name http
Match mypattern
Host ingress.cx498-aws-us-west-2.coralogix.com
Port 443
URI /logs/v1/singles
Format json_lines
TLS On
Header Authorization Bearer <redacted>
compress gzip
Retry_Limit 10
match: mypattern
However, we were still observing that in the fluent-bit logs, it was trying to send the logs to:
The config generated by the fluent-operator was correct as well:
[Output]
Match mypattern
Alias mypattern
Name http
Match mypattern
Host ingress.cx498-aws-us-west-2.coralogix.com
Port 443
URI /logs/v1/singles
Format json_lines
TLS On
Header Authorization Bearer <redacted>
compress gzip
Retry_Limit 10
This only resolved after I did a kubectl rollout restart daemonset fluent-bit and then it began sending via the correct configuration:
I see something similar with changes in a Lua script / ClusterFilter, but not even a DaemonSet restart helps there. The rendered fluentbit.conf and other ConfigMaps / Secrets look fine. I have to re-create the whole Fluentbit resource to get fluent-bit picking up the changes.
Describe the issue
I'm not sure if this is the right repository to open this issue, but the problem seems to be in the live-reloading feature in the
kubesphere/fluent-bit
image. It seems the configuration is not updated until I forcibly restart the pod.To Reproduce
We updated the host value in the cluster output:
Before:
After:
However, we were still observing that in the fluent-bit logs, it was trying to send the logs to:
The config generated by the fluent-operator was correct as well:
This only resolved after I did a
kubectl rollout restart daemonset fluent-bit
and then it began sending via the correct configuration:Expected behavior
The new configuration should be loaded by the fluent-bit pods without a forced restart.
Your Environment
How did you install fluent operator?
I'm using the helm chart. This is deployed to AWS EKS.
Additional context
No response
The text was updated successfully, but these errors were encountered: