Match directive not working #4457
-
Describe the bugI have writted one fluentd configuration output as elasticsearch plugin but it's not working as it should work please check the below configurationn image :- FROM fluentd:v1.16-debian-1 To ReproduceTried the latest version builted fluentd elasticsearch plugin changes configuration used file plugin but match directive is not working with tag Expected behaviorThe match pattern should work for the source tag and output the logs to elasticsearch,file or anywhere Your Environment- Fluentd version: 1.16.2
- TD Agent version: 1.16.2
- Operating system: dockerimage
- Kernel version: 1.16.2 dockerversion
- fluent-plugin-elasticsearch-version '5.4.3'Your Configuration<source>
@type tail
@id tail_mylogs
tag xxxx.logs
path "/var/log/containers/xxx-yyyy-*.log"
pos_file "/var/log/containers/fluentd-xxxxx.log.pos"
exclude_path ["/var/log/containers/fluentd-*.log"]
refresh_interval 30
read_from_head true
<parse>
@type "json"
time_format "%Y-%m-%dT%H:%M:%S.%NZ"
time_type string
unmatched_lines
</parse>
</source>
<match xxxx.logs>
@type elasticsearch
@log_level "debug"
host "elastic-cluster-es-http.elastic-system.svc.cluster.local"
port 9200
scheme https
ssl_verify false
user "elastic"
password xxxxxx
reload_connections true
logstash_format true
<buffer>
flush_thread_count 8
flush_interval 5s
chunk_limit_size 2M
queue_limit_length 32
retry_max_interval 30
retry_forever true
</buffer>
</match>
<system>
log_level debug
</system>
</ROOT>Your Error LogMatch directive not workingAdditional contextNo response |
Beta Was this translation helpful? Give feedback.
Answered by
daipom
Apr 2, 2024
Replies: 10 comments 2 replies
-
|
anyone here?? |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@ankit-groverDevops <source>
@type sample
tag xxxx.logs
</source>
<match xxxx.logs>
@type stdout
</match>So, with the information you reported, we can't find what is the cause.
Mayby We can use <source>
@type sample
tag xxxx.logs
</source><match xxxx.logs>
@type stdout
</match> |
Beta Was this translation helpful? Give feedback.
2 replies
-
|
Thanks for your reply @daipom can you please check what's the issue in this configuration as it is not working <source>
@type tail
@id tail_resumeparserLogs
tag resumeparser.*
path "/var/log/containers/rchilli-resumeparser-*.log"
pos_file "/var/log/containers/fluentd-resumeparser.log.pos"
exclude_path ["/var/log/containers/fluentd-*.log"]
refresh_interval 30
read_from_head true
<parse>
@type "json"
time_format "%Y-%m-%dT%H:%M:%S.%NZ"
time_type string
</parse>
</source>
<match resumeparser.var.log.containers.rchilli-resumeparser**>
@type file
path /var/log/fluent/accessuuu
</match>
<system>
log_level debug
</system>NOTE:- i have tried both way for the match directive resumeparser.var.log.containers.rchilli-resumeparser** or resumeparser.** |
Beta Was this translation helpful? Give feedback.
-
|
Hmm, please check the output with the following config? <source>
@type tail
@id tail_resumeparserLogs
tag resumeparser.*
path "/var/log/containers/rchilli-resumeparser-*.log"
pos_file "/var/log/containers/fluentd-resumeparser.log.pos"
exclude_path ["/var/log/containers/fluentd-*.log"]
refresh_interval 30
read_from_head true
<parse>
@type "json"
time_format "%Y-%m-%dT%H:%M:%S.%NZ"
time_type string
</parse>
</source>
<match resumeparser.**>
@type stdout
</match> |
Beta Was this translation helpful? Give feedback.
-
|
@daipom Thanks for your reply tried this configuration already but it seems that tail logs are comming not stdout one. logs:- <source>
@type tail
@id tail_resumeparserLogs
tag resumeparser
path "/var/log/containers/rchilli-resumeparser-*.log"
pos_file "/var/log/containers/fluentd-resumeparser.log.pos"
exclude_path ["/var/log/containers/fluentd-*.log"]
refresh_interval 30
read_from_head true
<parse>
@type "json"
time_format "%Y-%m-%dT%H:%M:%S.%NZ"
time_type string
</parse>
</source>
<match resumeparser>
@type stdout
</match>
<system>
log_level debug
</system> |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@daipom You there?? it would be greatly appreciated if you help me out for this |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@daipom if we try ** in match block it works as stdout but if we try to use resumeparser.** it doesnt work . Please help me out if you can otherwise i have to change the whole approach for application logging |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@ankit-groverDevops Sorry for my late response.
Could you show me the stdout? |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@daipom Hi daipom |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
Logs for the ** match |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
@ankit-groverDevops I found the cause! <parse>
@type json
time_format "%Y-%m-%dT%H:%M:%S.%NZ"
time_type string
</parse>If you don't need to parse it, you can read them as is with the following settings. <parse>
@type none
</parse> |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
ankit-groverDevops
-
|
@daipom Thankyou daipom it worked |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
@ankit-groverDevops I found the cause!
This
parsesetting would not be appropriate for the logs.Is the format of the logs really JSON?
If you don't need to parse it, you can read them as is with the following settings.