You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Similar to #1013, I try to parse a timestamp field located under a nested key in a json structured log.
My particular logs are from MongoDB. Please see it's documentation for more example logs.
[PARSER]
Name mongodb-json
Format json
Time_Key t.$date
Time_Format %Y-%m-%dT%H:%M:%S.%L%z
[SERVICE]
Log_Level debug
Parsers_File parsers.conf
[INPUT]
Name tail
Path mongos.log
Parser mongodb-json
Read_from_Head true
Exit_On_Eof true
[OUTPUT]
Name stdout
Match *
The problem is; The field containing the timestamp is impossible to parse as such. I have been unable to find any working workaround to get this field parsed.
Please either add support to use Time_Key with nested fields, or suggest a reasonable & working workaround.
I have tried;
to set the Parser Time_Key in various different variations to suit the nested key;
Time_Key t.$date
Time_Key t.\$date
Time_Key "t.$date"
Time_Key "t.\$date"
Time_Key t.$$date
...
to use the "Record Accessor" in the parser to reach the timestamp. Like so; Time_Key $t.["$date"].
a whole bunch of different nest-filter variations, including lift $date from t without luck.
to parse the t key as json with another parser-filter:
[PARSER]
Name json
Format json
[PARSER]
Name mongodb-json
Format json
Time_Key $date
Time_Format %Y-%m-%dT%H:%M:%S.%L%z
[INPUT]
Name tail
Path mongos.log
Parser json
Read_from_Head true
Exit_On_Eof true
[FILTER]
Name parser
Parser mongodb-json
Key_name t
Initially I figured that the key-name$date containing a $ was the issue, but it seem not to be.
A manually crafted log-entry where the t-key have been flatted out works great with bellow parser;
Describe the bug
Similar to #1013, I try to parse a timestamp field located under a nested key in a json structured log.
My particular logs are from MongoDB. Please see it's documentation for more example logs.
To Reproduce
Example log that should be possible to parse;
The problem is; The field containing the timestamp is impossible to parse as such. I have been unable to find any working workaround to get this field parsed.
Please either add support to use
Time_Key
with nested fields, or suggest a reasonable & working workaround.I have tried;
Time_Key
in various different variations to suit the nested key;Time_Key t.$date
Time_Key t.\$date
Time_Key "t.$date"
Time_Key "t.\$date"
Time_Key t.$$date
Time_Key $t.["$date"]
.$date
fromt
without luck.t
key as json with another parser-filter:Initially I figured that the key-name
$date
containing a$
was the issue, but it seem not to be.A manually crafted log-entry where the
t
-key have been flatted out works great with bellow parser;Expected behavior
That the nested
t.$date
may be possible to use as a timestamp field.Screenshots
Your Environment
Additional context
The text was updated successfully, but these errors were encountered: