Concatenate files before merge

If you think LogMX needs a new useful feature!
Post Reply
cagnionm
Posts: 1
Joined: Thu Sep 23, 2021 4:15 pm

Concatenate files before merge

Post by cagnionm »

Bonjour,

Our log files are often truncated in the middle of the last line to rotate the log files.

E.g. :

Code: Select all

File 1
< Line 1 > EOL
< Line 2 > EOL
...
< Begin of Line N > NO_EOL

File 2
< End of previous Line > EOL
< Line 1 > EOL
...
When I merge these truncated files with the type "Append files", LogMX considers the truncated lines as full line whereas there is no end-of-line (EOL).
Thus LogMX does not parse correctly the first line from the following file.

Could you add an option in order to take into account the presence of the EOL character(s) when merging files and thus append the first line of the folowing log file at the end of the last line of the previous log file. This would prevent creating a new incomplete entry.

Thank you
Mickael
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Re: Concatenate files before merge

Post by admin »

Bonjour,

First, really sorry for the very late response, all the posts on this forum were silently queued, but the forum is now fixed.

Sadly, I am not sure we could implement such feature, because it would mean that a LogMX Parser for one file would have to be aware of other files, and of the order of the merged files. Usually, this is not required because the logging framework makes sure that a log entry is not split in half between 2 rolled files. I tested this with Log4j and Logback, and even with complex JSON log entries and small rolling-file size (10KB), they both opened a new file if the next log entry wouldn't fit in the current file (so most of the files were 9.7KB instead of 10KB).

Which logging framework are you using to produce your logs?

Xavier
Post Reply