Custom Paser

Here developers can talk about how to write a Parser for LogMX

Moderator: admin

Post Reply
WendyLC
Posts: 2
Joined: Thu Jan 09, 2020 8:28 pm

Custom Paser

Post by WendyLC » Thu Jan 09, 2020 8:34 pm

I am new to LogMX.

I have tried to write a Log4, and Regex and can't get it to work. :(

Log line format: Timestamp [ default task-Line] Level Emitter : Message

Example:
2020-01-08 11:20:40,453 [ default task-390] DEBUG app.wss.server.CommonAuthHandler : << invoke, authorization succeeded

Can you help me?

Thanks Wendy
Attachments
logmx_log4.JPG
logmx_log4.JPG (76.98 KiB) Viewed 4033 times

admin
Site Admin
Posts: 529
Joined: Sun Dec 17, 2006 10:30 pm

Re: Custom Paser

Post by admin » Fri Jan 10, 2020 1:01 am

Hello,

Here are 2 possible Log4j and Regex Parsers, but I think the Regex will be better because of all the white space characters present in your logs: I'm not sure which exact padding is used to produce these logs, so depending on the number of characters next to the padded zone, you may have 4, 5, or 6 spaces for example.
One thing that was not right in your parser is the date format (".SSS" instead of ",SSS").
Also, when posting your logs on this forum, the formatting was altered because the text you posted doesn't contain multiple spaces compared to your screenshot (duplicated spaces removed), so please be aware of the number of spaces if you use the Log4j parser because the log I used in my example is surely not what you have ;)

Log4j:

Code: Select all

%d{yyyy-MM-dd HH:mm:ss,SSS} [   default task-%L] %-5level %M : %m
2020-01-08 11:20:40,453 [   default task-390] DEBUG app.wss.server.CommonAuthHandler : << invoke, authorization succeeded
parser-log4j.png
parser-log4j.png (31.09 KiB) Viewed 4032 times

Regex:

Code: Select all

(\S+? \S+?)\s+?\[\s*?default task-(\d+?)\]\s+?(\S+?)\s+?(\S+?)\s+?:\s+?(.*)
2020-01-08 11:20:40,453 [   default task-390] DEBUG app.wss.server.CommonAuthHandler : << invoke, authorization succeeded
parser-regex.png
parser-regex.png (34.14 KiB) Viewed 4032 times
Let me know how it goes!
Xavier

WendyLC
Posts: 2
Joined: Thu Jan 09, 2020 8:28 pm

Re: Custom Paser

Post by WendyLC » Fri Jan 10, 2020 1:12 pm

Hi, Thanks for answer me.
It's working to some files and othes no.

Some files have multiple lines, I don't know if this change the format.
I attach the log file and the error, can you help me?

Thanks Wendy
Attachments
APPSVR.log.3000.zip
Log file.
(3.07 KiB) Downloaded 123 times
Error to import the file.
Error to import the file.
ErrorLogMX.jpg (31.87 KiB) Viewed 4031 times

admin
Site Admin
Posts: 529
Joined: Sun Dec 17, 2006 10:30 pm

Re: Custom Paser

Post by admin » Sat Jan 11, 2020 2:00 am

Since this file starts with a log entry being spread over 158 lines, I think you need to change a setting: in menu "Tools" > item "Options" > tab "Parsers", at the bottom you have a text field next to "Switch to next parser when no entry found after XXX lines". By default, the value is 100, which means that if a log entry is not completed after 100 lines when starting to read a file, LogMX will consider that the Parser is not suitable for this file. So if you increase this number (like 200 or 300 for example), it should resolve the issue (but it also means that when LogMX will auto-detect the log parser to use, each Parser will parse the first 200 or 300 lines, hence a small performance hit if the actual parser is the last one on the list)

Let me know!
Xavier

Post Reply