Parsing JSON output from log4j2

Notes, tips, and other usefull things on how to use LogMX

Moderator: admin

Post Reply
wlfbck
Posts: 2
Joined: Tue Oct 27, 2020 3:36 pm

Parsing JSON output from log4j2

Post by wlfbck »

Hi,

i'm trying to simply parse a json log generated with the configuration below (which is pretty basic) but i can't get it to work. Always getting "No suitable Log Parser found for logs in:...". If i remove the settings of the JsonLayout i actually get it to open but it's definitely bugged: https://i.imgur.com/oAHiJxz.png

Anyone has any ideas what i'm doing wrong here? Because i've tried so many different thigns now...

Code: Select all

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="WARN" monitorInterval="30">
  
  <Appenders>
    <!-- Rolling File Appender -->
    <RollingFile name="rolling">
      <FileName>system.log</FileName>
      <FilePattern>system%d{yyyy-MM-dd}.log.zip</FilePattern>
      <JsonLayout compact="true" eventEol="true"/>
      <Policies>
      	<OnStartupTriggeringPolicy />
        <TimeBasedTriggeringPolicy interval="1" modulate="true" />
      </Policies>
      <DefaultRolloverStrategy max="5" />
    </RollingFile>

  </Appenders>
  <Loggers>
    <Root level="ALL">
      <AppenderRef ref="rolling" />
    </Root>
  </Loggers>
</Configuration>
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Re: Parsing JSON output from log4j2

Post by admin »

Hello,

You have 3 options to open these logs in LogMX:
  1. Replace "compact="true"" with "compact="false"" in the Log4j config file you shared in your message (inside the "<JsonLayout>" element): The LogMX Java parser "Log4j2JSONLogFileParser" only supports this format for now, and according to your screenshot, this is the one you're using.

  2. Create your own Parser to handle your current logs (i.e. 1 giant line of JSON per log entry, instead of 1 line per field). If you can't use Option 1 because you can't change your log format, please let me know and I'll create a Parser for you (probably a Regex parser - which you will be able to import and modify in LogMX)

  3. Wait for LogMX v8 that should be available in a couple of weeks, as it now provides a JSON Parser natively:
    json-parser.png
    json-parser.png (65.7 KiB) Viewed 8418 times
Please let me know if you want to go with Option 2 and I will create a Parser for you :)

Xavier
wlfbck
Posts: 2
Joined: Tue Oct 27, 2020 3:36 pm

Re: Parsing JSON output from log4j2

Post by wlfbck »

First of all, thank you for the quick answer! :)

Overall i can wait a few weeks, good to know the new version is just around the corner. So i'm going with option 3 :)

Regarding option 1: I actually did that (to be more precise: it looked liked this "<JsonLayout />") and get the view shown in the imgur link. It was generated with this file content:

Code: Select all

{
  "instant" : {
    "epochSecond" : 1603813178,
    "nanoOfSecond" : 896000000
  },
  "thread" : "main",
  "level" : "ERROR",
  "loggerName" : "Test",
  "message" : "bla",
  "endOfBatch" : false,
  "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
  "threadId" : 1,
  "threadPriority" : 5
}
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Re: Parsing JSON output from log4j2

Post by admin »

You are right, the new version of Log4j2 now uses an { instant } object instead of "timeMillis" (I used Log4j2 v2.10 to generate these logs, but v2.11 introduced this new JSON format). We will then also update the current "Log4j2JSONLogFileParser" shipped with LogMX. But the good news is that the generic JSON parser in LogMX v8 handles it smoothly :) :
json-parser-2.png
json-parser-2.png (66.7 KiB) Viewed 8413 times


I will post a message here when LogMX v8 will be available!

Xavier
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Re: Parsing JSON output from log4j2

Post by admin »

Hello,

As promised, LogMX v8.0.0 is now available for download, and includes the JSON parser I mentioned :)
Let me know if you have any questions!

Xavier
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Re: Parsing JSON output from log4j2

Post by admin »

Hello,

As promised, LogMX v8.1.0 is now available for download, and includes the support of Log4j v2.11+ JSON format ("compacted" or not, "complete" or not, as well as exceptions and stack traces) :)

Let me know if you have any questions!

Xavier
Post Reply