file format doesn't handle absolute dates

Here developers can talk about how to write a Parser for LogMX

Moderator: admin

Post Reply
lili
Posts: 4
Joined: Mon Nov 03, 2014 3:19 pm

file format doesn't handle absolute dates

Post by lili » Mon Nov 03, 2014 7:49 pm

Hi,
I don't write a date in my log (the log file is replaced daily). So the log looks like this:

Code: Select all

14:37:04,995 INFO  [8080-exec-5] (Main.java:291) - Start main
14:37:05,151 INFO  [8080-exec-5] (Mainjava:293) - Finish main
And the parser is:

Code: Select all

%d{HH:mm:ss,S} %p [%t] (%c) - %m
Though I get a message that the file format doesn't handle absolute dates. How can I configure a parser to handle absolute dates?

thank you

admin
Site Admin
Posts: 419
Joined: Sun Dec 17, 2006 10:30 pm

Re: file format doesn't handle absolute dates

Post by admin » Mon Nov 03, 2014 8:52 pm

Hello lili,

Using this log format and a Log4j/Logback Pattern Parser, you can't. An "absolute date" means that you can locate it on a timeline, so here you are missing day/month/year.
If you want to consider that this date always belongs to the current day, you can do so using a Java Parser, like for example:

Code: Select all

package sample.parser;

import java.text.SimpleDateFormat;
import java.util.Calendar;
import java.util.Date;
import java.util.regex.Matcher;
import java.util.regex.Pattern;

import com.lightysoft.logmx.business.ParsedEntry;
import com.lightysoft.logmx.mgr.LogFileParser;

/**
 * Sample LogMX Parser able to parse a log file with multi-line support and Absolute/Relative Date support.<BR/>
 * Here is an example of log file suitable for this parser:<BR/>
 * 
 *   14:37:04,995 INFO  [8080-exec-5] (Main.java:291) - Start main
 *   14:37:05,151 INFO  [8080-exec-5] (Main.java:293) - Finish main
 */
public class LiliParser extends LogFileParser {
    /** Current parsed log entry */
    private ParsedEntry entry = null;

    /** Entry short-date format */
    private static SimpleDateFormat shortDateFormat = new SimpleDateFormat("HH:mm:ss,S");

    /** Entry long-date format */
    private static SimpleDateFormat longDateFormat = new SimpleDateFormat("d/M/yy HH:mm:ss,S");

    /** Pattern for entry begin */
    private final static Pattern ENTRY_BEGIN_PATTERN = Pattern
        .compile("^(\\d+:\\d+:\\d+,\\d+)\\s+(\\S+)\\s+\\[(.*?)\\]\\s+\\((.*?)\\)\\s+-\\s*(.*)$");

    /** Buffer for Entry message (improves performance for multi-lines entries)  */
    private StringBuilder entryMsgBuffer = null;


    /** 
     * Returns the name of this parser
     * @see com.lightysoft.logmx.mgr.LogFileParser#getParserName()
     */
    public String getParserName() {
        return "Lili Parser";
    }

    /**
     * Returns the supported file type for this parser
     * @see com.lightysoft.logmx.mgr.LogFileParser#getSupportedFileType()
     */
    public String getSupportedFileType() {
        return "Lili log files";
    }

    /**
     * Process the new line of text read from file 
     * @see com.lightysoft.logmx.mgr.LogFileParser#parseLine(java.lang.String)
     */
    protected void parseLine(String line) throws Exception {
        // If end of file, records last entry if necessary, and exits
        if (line == null) {
            recordPreviousEntryIfExists();
            return;
        }

        Matcher matcher = ENTRY_BEGIN_PATTERN.matcher(line);
        if (matcher.matches()) {
            // Record previous found entry if exists, then create a new one
            prepareNewEntry();

            entry.setDate(matcher.group(1));
            entry.setLevel(matcher.group(2));
            entry.setThread(matcher.group(3));
            entry.setEmitter(matcher.group(4));
            entryMsgBuffer.append(matcher.group(5));
        } else if (entry != null) {
            entryMsgBuffer.append('\n').append(line); // appends this line to previous entry's text
        }
    }

    /**
     * Returns a relative Date for the given entry 
     * @see com.lightysoft.logmx.mgr.LogFileParser#getRelativeEntryDate(com.lightysoft.logmx.business.ParsedEntry)
     */
    public Date getRelativeEntryDate(ParsedEntry pEntry) throws Exception {
        synchronized (shortDateFormat) { // Java date formatter is not thread-safe
            return shortDateFormat.parse(pEntry.getDate());
        }
    }

    /**
     * Returns the absolute Date for the given entry 
     * @see com.lightysoft.logmx.mgr.LogFileParser#getAbsoluteEntryDate(com.lightysoft.logmx.business.ParsedEntry)
     */
    public Date getAbsoluteEntryDate(ParsedEntry pEntry) throws Exception {
        Calendar now = Calendar.getInstance();
        StringBuilder dateToParse = new StringBuilder(80);
        dateToParse.append(now.get(Calendar.DAY_OF_MONTH)).append('/')
            .append(now.get(Calendar.MONTH) + 1).append('/').append(now.get(Calendar.YEAR))
            .append(' ').append(pEntry.getDate());
        synchronized (longDateFormat) { // Java date formatter is not thread-safe
            return longDateFormat.parse(dateToParse.toString());
        }
    }

    /**
     * Send to LogMX the current parsed log entry
     * @throws Exception
     */
    private void recordPreviousEntryIfExists() throws Exception {
        if (entry != null) {
            entry.setMessage(entryMsgBuffer.toString());
            addEntry(entry);
        }
    }

    /**
     * Send to LogMX the current parsed log entry, then create a new one
     * @throws Exception
     */
    private void prepareNewEntry() throws Exception {
        recordPreviousEntryIfExists();
        entry = createNewEntry();
        entryMsgBuffer = new StringBuilder(80);
    }
}
To automatically import this Parser, go to menu "Tools" > "Options" > "General", click on button "Import options...", and choose this file (then don't forget to disable your own previous Parser in "Parsers" Options tab). To get more information on how to create/modify Java class Parsers, you can read this, or ask any question here :)

admin
Site Admin
Posts: 419
Joined: Sun Dec 17, 2006 10:30 pm

Re: file format doesn't handle absolute dates

Post by admin » Tue Nov 04, 2014 4:30 pm

Sorry, the exported Parser was corrupted during upload. The file is now fixed (the above link is valid)

lili
Posts: 4
Joined: Mon Nov 03, 2014 3:19 pm

Re: file format doesn't handle absolute dates

Post by lili » Wed Nov 05, 2014 5:13 pm

thank you for the prompt reply!
where can I use the absolute dates in the basic version of the program? Is it used only to calculate the time between 2 selected log entries? Or can I find 2 subsequent entry logs that are written with a distance of more than x milliseconds?


thanks again

admin
Site Admin
Posts: 419
Joined: Sun Dec 17, 2006 10:30 pm

Re: file format doesn't handle absolute dates

Post by admin » Wed Nov 05, 2014 6:17 pm

Here are the main features (Pro & Eval versions) that use the entries Absolute date:
  1. Compute elapsed time since the entry was emitted ("Compute elapsed time" feature while only 1 entry is selected: time between entry creation and now)
  2. Compute elapsed time between two entries emission ("Compute elapsed time" feature while at least 2 entries are selected, and if Parser could not provide a Relative date: time between first entry creation and last entry creation)
  3. Filter entries by date (Calendar feature / date filters)
  4. To interlace log entries by date during a file merge
  5. When a Parser could not provide a Relative date (an absolute date can be seen as a relative date for which the time origin is 01/01/1970)
  6. Some other minor/internal features
Here are the main features (Pro & Eval versions) that use the entries Relative date:
  1. Compute elapsed time between two entries emission ("Compute elapsed time" feature while at least 2 entries are selected: time between first entry creation and last entry creation)
  2. Correlation-by-time feature (when no Absolute date is provided by the Parser)
  3. Some other minor/internal features
So to answer your questions:
  • Absolute dates are used to calculate the time between 2 selected log entries only if Parser could not provide a Relative date
  • I'm not sure to understand your question about 2 subsequent entries. If you want to see, in LogMX main view, entries that are separated with at least x milliseconds or more, you may want to:
    • try the Correlation feature (a dark horizontal line will appear when two entries are separated by X milliseconds or more)
    • or when implementing your own Parser, compute for each entry the elapsed time (=T) since the previous entry was emitted, and if T > X milliseconds, then mark this entry using a user-defined field (e.g. new field "comes lately" equals to "Y" when T>X and equals to "N" otherwise), or prefix its "Message" field with a special string. This way, you will be able to easily filter such entries in main LogMX view.
Xavier

Post Reply