Different timestamp formats in log files and merging

Notes, tips, and other usefull things on how to use LogMX

Moderator: admin

Post Reply
isheedm
Posts: 109
Joined: Wed Sep 12, 2007 2:16 pm

Different timestamp formats in log files and merging

Post by isheedm »

I used the Tools->Merge Files option on 2 log files. The first log file used 12-hr format timestamps (4:00:25 PM), and the second used 24-hr format (16:00:25,608). When they got merged, it didn't work like I expected. All the 12-hr formats were at the top of the merged window, and the 24-hr were at the bottom. With a millisecond comparison, they should have sorted correctly. Is this a bug?
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Re: Different timestamp formats in log files and merging

Post by admin »

Hello,

It depends on which Log Parsers are used.
  1. First, make sure you check the box "Interlaced merge" in the Merge window.
  2. In order to perform a valid interlaced merge, both Parsers (i.e. the parser of each log file) shall support absolute dates (i.e. elapsed time since 1/1/1970): to check if a parser support it, before the merge, right-click on a log entry (no entry should be selected before that), and click on "Compute elapsed time".
  3. If both Parsers claim the support of absolute dates (see point 2) but the merge is still not interlaced correctly, then at least one Parser doesn't parse the log entry date correctly. If it's your case, please let me know which Parsers are used.
Please let me know

Xavier
isheedm
Posts: 109
Joined: Wed Sep 12, 2007 2:16 pm

Re: Different timestamp formats in log files and merging

Post by isheedm »

1. Confirmed

2. Elapsed time gives results, but they don't exactly agree. I opened some log files from yesterday and got these results:

Pattern 1: elapsed time for 2013-06-26 17:17:55,706 = 0 days 16 h 7m 25s 355ms.
Pattern 2: elapsed time for June 26, 2013 5:17:44 PM = 1 day, 4h 6m 20s 929ms.

They are different by 12 hrs.

3. I am using 2 Log4j parsers.

Pattern 1 = %p %d{ISO8601} [%t] (%F:%L) %c - %m%n
Pattern 2 = %p| %u{username}| %d{MMM dd, yyyy H:m:s aa}| %c| %M| %t| %m
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Re: Different timestamp formats in log files and merging

Post by admin »

Hello,

I've reproduced your test case. I think the Log4j pattern for your 2nd Parser is not correct: you may want to replace "H:m:s" with "h:m:s", since
  • "H" is "Hour in day (0-23)"
  • "h" is "Hour in am/pm (1-12)" :wink:
(see http://docs.oracle.com/javase/6/docs/ap ... ormat.html)

Xavier
isheedm
Posts: 109
Joined: Wed Sep 12, 2007 2:16 pm

Re: Different timestamp formats in log files and merging

Post by isheedm »

That fixed it! Thanks.
Post Reply