Emitter Tree Wrong

Notes, tips, and other usefull things on how to use LogMX

Moderator: admin

Post Reply
c0nnex
Posts: 8
Joined: Tue Sep 27, 2016 11:00 pm
Contact:

Emitter Tree Wrong

Post by c0nnex » Tue Sep 27, 2016 11:44 pm

(5.5.1 pro)

I have the following Emitters:

A.B.C
A.B.C.D1
A.B.C.D2
X.Y.Z

In the Emitter tree A.B.C is not selectable/filterable and it will still show up in the display when i do "Hide other emitters" on "X.Y.Z"
A.B.C shows as package (ok) and numbers are showing up as well, but no way to filter/hide this from UI.

What am I missing?

admin
Site Admin
Posts: 418
Joined: Sun Dec 17, 2006 10:30 pm

Re: Emitter Tree Wrong

Post by admin » Wed Sep 28, 2016 2:40 am

Hello,

I'm sorry but I'm not sure to understand what "A.B.C is not selectable/filterable" means.
Are A,B,C,X,Y packages and D1,D2,Z emitters? If yes, I've just checked and doing "Hide other emitters" on "X.Y.Z" works: A.B.C.D1 and A.B.C.D2 are not displayed anymore (i.e. entries emitted by these emitters are not displayed anymore).
When a package or emitter is "hidden", their name still appears on the Emitters tree (semi-transparent effect), as well as their number of log entries is still displayed at the right of their name.
Maybe sending a screenshot will help me understand?

Xavier

c0nnex
Posts: 8
Joined: Tue Sep 27, 2016 11:00 pm
Contact:

Re: Emitter Tree Wrong

Post by c0nnex » Fri Sep 30, 2016 9:35 pm

Ok hard to understand.
I created a Video to show the issue:
https://www.fsgs.com/download/logmx_emitters.mp4

In the Log Display , you see there is an Emitter "SPAD.net.SimConnect" (which has several Child emitters, but emits logmessages as well)
The "SPAD.net.SimConnect" Emitter is not shown in emitter Tree.
Then I select the Emitter "SPAD.neXt.Events.EventDefinition" -> Hide other emitters
The Emitter "SPAD.net.SimConnect" is still shown in the Display.

Since it does not appear in the emitter-tree there is no way to select/deselect it.

admin
Site Admin
Posts: 418
Joined: Sun Dec 17, 2006 10:30 pm

Re: Emitter Tree Wrong

Post by admin » Fri Sep 30, 2016 10:12 pm

Hello,

You are right, it looks like a bug, sorry about that. Yet, we didn't manage to reproduce this bug. Does it also happen when you re-open the same log file in a new tab? The question is: do you use Auto-refresh and/or multiple manual refreshes to get to this point? If the bug also happens in a freshly opened tab, then it would be easier to reproduce and debug.

In case we still don't manage to reproduce this bug, could you please send us your log file? (support@logmx.com, with title "Forum - c0nnex")

Thank you.
Xavier

c0nnex
Posts: 8
Joined: Tue Sep 27, 2016 11:00 pm
Contact:

Re: Emitter Tree Wrong

Post by c0nnex » Sat Oct 01, 2016 2:19 am

I sent you a logfile to reproduce it.

The issue might be related to refreshing.
I noticed two other oddities with it:

Autorefresh is on , LocalFileManager RefreshDelay = 100 , MaxLines = 1000
Only only logfile (the live logfile) is open in LogMX

Logging (nlog) keeps logfile open, flushes every 250 ms

a) The logfile refreshes very very slowly. Sometimes it needs 10-15 seconds to refresh, sometimes it does not refresh at all until I manually refresh it.
(Logfiles is written to disk by that time already, Notepad and log4view see the new logentries already long time before LogMX sees them)

b) The program logging creates a new logfile (same name) every time it is started (and rotates the old logfiles).
It's really annoying, that if this happens, LogMX forgets about the Emitters that are visible/hidden :(
Usually I do a test run, disable all the emitters i don't want to see, change something in code, run the test again to observe logging -> bum all my emitter settings are gone and i have to start over. With over 400 emitters that's quite a job :(
Would be nice to add options for
1) auto save current filterstate if logfile resets
2) auto load last filterstate if logfile resets
(so you would have one "last filterstate")

Nice to have:
- a "reset filter" in the Filterstate menu :-D
- a comfortable way to set level-filter by emitter. Eg. I want to see TRACE of SPAD.Global , but only INFO of SPAD.Detail. I use that very very often, and creating the filter condition for that takes a very long time.

Sorry to be picky, but log4view has all this nice "make it easy to use" features (while it has other drawbacks, specially with big logfiles).


-Ulrich

c0nnex
Posts: 8
Joined: Tue Sep 27, 2016 11:00 pm
Contact:

Re: Emitter Tree Wrong

Post by c0nnex » Sat Oct 01, 2016 2:24 am

Argl can't edit post anymore ...

Just checked with a non-live logfile, and there the problem shown in the video is not reproducible.

c0nnex
Posts: 8
Joined: Tue Sep 27, 2016 11:00 pm
Contact:

Re: Emitter Tree Wrong

Post by c0nnex » Sat Oct 01, 2016 2:35 am

Further Investigation:

This only happens in auto-refresh if the parent-level emitter writes the first log entry AFTER child emitter(s) wrote log entries.

So "SPAD.net.SimConnect.Something" writes log entries -> "SPAD.net.SimConnect" -Dummy-Package created implicitly
now after a while "SPAD.net.SimConnect" writes log entries , then the problem occurs.

Looks like LogMX does not realize that dummy package became a real emitter
Only occurs with parent emitters. Child emitters are filtered and added correctly over time, unless they have childs also.

admin
Site Admin
Posts: 418
Joined: Sun Dec 17, 2006 10:30 pm

Re: Emitter Tree Wrong

Post by admin » Mon Oct 03, 2016 4:30 am

Hello,

Thank you for this detailed analysis and this feedback.
Thanks to your e-mails and your posts here we have reproduced the Emitters tree bug and fixed it in v6.0.0 that should be released in November.

Concerning the other issues:

a) I don't understand why LogMX would take this long to see the modification done on a local file. Could you please tell me what is displayed at the left of the Auto-Refresh toolbar? (e.g. "Status: <icon> (+1 new entry, <T> ago)"). Sometimes LogMX can detect a file size change or a file date modification date without any new data (invalid content, incomplete log entry, useless white-space character, ...) and then it displays "Status: <green synch icon> (+0 new entry, <T> ago)". Also, could you please let me know which Parser was used when it happened (if it's not one of the Parsers you sent us by e-mail, could you please send it too?)

b) Thanks, good idea. I'm adding this to our TODO list (new feature surely as a new option, enabled by default, to restore filtering state when file is rolled/rotated).

c) ("Nice to have")
- I'm not sure what would the "reset filter" feature do? Is it something like removing all filters, showing all emitters and all log levels?
- Log level-filter by emitter: in fact it's already in our roadmap, but not for v6.0.0 at this time. But now you told us this, we may include it in this v6.0.0... :wink:

Anyway, thanks again for all this feedback, it really helps us improving LogMX :D
I will keep you posted in this thread if something happens and when one of these new features/bug fixes are released.

Xavier

c0nnex
Posts: 8
Joined: Tue Sep 27, 2016 11:00 pm
Contact:

Re: Emitter Tree Wrong

Post by c0nnex » Mon Oct 03, 2016 4:49 am

admin wrote: Thank you for this detailed analysis and this feedback.
Thanks to your e-mails and your posts here we have reproduced the Emitters tree bug and fixed it in v6.0.0 that should be released in November.
Great!
admin wrote: a) I don't understand why LogMX would take this long to see the modification done on a local file. Could you please tell me what is displayed at the left of the Auto-Refresh toolbar? (e.g. "Status: <icon> (+1 new entry, <T> ago)"). Sometimes LogMX can detect a file size change or a file date modification date without any new data (invalid content, incomplete log entry, useless white-space character, ...) and then it displays "Status: <green synch icon> (+0 new entry, <T> ago)". Also, could you please let me know which Parser was used when it happened (if it's not one of the Parsers you sent us by e-mail, could you please send it too?)
using log4j Parser "%d{yyyy-MM-dd HH:mm:ss.SSSS}|%p|%t|%c|%m" , LocalFileManager

Refresh-Status is constantly say "Status: <green synch icon> (+206 new entry, [counting 0 to 8]s ago)"
(number of entries differs but never less then 150) , refresh happens every 8-10 seconds , nevertheless what i set in the refresh rates.

when restarting programe (logfile rotates ..), it says "Status: <green synch icon> (*from scratch*)" and loads about 500 logentries (while there are about 4000 in the log).
Needs about a minute to catchup and be at live-log. then above schedule continues.

(This is really unreliable. Sometimes it will take a very long time before the log is catched up completely, and in the meantime only 5-10 logentries are shown)

(log4view delay is < 2 seconds , btw. )
admin wrote: c) ("Nice to have")
- I'm not sure what would the "reset filter" feature do? Is it something like removing all filters, showing all emitters and all log levels?
exactly.

While it would be nice also to have a default option for new emitters, coming in over time, to set them to hidden by default.
This is mainly because LogMX need so long to catchup, and it really disturbs live-analyzing if new emitter pop into my filtered log ;-D


-Ulrich

admin
Site Admin
Posts: 418
Joined: Sun Dec 17, 2006 10:30 pm

Re: Emitter Tree Wrong

Post by admin » Tue Oct 04, 2016 12:20 am

Hello Ulrich,

Concerning the AutoRefresh delay, we tried to reproduce the issue but we didn't manage to. Using the parser and the log file you sent us, we prepared to files (F1 and F2), with around 4000 log entries in each file, and a batch script that renames F1 to F2 and vice-versa to simulate a rolled/rotated file. With AutoRefresh enabled for F1, we executed the batch script several times. Each time, LogMX displayed (within less than a second) "(+4778 new entries *from scratch*)" and all the log entries of the file were loaded.
Here are two questions to understand what happens on your side:
  1. Are you using the "Tail" feature?
  2. When you are monitoring the file with LogMX in AutoRefresh mode, did you manually check that the monitored file contains the number of log entries that you expect? Because when using the "Tail" feature, we have seen some logging applications/libraries that were not able to rotate the monitored log file properly in some cases.
I'm pretty sure that once we managed to reproduce this issue we will be able to fix it. The first bug you found concerning the Emitters tree was due to the fact that an emitter's parent produced some logs: it's possible with nLog, but it's impossible/very uncommon for all other logging libraries. For this AutoRefresh delay bug, I also suspect an issue with how nLog processes the log file rotation (it may do it properly, but in a way LogMX is not used to for now...)

Xavier

admin
Site Admin
Posts: 418
Joined: Sun Dec 17, 2006 10:30 pm

Re: Emitter Tree Wrong

Post by admin » Thu Oct 13, 2016 12:50 am

Here are some news about this issue in case other users have the same issue (we solved the problem by e-mail with the help of c0nnex): the problem was that the log file monitored using AutoRefresh was opened by a Bookmark that contained the "Tail" setting. Once a file is opened through such a bookmark, there is no way to know that AutoRefresh uses the Tail mode (i.e. showing only the end of the file). We will fix this in the next release by displaying a visual reminder on the AutoRefresh status bar to indicate that only the end of the file is currently displayed.

admin
Site Admin
Posts: 418
Joined: Sun Dec 17, 2006 10:30 pm

Re: Emitter Tree Wrong

Post by admin » Wed Feb 15, 2017 3:20 am

Hello Ulrich,

LogMX v6.0.0 is now available for download, and it includes (among many other things) the fixes/improvements we talked about in this thread:
  1. Can now choose a log level for each Emitter, in order to virtually reduce its verbosity (option also available in Filtering States so that this filtering element can be reused quickly)
  2. Fixed the bug in Emitters tree when AutoRefresh was used
  3. When a file is opened in Tail mode, now displaying a visual reminder explaining that only the end of the file is displayed
  4. Can now cancel all filtering elements through new menu item in "Filtering state" menu (i.e. display all log levels, cancel all filters, and enable all Emitters)
Thanks again for this helpful feedback :)

Xavier

Post Reply