http manager

Here developers can talk about how to write a Manager for LogMX

Moderator: admin

Post Reply
jyemin
Posts: 2
Joined: Fri May 23, 2008 9:35 pm

http manager

Post by jyemin »

I'd like to create an http-based manager that supports the following:

1. auto refresh
2. http basic/digest authentication with password caching on the client

The server will support URLs like this:

http://my.server.com:39871/global/diag/ ... 00&len=100

and the page will be password protected.

The reason for this is that our log files get too big for scp. I want to merge the results from 20 app servers in a cluster.

Has anyone done anything similar already that they want to share? It 's not clear to me how this manager would get the password from the password cache. The only method I see is getOrAskUserPassword, and that does not seem sufficient.

Any guidance would be much appreciated. If I can get this to work, I'd like to buy a license for 100 or so developers.

Thanks,
Jeff
jyemin
Posts: 2
Joined: Fri May 23, 2008 9:35 pm

Post by jyemin »

I think I misunderstood getOrAskUserPassword. I would call that from init or prepareForReading and store it in my Manager implementation. Right?
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Post by admin »

Hello,

Your Manager can get the user password (from LogMX password repository) by calling getOrAskUserPassword() with parameters:
- pLogin : user HTTP login
- pHost : remote HTTP host to read data from
ex:

Code: Select all

String userPassword = getOrAskUserPassword("jeff", "my.server.com");
(if the user isn't always "jeff", you can add the user login in URL, like "http://user@my.server.com:39871/global/ ... 00&len=100")
The first time, no password will be found by LogMX. LogMX will then display a dialog box so that you can type your password. If (and only if) your Manager successfully read some bytes, LogMX will save this password in its repository. Next calls to "getOrAskUserPassword()" will return user password without dialog box.
You should call this method from "prepareForReading()" (method "init()" should only store URL parameters in Manager implementation). Indeed, you may store this password in Manager implementation, but you can also save it only in a local variable in "prepareForReading()".

Tips: Feel free to post other questions in this topic if you didn't manage to write this Manager.

Xavier.
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Post by admin »

Hello,

LogMX v1.2.4 is now available on this web site. You can now use Auto-refresh (and Tail mode) on SFTP files.
kavita
Posts: 2
Joined: Tue Jan 29, 2013 11:34 am

Re: http manager

Post by kavita »

Hi,
SampleHTTPManager currently does not support AutoRefresh and Flush in v3.2.0.
How can we enable these features for this manager? What we need to do?
If we want to edit the manager class for these features, how we can write code for such features?

Kavita
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Re: http manager

Post by admin »

Hello Kavita,
  1. Supporting Flush
    With or without LogMX, I don't see how you could perform a 'Flush' on a file through HTTP... :shock: well, I guess maybe if you own the HTTP server you could provide a URL to request a flush (ex: http://myserver/flush.php?file=myfile or even http://myserver/my/file?flush) and implement the flush on the server-side... If you use this kind of tricks, I suppose you would just have to read this "Flush request" URL (just like you do to read your log file) to get the result of the flush operation for example...
  2. Supporting Auto-refresh
    The code to add to support Auto-refresh is described here and more precisely here:
    To support Auto Refresh, a Manager must:
    - return true in supportHeaderReading()
    - return true in supportRandomAccess()
    - implement readFileHeader(int)
    - implement readLineAtOffset(long)
    - implement getCurrentOffset()
    - implement at least one of getCurrentFileSize() or getCurrentFileLastModifDate(). The unimplemented method should always return the same value
    I think the tricky part will be to read from a specific offset (position) in the HTTP file. Here, "HttpURLConnection.setRequestProperty()" seems to be what you need:
    http://stackoverflow.com/questions/3411 ... d-download
    http://stackoverflow.com/questions/3428 ... oad-part-2
    http://www.javadocexamples.com/java/net ... ue%29.html
Please let us know how if you have any problem (or a working solution, so we can all share it! :P)

Xavier
kavita
Posts: 2
Joined: Tue Jan 29, 2013 11:34 am

Re: http manager

Post by kavita »

1. I took referenece of existing SFTPFileManager and it seems it is difficult to understand its class logic and other mehtods.
and also for this feature looking at the code it seems code can be common to HTTPManager while reading header, lineoffset,etc.
So to implement these methods, can I use the same implementation code as in SFTPFileManager?
Because it is very difficult to understand the code of existing jar classes and implement in SampleHttpManager class.
Also why this version itself does not provide autorefresh functionality? So that all default managers would have main features available. :)
2. When we edit existing SampleHTTPManager from Options > Managers, LogMx/lib/*.jar and LogMx/classes are missing.
Am I missing any configuration/path here?


Thanks,
Kavita
admin
Site Admin
Posts: 555
Joined: Sun Dec 17, 2006 10:30 pm

Re: http manager

Post by admin »

Hello,

I'm not sure I understand what you mean:
  1. SFTPManager is a private and internal LogMX manager... so its code is obfuscated, so understanding its logic and method is indeed hard, on purpose :?
  2. The code of SFTPManager and HTTPManager are indeed very similar, since they extend the same abstract class. But looking at SFTPManager will really don't help you to implement Flush or AutoRefresh on HTTPManager
  3. You didn't answer me about how you plan to implement file flush for HTTP... that still let me " :?: "... maybe you thought about "only" clearing all log entries in LogMX GUI, without really flushing the remote file?
  4. We didn't implement Auto-refresh by default on HTTP manager to make it easier to understand.
  5. You're right, LogMx/lib/*.jar and LogMx/classes are missing, but that's OK, LogMX doesn't need this to add/remove Managers (all user-defined Managers should be located in "managers" directory
  6. In case you have not read this yet: http://www.logmx.com/p_manager_dev.php
  7. If you want, we can implement file flush and Auto-refresh for you, depending on some conditions, I send you a private message about this.
Xavier
Post Reply