The task is to produce these JSON trees. That is, given a sequence of log entries, output a JSON for each trace.
- The solution should be a Java program, executable from the command line.
- The input should be read from standard input or a file (chooseable by the user)
- The output should be one JSON per line, written to standard output, or a file (chooseable by the user).
- As said, there can be lines out of order.
- There can be orphan lines (i.e., services with no corresponding root service); they should be tolerated but not included in the output (maybe summarized in stats, see below).
- Lines can be malformed, they should be tolerated and ignored.
- A nice command-line interface, that allows to specify inputs and outputs from and to files.
- Optionally report to standard error (or to a file), statistics about progress, lines consumed, line consumption rate, buffers, etc. Both at the end of the processing and during it.
- Optionally report to standard error (or to a file), statistics about the traces themselves, like number of orphan requests, average size of traces, average depth, etc.
- As the file could be quite big, try to do the processing using as many cores as the computer has, but only if the processing is actually speeded that way.
As the log entries can be (a bit) out of order, and there could be orphans, managing “pending” entries in memory can be important. This would be the case if we indent this to be a long running program (which is not the case in this particular exercise, but is the suggested situation).
As a bonus, you should implement some management of “pending” entries, based on the date of the logs and an “expiration”. As log entries arrive, seeing one with some timestamp would mean that any pending log which is older than some configurable expiration threshold (relative to that timestamp) should be declared orphan (even if its family comes afterwards). This works in the assumption that traces can be a bit delayed, but not come from the future: we are assuming that the timestamp that we receive is, at the most pessimistic case, current.
● Two pairs of files with log and traces. One small to start and one bigger to test more thoroughly.
● Since there is no deterministic ordering for the traces/calls, it is not enough to just compare two files to know if a solution is correct. For this reason a program (“[login to view URL]”, an executable jar file) is supplied for your convenience: it reports if two trace files are the same, regardless of the order. It also parses the traces, which also allows to check if the format is correct. This is just for convenience, to compare a potential solution with the supplied references.
7 freelancere byder i gennemsnit $117 på dette job
Hi, I am experienced programmer and I can implement such log parsing and trace generation program as described in your document. Please let me know, if you are interested in cooperation.
Hi. I have hedge experience with core java. Your have very interesting task. Could I help you with it so soon as you want? I hope for your positive decision.