Re: [Caml-list] A question about Ocaml logging

[Malcolm Matalka <mmatalka@gmail.com>]

Jesper Louis Andersen <jesper.louis.andersen@gmail.com> writes:

> Another point from the trenches of the real world:
>
> I've had great success with metrics over logging in many situations.
>
> Logs are really nice for knowing what went wrong in a system. And if the
> logs have some kind of structure (JSON, S-exps, Erlang Terms, ...) you can
> also dump all context information relevant to the log entry. Modern systems
> such as Kibana can read such structure and build a search index over them.
> It is very useful when trying to figure out what went wrong. In a
> distributed system, you keep a unique request id in the structural log so
> you can join log entries from multiple subsystems easily in the centralized
> logging platform.
>
> Likewise, whenever a given event occurs, you bump a counter or add an entry
> to a histogram (HdrHistogram comes to mind). This allows you to export
> counts to a foreign system for plotting of how the system fares internally.
> It is much cheaper than rendering a log line in the system, and it is
> almost as informative if you don't need the log line, but rather its
> occurrence. Live timing histograms also has the advantage that problems
> tend to show up in the small before a catastrophe. So you can alter the
> operating point of the system dynamically long before the catastrophe occur
> in the real world.
>
> I almost never have debug-style logging lines in my code anymore. I much
> prefer adding a live tracing to the system instead (Erlang has tracing
> facilities, DTrace is useful on Illumos and FreeBSD, etc). Granted,
> you

Is it possible to create dtrace probes in Ocaml code?


> can't do this posthumously of an error, but on the other hand, you can
> tailor it to the situation you have. It also takes care of the need to
> recompile and redeploy with debug logging (in which case you can't
> posthumously handle an error).
>
> The right way to handle system failure is to take a snapshot of the systems
> state (or of part of the system to which the problem pertains). Store this
> snapshot somewhere (in the cloud) and index them, so you can go back with a
> debugger, attach to the core dump, and inspect what went wrong. Post-mortem
> debugging is needed because many of the errors which occur lies outside the
> imagination of the programmers: people abuse systems in ways nobody thought
> possible.
>
>
> On Mon, Jan 16, 2017 at 4:15 PM Anil Madhavapeddy <anil@recoil.org> wrote:
>
>> > On 9 Jan 2017, at 18:52, Chet Murthy <chetsky@gmail.com> wrote:
>> >
>> > All,
>> >
>> > I hope this is the right place to ask this question.  I've been
>> > writing a nontrivial distributed system (well, a number of them over
>> > the last few years) and have had need of a robust and flexible logging
>> > framework.  Specifically, I've been using "bolt" and its descendant,
>> > "volt", which provide camlp4 syntax extensions.  These extensions make
>> > the syntax of the logging statements significantly less verbose, and
>> > that in itself ia a valuable thing.
>> >
>> > With the arrival of ppx rewriters, I realize that the camlp4/camlp5
>> > method of adding syntax to ocaml is deprecated.  So I wonder: is there
>> > some really good logging toolkit out there, that I've overlooked.
>> >
>> > I'm aware of a number of different packages, but only bolt/volt have
>> > syntax extensions, and it's my belief that they're essential to making
>> > effortless pervasive log-line instrumentation.
>> >
>> > But perhaps I just haven't looked hard enough ....  So .... before I
>> > go write my own, I figured I'd ask the list if there were such a
>> > thing.
>>
>> Dear Chet,
>>
>> In MirageOS, we've been moving away from syntax extensions and
>> towards placing the logging directives as closures directly within the
>> code.  This is slightly slower in the case of debug logging, but in
>> practise for distributed systems we are finding that having "permanent"
>> logging at different levels is more valuable than the "recompile with
>> debug logging" that we used to use.
>>
>> The basis library for this that our libraries are mostly using now in the
>> forthcoming MirageOS3 is Daniel Buenzli's Logs library:
>>   http://erratique.ch/software/logs
>>
>> It does not have a syntax extension out of the box, but it does provide
>> flexible support for multiple backends, and has an Lwt module included.
>>
>> Hope that helps!
>>
>> regards,
>> Anil
>>
>>
>>
>>
>> --
>> Caml-list mailing list.  Subscription management and archives:
>> https://sympa.inria.fr/sympa/arc/caml-list
>> Beginner's list: http://groups.yahoo.com/group/ocaml_beginners
>> Bug reports: http://caml.inria.fr/bin/caml-bugs
>>