svlogd / multilog -> SQL database

svlogd / multilog -> SQL database

[Daniel Clark]

Has anyone happened to have done any work to get svlogd/multilog data
into a SQL database, such as PostgreSQL?

I'm currently using RSyslog [1] for this, but am switching over to
runit/svlogd for a bunch of services, and also would prefer PostgreSQL
to MySQL (rsyslogd is tied somewhat tightly to MySQL)

[1] http://www.rsyslog.com/

-- 
Daniel Clark # http://dclark.us # http://opensysadmin.com

Re: svlogd / multilog -> SQL database

[Charlie Brady]

On Thu, 4 Jan 2007, Daniel Clark wrote:

> Has anyone happened to have done any work to get svlogd/multilog data
> into a SQL database, such as PostgreSQL?

I think what you mean to say is, has anyone written a logger which can be 
used by supervise/runit in place of svlogd/multilog. It will read from 
standard input, and write log records into an SQL database.

I don't see where or why you'd use svlogd or multilog in that scenario.

[I'd be concerned about reliability and load issues.]

Alternatively, you might post-process svlogd log files into SQL (at normal 
log rotation time, and from time to time otherwise), depending on what 
analysis capabilities you are seeking.

Re: svlogd / multilog -> SQL database

[Daniel Clark]

On 1/11/07, Charlie Brady <charlieb-supervision@budge.apana.org.au> wrote:
> On Thu, 4 Jan 2007, Daniel Clark wrote:
>
> > Has anyone happened to have done any work to get svlogd/multilog data
> > into a SQL database, such as PostgreSQL?
>
> I think what you mean to say is, has anyone written a logger which can be
> used by supervise/runit in place of svlogd/multilog. It will read from
> standard input, and write log records into an SQL database.
>
> I don't see where or why you'd use svlogd or multilog in that scenario.
>
> [I'd be concerned about reliability and load issues.]

Yes, you are right; looking at this again, it seems like I would want
to use socklog (with svlogd) on the client machines to get the logs to
the central server [1]. So the question is, once this is done, what
runs on the central server to get the information into PostgreSQL?
Based on the rousing response to this thread, I'm guessing the answer
is some script I go and write myself :-)

> Alternatively, you might post-process svlogd log files into SQL (at normal
> log rotation time, and from time to time otherwise), depending on what
> analysis capabilities you are seeking.

Yes, that would probably make the most sense.

[1] http://smarden.org/socklog/network.html

Thanks for the reply,
-- 
Daniel Clark # http://dclark.us # http://opensysadmin.com