Development discussion of WireGuard
 help / color / mirror / Atom feed
* Windows Log Output to Event Viewer or Text File
@ 2021-10-08 20:50 Frank Wayne
  2021-10-08 22:01 ` Jason A. Donenfeld
  0 siblings, 1 reply; 15+ messages in thread
From: Frank Wayne @ 2021-10-08 20:50 UTC (permalink / raw)
  To: WireGuard mailing list

1. Is it possible to get WireGuard on Windows to output its logs to something other than the binary file "Data\log.bin"? 

2. Is there a way to access the contents of the binary file outside of the WireGuard UI?

Frank Wayne

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Windows Log Output to Event Viewer or Text File
  2021-10-08 20:50 Windows Log Output to Event Viewer or Text File Frank Wayne
@ 2021-10-08 22:01 ` Jason A. Donenfeld
  2021-10-12 21:39   ` Frank Wayne
  0 siblings, 1 reply; 15+ messages in thread
From: Jason A. Donenfeld @ 2021-10-08 22:01 UTC (permalink / raw)
  To: Frank Wayne; +Cc: WireGuard mailing list

https://git.zx2c4.com/wireguard-windows/about/docs/enterprise.md#diagnostic-logs

^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: Windows Log Output to Event Viewer or Text File
  2021-10-08 22:01 ` Jason A. Donenfeld
@ 2021-10-12 21:39   ` Frank Wayne
  2021-10-12 21:40     ` Jason A. Donenfeld
  0 siblings, 1 reply; 15+ messages in thread
From: Frank Wayne @ 2021-10-12 21:39 UTC (permalink / raw)
  To: Jason A. Donenfeld; +Cc: WireGuard mailing list

Thanks for the link, Jason.

That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.

Frank Wayne

-----Original Message-----
From: Jason A. Donenfeld <Jason@zx2c4.com> 
Sent: Friday, 8 October, 2021 17:01
To: Frank Wayne <frank.wayne@northwestern.edu>
Cc: WireGuard mailing list <wireguard@lists.zx2c4.com>
Subject: Re: Windows Log Output to Event Viewer or Text File

https://urldefense.com/v3/__https://git.zx2c4.com/wireguard-windows/about/docs/enterprise.md*diagnostic-logs__;Iw!!Dq0X2DkFhyF93HkjWTBQKhk!EnH5c-_WQuKGxDYwm737RpDPw0mKqlouc_l-2cGs4omjuN-SafYq-HHpnKKjvknPsn0SLhpq$ 

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Windows Log Output to Event Viewer or Text File
  2021-10-12 21:39   ` Frank Wayne
@ 2021-10-12 21:40     ` Jason A. Donenfeld
  2021-10-13 13:29       ` Frank Wayne
  0 siblings, 1 reply; 15+ messages in thread
From: Jason A. Donenfeld @ 2021-10-12 21:40 UTC (permalink / raw)
  To: Frank Wayne; +Cc: WireGuard mailing list

On Tue, Oct 12, 2021 at 3:39 PM Frank Wayne
<frank.wayne@northwestern.edu> wrote:
> That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.

Your "that's pretty awful" aside, is what you're asking for some kind
of CLI "follow" mode that doesn't terminate and spits out logs to
stdout perpetually?

Jason

^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: Windows Log Output to Event Viewer or Text File
  2021-10-12 21:40     ` Jason A. Donenfeld
@ 2021-10-13 13:29       ` Frank Wayne
  2021-10-13 18:16         ` Jason A. Donenfeld
  0 siblings, 1 reply; 15+ messages in thread
From: Frank Wayne @ 2021-10-13 13:29 UTC (permalink / raw)
  To: Jason A. Donenfeld; +Cc: WireGuard mailing list


>> On Tue, Oct 12, 2021 at 3:39 PM Frank Wayne <frank.wayne@northwestern.edu> wrote:
>> That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.

> Your "that's pretty awful" aside, is what you're asking for some kind of CLI "follow" mode that doesn't terminate and spits out logs to stdout perpetually?

> Jason

No. I'm not sure that would be much of an improvement.

In Linux (under systemd), kernel logs are accessible in journald, can be forwarded to (r)syslog, and from there to a text file or external syslog or wherever.

In Windows, logs would ideally get sent to Event Logging into a WireGuard log. That way, the user or administrator can use Event Viewer to view the log, forward the log,  or use a collector (like Splunk) to retrieve and aggregate the events. Using a proprietary log makes it difficult to monitor this or any other app.

I'm not sure why WireGuard doesn't use Windows Event Logging. I can't imagine that a proprietary log format would fly in Linux, or even be contemplated. Is there something that precludes the use of Event Logging by WireGuard?

Frank Wayne

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Windows Log Output to Event Viewer or Text File
  2021-10-13 13:29       ` Frank Wayne
@ 2021-10-13 18:16         ` Jason A. Donenfeld
  2021-10-14 17:41           ` Frank Wayne
  0 siblings, 1 reply; 15+ messages in thread
From: Jason A. Donenfeld @ 2021-10-13 18:16 UTC (permalink / raw)
  To: Frank Wayne; +Cc: WireGuard mailing list

Hi Frank,

On Wed, Oct 13, 2021 at 7:30 AM Frank Wayne
<frank.wayne@northwestern.edu> wrote:
> >> On Tue, Oct 12, 2021 at 3:39 PM Frank Wayne <frank.wayne@northwestern.edu> wrote:
> >> That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.
>
> > Your "that's pretty awful" aside, is what you're asking for some kind of CLI "follow" mode that doesn't terminate and spits out logs to stdout perpetually?
>
> > Jason
>
> No. I'm not sure that would be much of an improvement.

Why not? It would make it possible to tail the log more easily and
pipe it into whatever log collection daemon you want. And if that's
indeed still not an improvement, what is the relevance of your
previous mention, "It is only possible to get the last 2048 events and
no way to get just the events since the last update"? I find the tone
of your messages quite abrasive rather than informative. Can you slow
down a bit and try to describe the constraints and requirements of
your system, and then we can try to figure out what would be a good
path toward realizing a good design there? Let's start with: what's
missing in a tail mode that you can pipe to whatever, other than,
"it's not ms event logging"?

> In Linux (under systemd), kernel logs are accessible in journald, can be forwarded to (r)syslog, and from there to a text file or external syslog or wherever.

I'm pretty sure systemd-journald epolls on /dev/kmsg. In other words,
it aggregates logs from different sources. That's not a whole lot
different from a follow mode, right? But why are you talking about
Linux, or about kernel logs for that matter?

> I can't imagine that a proprietary log format would fly in Linux, or even be contemplated.

On Linux, wireguard.ko simply uses printk, like other kernel drivers,
and wg-quick(8) uses stdio, like many userspace programs. But why are
you talking about Linux? What's the relevance?

You've used the word "proprietary" but I think "bespoke" might be more
clear. There are open source implementations in C, C#, and Go in the
git repos, and it should be rather trivial to parse in any other
language too.

> In Windows, logs would ideally get sent to Event Logging into a WireGuard log. That way, the user or administrator can use Event Viewer to view the log, forward the log,  or use a collector (like Splunk) to retrieve and aggregate the events.
> I'm not sure why WireGuard doesn't use Windows Event Logging. Is there something that precludes the use of Event Logging by WireGuard?

Event Logging appears to be rather slow and clunky, and I'm not sure
I'd be too happy about blocking on that during packet events. It's
also very cumbersome to use -- especially for things like crash dumps,
which require a separate process or dll -- and the boilerplate
involved doesn't look very appealing. In contrast, memory mapping a
file, memcpy'ing buffers into it, and getting timestamps by reading
0x7ffe0014, means no calls to libraries or any interactions with other
moving components that might be in an undefined state.

Event Logging might yet be possible to use, though. But it seems to me
that'd be some significant research and development work, to figuring
out how it could work in a lightweight way, and also revisiting the
way wgnt logs things.

If this is something you'd like to work on, I'd be happy to review
patches and read descriptions of a simplified event logging
implementation. You're probably not the only user with this concern,
and in theory I'd be open to considering it, provided there's a way to
make it less clunky than initially meets the eye.

Jason

^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: Windows Log Output to Event Viewer or Text File
  2021-10-13 18:16         ` Jason A. Donenfeld
@ 2021-10-14 17:41           ` Frank Wayne
  2021-10-14 18:40             ` StarBrilliant
  2021-10-14 20:02             ` Jason A. Donenfeld
  0 siblings, 2 replies; 15+ messages in thread
From: Frank Wayne @ 2021-10-14 17:41 UTC (permalink / raw)
  To: Jason A. Donenfeld; +Cc: WireGuard mailing list

Jason,

> Why not? It would make it possible to tail the log more easily and pipe it into whatever log collection daemon you want. And if that's indeed still not an improvement, what is the relevance of your previous mention, "It is only possible to get the last 2048 events and no way to get just the events since the last update"? I find the tone of your messages quite abrasive rather than informative. Can you slow down a bit and try to describe the constraints and requirements of your system, and then we can try to figure out what would be a good path toward realizing a good design there? Let's start with: what's missing in a tail mode that you can pipe to whatever, other than, "it's not ms event logging"?

The requirements are providing a simple, timely, and complete feed of WireGuard events to a log aggregator.

If I could get "the events since the last update", then a scheduled process could run to copy the events to some text log. That data could be aggregated. Even so, it assumes that the number of events that occur between iterations of the scheduled job does not exceed 2048. It makes things more complicated and the chance of losing events remains. As such, this does not fit the requirements well, as it is not simple (it requires the scheduling of a job, the choice of intervals based on the expected rate of events, and may be different for each host), nor timely (since events are delayed by the duration between iterations of the scheduled job), nor complete (since it is prone to event overflow). That's why I don't consider tailing the log *much* of an improvement. It would be an improvement to be sure, just not like what is available in Linux.

Since I have mainly worked on collecting logs from Linux WireGuard hosts where the process of getting the logs into a file is simple, timely, and complete with little effort, my reaction to the Windows version was that, relative to Linux, the logging is "awful" to deal with. I apologize for using that term blithely and that it came across as abrasive. That was not my intention. I did not mean to characterize any component of WireGuard as awful, and certainly the logging for the Windows version works well (as far as I can tell), it just doesn't fit the logging paradigms that Linux or Windows applications use that lend themselves to aggregation.

I have no particular love for Windows Event Log, but it is the de facto logging facility in Windows. If the logs went to a text file, it would be even better, frankly, but there's this thing called Windows Event Log that many (most?) applications use. If WireGuard used it, the requirements would be met. My questions might imply a preference toward Windows Event Log, but it is so owing only to that feature's ubiquity.

> I'm pretty sure systemd-journald epolls on /dev/kmsg. In other words, it aggregates logs from different sources. That's not a whole lot different from a follow mode, right? But why are you talking about Linux, or about kernel logs for that matter?

Not too different from follow mode, but journald (Linux's binary log file solution) *can* forward to rsyslog. Windows WireGuard's binary log file solution can't. I'm talking about Linux because its WireGuard logging meets the requirements. Linux WireGuard is easy to deal with because it uses standard Linux logging components. I'm talking about kernel logs because Linux WireGuard logs are kernel messages; i.e., they have a facility number of 0.

> On Linux, wireguard.ko simply uses printk, like other kernel drivers, and wg-quick(8) uses stdio, like many userspace programs. But why are you talking about Linux? What's the relevance?

Again, I'm talking about Linux because Linux WireGuard uses Linux logging facilities that are easy to work with and Windows WireGuard does not use Windows logging facilities that would be easy to work with.

> You've used the word "proprietary" but I think "bespoke" might be more clear. There are open source implementations in C, C#, and Go in the git repos, and it should be rather trivial to parse in any other language too.

I agree, but it is harder than it could be, and still prone to overflow.

> Event Logging appears to be rather slow and clunky, and I'm not sure I'd be too happy about blocking on that during packet events. It's also very cumbersome to use -- especially for things like crash dumps, which require a separate process or dll -- and the boilerplate involved doesn't look very appealing. In contrast, memory mapping a file, memcpy'ing buffers into it, and getting timestamps by reading 0x7ffe0014, means no calls to libraries or any interactions with other moving components that might be in an undefined state.

> Event Logging might yet be possible to use, though. But it seems to me that'd be some significant research and development work, to figuring out how it could work in a lightweight way, and also revisiting the way wgnt logs things.

I don't pretend to understand the performance or convenience implications of using Windows Event Log instead. If using it would impact WireGuard performance, avoiding it is understandable. Thanks for describing how WireGuard writes the logs, which is both elegant and efficient. I just wish there were an easier way to reliably get the events into a sequential log (file?) of indefinite length.

Thank you for taking the time to respond to my questions. My objective was first to know that I wasn't missing something, and then more about understanding why things are the way they are than about influencing change. I imagine that other people have or will have a need to monitor WireGuard on Windows and would benefit from a simpler, external interface to the log events. I hope you keep this in mind, but for now I will try to make do with what I have.

Frank Wayne


-----Original Message-----
From: Jason A. Donenfeld <Jason@zx2c4.com> 
Sent: Wednesday, 13 October, 2021 13:17
To: Frank Wayne <frank.wayne@northwestern.edu>
Cc: WireGuard mailing list <wireguard@lists.zx2c4.com>
Subject: Re: Windows Log Output to Event Viewer or Text File

Hi Frank,

On Wed, Oct 13, 2021 at 7:30 AM Frank Wayne <frank.wayne@northwestern.edu> wrote:
> >> On Tue, Oct 12, 2021 at 3:39 PM Frank Wayne <frank.wayne@northwestern.edu> wrote:
> >> That's pretty awful. It is only possible to get the last 2048 events and no way to get just the events since the last update. There is no way for an aggregator to simply collect WireGuard logs on Windows.
>
> > Your "that's pretty awful" aside, is what you're asking for some kind of CLI "follow" mode that doesn't terminate and spits out logs to stdout perpetually?
>
> > Jason
>
> No. I'm not sure that would be much of an improvement.

Why not? It would make it possible to tail the log more easily and pipe it into whatever log collection daemon you want. And if that's indeed still not an improvement, what is the relevance of your previous mention, "It is only possible to get the last 2048 events and no way to get just the events since the last update"? I find the tone of your messages quite abrasive rather than informative. Can you slow down a bit and try to describe the constraints and requirements of your system, and then we can try to figure out what would be a good path toward realizing a good design there? Let's start with: what's missing in a tail mode that you can pipe to whatever, other than, "it's not ms event logging"?

> In Linux (under systemd), kernel logs are accessible in journald, can be forwarded to (r)syslog, and from there to a text file or external syslog or wherever.

I'm pretty sure systemd-journald epolls on /dev/kmsg. In other words, it aggregates logs from different sources. That's not a whole lot different from a follow mode, right? But why are you talking about Linux, or about kernel logs for that matter?

> I can't imagine that a proprietary log format would fly in Linux, or even be contemplated.

On Linux, wireguard.ko simply uses printk, like other kernel drivers, and wg-quick(8) uses stdio, like many userspace programs. But why are you talking about Linux? What's the relevance?

You've used the word "proprietary" but I think "bespoke" might be more clear. There are open source implementations in C, C#, and Go in the git repos, and it should be rather trivial to parse in any other language too.

> In Windows, logs would ideally get sent to Event Logging into a WireGuard log. That way, the user or administrator can use Event Viewer to view the log, forward the log,  or use a collector (like Splunk) to retrieve and aggregate the events.
> I'm not sure why WireGuard doesn't use Windows Event Logging. Is there something that precludes the use of Event Logging by WireGuard?

Event Logging appears to be rather slow and clunky, and I'm not sure I'd be too happy about blocking on that during packet events. It's also very cumbersome to use -- especially for things like crash dumps, which require a separate process or dll -- and the boilerplate involved doesn't look very appealing. In contrast, memory mapping a file, memcpy'ing buffers into it, and getting timestamps by reading 0x7ffe0014, means no calls to libraries or any interactions with other moving components that might be in an undefined state.

Event Logging might yet be possible to use, though. But it seems to me that'd be some significant research and development work, to figuring out how it could work in a lightweight way, and also revisiting the way wgnt logs things.

If this is something you'd like to work on, I'd be happy to review patches and read descriptions of a simplified event logging implementation. You're probably not the only user with this concern, and in theory I'd be open to considering it, provided there's a way to make it less clunky than initially meets the eye.

Jason

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Windows Log Output to Event Viewer or Text File
  2021-10-14 17:41           ` Frank Wayne
@ 2021-10-14 18:40             ` StarBrilliant
  2021-10-14 19:40               ` Frank Wayne
  2021-10-14 19:52               ` Jason A. Donenfeld
  2021-10-14 20:02             ` Jason A. Donenfeld
  1 sibling, 2 replies; 15+ messages in thread
From: StarBrilliant @ 2021-10-14 18:40 UTC (permalink / raw)
  To: wireguard

On Thu, Oct 14, 2021, at 17:41, Frank Wayne wrote:
> On Wed, Oct 13, 2021, at 18:16, Jason A. Donenfeld wrote:
> > Event Logging appears to be rather slow and clunky [...]

In fact, Windows Event Logging has two APIs: ETW and WPP.
The ETW API is, indeed, slow and clunky.
However, the WPP API is very high-performance. The trace function in Windows native TCP stack is implemented with WPP. 

If someone like Frank has the time and ability, they could check this MSDN documentation and try to implement it:
https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/wpp-software-tracing
However, I also think this feature is *not* a prioritized task, at least for average users.

I am not sure if I get Jason's idea: Is current Wireguard driver using a ring buffer of 2,048 messages for logging?
I am not sure if it has a notify mechanism: Otherwise, the userspace collector will have to poll the logs. Polling too fast consumes power, polling too slow may skip messages.


Best wishes,
StarBrilliant

^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: Windows Log Output to Event Viewer or Text File
  2021-10-14 18:40             ` StarBrilliant
@ 2021-10-14 19:40               ` Frank Wayne
  2021-10-14 19:52               ` Jason A. Donenfeld
  1 sibling, 0 replies; 15+ messages in thread
From: Frank Wayne @ 2021-10-14 19:40 UTC (permalink / raw)
  To: StarBrilliant, wireguard

> I am not sure if I get Jason's idea: Is current Wireguard driver using a ring buffer of 2,048 messages for logging?

Yes. There is a file (relative to the installation directory) at Data\log.bin. It contains a WORD with "0BADBABE" (base 16, converted to big endian; a signature?), another WORD with something, followed by (in my case) 2050 structures of [a QWORD epoch time (with nanosecond precision) followed by 512 bytes of event text (zero padded)]. When I export the file in the WireGuard UI, it produces a list of 2048 events.

> I am not sure if it has a notify mechanism: Otherwise, the userspace collector will have to poll the logs. Polling too fast consumes power, polling too slow may skip messages.

Hear, hear! Alas, it does not have a notify mechanism.

Frank Wayne

-----Original Message-----
From: WireGuard <wireguard-bounces@lists.zx2c4.com> On Behalf Of StarBrilliant
Sent: Thursday, 14 October, 2021 13:40
To: wireguard@lists.zx2c4.com
Subject: Re: Windows Log Output to Event Viewer or Text File

On Thu, Oct 14, 2021, at 17:41, Frank Wayne wrote:
> On Wed, Oct 13, 2021, at 18:16, Jason A. Donenfeld wrote:
> > Event Logging appears to be rather slow and clunky [...]

In fact, Windows Event Logging has two APIs: ETW and WPP.
The ETW API is, indeed, slow and clunky.
However, the WPP API is very high-performance. The trace function in Windows native TCP stack is implemented with WPP. 

If someone like Frank has the time and ability, they could check this MSDN documentation and try to implement it:
https://urldefense.com/v3/__https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/wpp-software-tracing__;!!Dq0X2DkFhyF93HkjWTBQKhk!CMEOhPSNaRk9va55Sq3P6hrPlsaEZR9cKugdVaKMMSFkQVvmvAwTk-w9efcePl7WnfDRnWcQ$ 
However, I also think this feature is *not* a prioritized task, at least for average users.

I am not sure if I get Jason's idea: Is current Wireguard driver using a ring buffer of 2,048 messages for logging?
I am not sure if it has a notify mechanism: Otherwise, the userspace collector will have to poll the logs. Polling too fast consumes power, polling too slow may skip messages.


Best wishes,
StarBrilliant

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Windows Log Output to Event Viewer or Text File
  2021-10-14 18:40             ` StarBrilliant
  2021-10-14 19:40               ` Frank Wayne
@ 2021-10-14 19:52               ` Jason A. Donenfeld
  1 sibling, 0 replies; 15+ messages in thread
From: Jason A. Donenfeld @ 2021-10-14 19:52 UTC (permalink / raw)
  To: StarBrilliant; +Cc: WireGuard mailing list

On Thu, Oct 14, 2021 at 12:43 PM StarBrilliant <coder@poorlab.com> wrote:
> In fact, Windows Event Logging has two APIs: ETW and WPP.
> The ETW API is, indeed, slow and clunky.
> However, the WPP API is very high-performance. The trace function in Windows native TCP stack is implemented with WPP.

Yes. I have no interest in using binary WPP traces. The kernel driver
now mimics linux's, having the exact same messaged logs in a simple
printk-like buffer.

> If someone like Frank has the time and ability, they could check this MSDN documentation and try to implement it:
> https://docs.microsoft.com/en-us/windows-hardware/drivers/devtest/wpp-software-tracing

Not interested. I won't take patches for that.

> I am not sure if I get Jason's idea: Is current Wireguard driver using a ring buffer of 2,048 messages for logging?

No. Frank is conflating the kernel driver and a simple userspace
service. The userspace service uses a very simple ringlogger format,
with multiple implementations, used for years on different platforms.
The kernel driver doesn't have an on-disk format; it uses a ring
buffer of sorts, but so far that remains irrelevant to this
discussion.

Jason

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Windows Log Output to Event Viewer or Text File
  2021-10-14 17:41           ` Frank Wayne
  2021-10-14 18:40             ` StarBrilliant
@ 2021-10-14 20:02             ` Jason A. Donenfeld
  2021-10-14 21:45               ` Frank Wayne
  1 sibling, 1 reply; 15+ messages in thread
From: Jason A. Donenfeld @ 2021-10-14 20:02 UTC (permalink / raw)
  To: Frank Wayne; +Cc: WireGuard mailing list

Hi Frank,

From what I can see, none of what you've written describes what's
lacking in a potential command that would tail the wireguard log and
print it to standard out, perpetually. At least I couldn't figure it
out from a close read. As I wrote before, you could then pipe this
into event log or into any place else. You spoke more about why
snapshotted logs are problematic, and I'm inclined to agree with you
for a few reasons (though not the one you mentioned). But what I'm
suggesting is a tail mode, that keeps spitting out new logs as they
arrive. Are pipes problematic? Is there no ingestor that reads data
from stdin that would be convenient to you? Where does the tail
approach fall short? Does the message-based approach of event log
clash with the line-based approach of wireguard's unix-like logs? I
would like to fully understand what about this approach fails to meet
your design criteria.

One thing we could pretty easily do is add a "WireGuardEventLogger"
service, {un,}installable with:

> wireguard /installeventlogger
> wireguard /uninstalleventlogger

which would then scoop up the binary log as it grows and spit it out
to event logger, in real time. This wouldn't be too hard to do.
However, I would really like to first understand precisely what the
shortcomings would be in a simpler tail subcommand. That seems a lot
more versatile and simpler to implement too.

Thanks,
Jason

^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: Windows Log Output to Event Viewer or Text File
  2021-10-14 20:02             ` Jason A. Donenfeld
@ 2021-10-14 21:45               ` Frank Wayne
  2021-10-14 21:56                 ` Jason A. Donenfeld
  0 siblings, 1 reply; 15+ messages in thread
From: Frank Wayne @ 2021-10-14 21:45 UTC (permalink / raw)
  To: Jason A. Donenfeld; +Cc: WireGuard mailing list

Jason,

> From what I can see, none of what you've written describes what's lacking in a potential command that would tail the wireguard log and print it to standard out, perpetually. At least I couldn't figure it out from a close read. As I wrote before, you could then pipe this into event log or into any place else. You spoke more about why snapshotted logs are problematic, and I'm inclined to agree with you for a few reasons (though not the one you mentioned). But what I'm suggesting is a tail mode, that keeps spitting out new logs as they arrive. Are pipes problematic? Is there no ingestor that reads data from stdin that would be convenient to you? Where does the tail approach fall short? Does the message-based approach of event log clash with the line-based approach of wireguard's unix-like logs? I would like to fully understand what about this approach fails to meet your design criteria.

I think tail mode could work. It would be easier than parsing the binary file. I can write a script that runs wireguard.exe continuously with the tail option and collect stdout. Maybe it can be made to work universally with minimal effort, so that it works problem-free on whatever WireGuard hosts I push it to. If you write it, I'll try it.

> One thing we could pretty easily do is add a "WireGuardEventLogger"
service, {un,}installable with:

>> wireguard /installeventlogger
>> wireguard /uninstalleventlogger

>which would then scoop up the binary log as it grows and spit it out to event logger, in real time. This wouldn't be too hard to do.
>However, I would really like to first understand precisely what the shortcomings would be in a simpler tail subcommand. That seems a lot more versatile and simpler to implement too.

The service approach is interesting, too. It would be simpler to ingest.

In terms of shortcomings, consider the typical organization that wants to collect logs with (for example) Splunk.

They want to collect Exchange Mail logs. The Splunk administrator writes a monitor stanza for each Event Log and each directory with text logs, pushes it to the servers, and log events begin pouring in to indexers.

They want to collect Postfix logs. Same process, different OS.

They want to collect appliance syslogs. Set up forwarding to a Splunk listener and done.

They want to collect WireGuard logs. Well, now that's different, but only on Windows. Here, you need to write a script that runs software external to Splunk, and runs it continuously, and collects the output. Will every host have WireGuard installed? Forever? The script will have to check that the product is installed or else continuously generate errors. Is wireguard.exe in the PATH? For the user that Splunk runs under? Does that user have permissions to the WireGuard program directory? Should the script check the registry for the executable's location if it's not in the path? Can we run that script on endpoints without checking each team's security policy regarding in-house software running executables outside of its scope?

I don't know what all the shortcomings of a tail subcommand are, but these questions come to mind even before any development is started. So is tail more versatile and simpler? For whom?

That said, I think it would be a good start.

Frank Wayne

-----Original Message-----
From: Jason A. Donenfeld <Jason@zx2c4.com> 
Sent: Thursday, 14 October, 2021 15:03
To: Frank Wayne <frank.wayne@northwestern.edu>
Cc: WireGuard mailing list <wireguard@lists.zx2c4.com>
Subject: Re: Windows Log Output to Event Viewer or Text File

Hi Frank,

From what I can see, none of what you've written describes what's lacking in a potential command that would tail the wireguard log and print it to standard out, perpetually. At least I couldn't figure it out from a close read. As I wrote before, you could then pipe this into event log or into any place else. You spoke more about why snapshotted logs are problematic, and I'm inclined to agree with you for a few reasons (though not the one you mentioned). But what I'm suggesting is a tail mode, that keeps spitting out new logs as they arrive. Are pipes problematic? Is there no ingestor that reads data from stdin that would be convenient to you? Where does the tail approach fall short? Does the message-based approach of event log clash with the line-based approach of wireguard's unix-like logs? I would like to fully understand what about this approach fails to meet your design criteria.

One thing we could pretty easily do is add a "WireGuardEventLogger"
service, {un,}installable with:

> wireguard /installeventlogger
> wireguard /uninstalleventlogger

which would then scoop up the binary log as it grows and spit it out to event logger, in real time. This wouldn't be too hard to do.
However, I would really like to first understand precisely what the shortcomings would be in a simpler tail subcommand. That seems a lot more versatile and simpler to implement too.

Thanks,
Jason

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Windows Log Output to Event Viewer or Text File
  2021-10-14 21:45               ` Frank Wayne
@ 2021-10-14 21:56                 ` Jason A. Donenfeld
  2021-10-15 13:25                   ` Frank Wayne
  0 siblings, 1 reply; 15+ messages in thread
From: Jason A. Donenfeld @ 2021-10-14 21:56 UTC (permalink / raw)
  To: Frank Wayne; +Cc: WireGuard mailing list

Hi Frank,

On Thu, Oct 14, 2021 at 3:45 PM Frank Wayne
<frank.wayne@northwestern.edu> wrote:
> The service approach is interesting, too. It would be simpler to ingest.

I started writing this, and got something basically working, but got a
bit tripped up in annoying lifetime issues of registering an event log
"source" in the registry, and the boggling configurability there. I'm
kind of shying away from it after an initial stab...

To answer the more concrete questions about a tail approach:

> Will every host have WireGuard installed? Forever?

This is a tougie, I guess in the same way that scooping up
non-streamed file-based logs are: at some point you have to do a sweep
to see if there are new files to grab, or in this case, if wireguard
has been installed. So on one hand, "polling" is pretty gnarly, but on
the other hand, you do that anyway for file-based logs I imagine.
There are probably other SCM-based or MSI-event based ways of doing
this without polling that are more complicated.

Does Splunk have any condition logic like, "do this collection routine
if file F exists; otherwise wait to do it until it exists"? If that
kind of thing is built in, then you're done. If not I agree this is an
annoying point.

> Is wireguard.exe in the PATH? For the user that Splunk runs under?

Yes. It's added to the system PATH.

> Should the script check the registry for the executable's location if it's not in the path?

No. The installer always sets PATH.

> Does that user have permissions to the WireGuard program directory?

Hmm. Here indeed is where the granular decoupled permission system of
Event Log comes in handy, I suppose. But in theory you should be able
to do the same for wireguard's log:
"%PROGRAMFILES%\wireguard\data\log.bin" is just a file path like any
other, and you can adjust its permissions accordingly. By default its
parent directory is O:SYG:SYD:PAI(A;OICI;FA;;;SY)(A;OICI;FA;;;BA), but
you could set a more particular ACL for log.bin itself.

> Can we run that script on endpoints without checking each team's security policy regarding in-house software running executables outside of its scope?

I imagine probably so in the sense that wireguard.exe is already
executed several times in a few different funny roles, so things might
already be sufficiently permissive. Have you seen any wireguard.exe
policies being passed around places? If you've got a link, I'd be
curious to see what people are doing.

Jason

^ permalink raw reply	[flat|nested] 15+ messages in thread

* RE: Windows Log Output to Event Viewer or Text File
  2021-10-14 21:56                 ` Jason A. Donenfeld
@ 2021-10-15 13:25                   ` Frank Wayne
  2021-10-26 10:05                     ` Jason A. Donenfeld
  0 siblings, 1 reply; 15+ messages in thread
From: Frank Wayne @ 2021-10-15 13:25 UTC (permalink / raw)
  To: Jason A. Donenfeld; +Cc: WireGuard mailing list

Jason,

Please understand that the questions, although real concerns, are rhetorical in the context of this discussion. I know the answers to them, but they are intended to serve as an example of why I consider the tail option only a partial solution. This is not to dissuade you from implementing it; it is merely a response to your incomprehension of "why it fails to meet" my expectations of convenience, etc.
 
Frank Wayne

^ permalink raw reply	[flat|nested] 15+ messages in thread

* Re: Windows Log Output to Event Viewer or Text File
  2021-10-15 13:25                   ` Frank Wayne
@ 2021-10-26 10:05                     ` Jason A. Donenfeld
  0 siblings, 0 replies; 15+ messages in thread
From: Jason A. Donenfeld @ 2021-10-26 10:05 UTC (permalink / raw)
  To: Frank Wayne; +Cc: WireGuard mailing list

`wireguard /dumplog /tail | select` should now work in 0.5.1. Let me
know how that goes with your log ingestor.

Jason

^ permalink raw reply	[flat|nested] 15+ messages in thread

end of thread, other threads:[~2021-10-26 10:06 UTC | newest]

Thread overview: 15+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-10-08 20:50 Windows Log Output to Event Viewer or Text File Frank Wayne
2021-10-08 22:01 ` Jason A. Donenfeld
2021-10-12 21:39   ` Frank Wayne
2021-10-12 21:40     ` Jason A. Donenfeld
2021-10-13 13:29       ` Frank Wayne
2021-10-13 18:16         ` Jason A. Donenfeld
2021-10-14 17:41           ` Frank Wayne
2021-10-14 18:40             ` StarBrilliant
2021-10-14 19:40               ` Frank Wayne
2021-10-14 19:52               ` Jason A. Donenfeld
2021-10-14 20:02             ` Jason A. Donenfeld
2021-10-14 21:45               ` Frank Wayne
2021-10-14 21:56                 ` Jason A. Donenfeld
2021-10-15 13:25                   ` Frank Wayne
2021-10-26 10:05                     ` Jason A. Donenfeld

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).