* Re: [Caml-list] IPv6 packet parsing
2013-10-18 12:20 ` Anil Madhavapeddy
@ 2013-10-18 12:26 ` rixed
2013-10-18 13:52 ` Gerd Stolpmann
` (2 subsequent siblings)
3 siblings, 0 replies; 18+ messages in thread
From: rixed @ 2013-10-18 12:26 UTC (permalink / raw)
To: OCaml Mailing List
> We have a number of use-cases that run OCaml in kernel mode, directly
> operating on packets read from a network driver that's also written in
> OCaml. Bigarrays are used as the mechanism for passing around externally
> allocated memory (i.e. network card buffers) directly, whereas inspecting
> them with a string-based Bigarray requires an expensive data copy.
Ok. We don't play in the same league :-)
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Caml-list] IPv6 packet parsing
2013-10-18 12:20 ` Anil Madhavapeddy
2013-10-18 12:26 ` rixed
@ 2013-10-18 13:52 ` Gerd Stolpmann
2013-10-18 14:13 ` rixed
2013-10-18 14:20 ` Markus Mottl
2013-11-06 13:57 ` Richard W.M. Jones
3 siblings, 1 reply; 18+ messages in thread
From: Gerd Stolpmann @ 2013-10-18 13:52 UTC (permalink / raw)
To: Anil Madhavapeddy; +Cc: rixed, OCaml Mailing List
[-- Attachment #1: Type: text/plain, Size: 3342 bytes --]
Am Freitag, den 18.10.2013, 13:20 +0100 schrieb Anil Madhavapeddy:
> On Fri, Oct 18, 2013 at 02:16:12PM +0200, rixed@happyleptic.org wrote:
> > -[ Fri, Oct 18, 2013 at 12:59:55PM +0100, Anil Madhavapeddy ]----
> > > One feature I'd really like to see in Bitstring is support for Bigarray,
> > > since that avoids a copy into the OCaml heap and lets us do quite high
> > > performance parsing. If I remember right, there was a patch on the
> > > Bitstring issue tracker, but it wasn't parameterised (so it's either
> > > Bistring+string or Bitstring+bigarray, which isn't ideal).
> >
> > Pardon my lack of familiarity with bigarrays, but I can't see what's the
> > difference between copying packets from pcap ring buffer into a bigarray
> > or into a string. Or do you mean using Bigarray.map_file on the whole
> > raw ring buffer and handle it without pcap help?
Without knowing details: maybe no copy is required at all? The pcap ring
buffer could be directly wrapped as Bigarray.
> We have a number of use-cases that run OCaml in kernel mode, directly
> operating on packets read from a network driver that's also written in
> OCaml. Bigarrays are used as the mechanism for passing around externally
> allocated memory (i.e. network card buffers) directly, whereas inspecting
> them with a string-based Bigarray requires an expensive data copy.
>
> See: http://anil.recoil.org/papers/2013-asplos-mirage.pdf
> or http://www.openmirage.org
For similar reasons, I also added some Bigarray functions to Ocamlnet:
http://projects.camlcity.org/projects/dl/ocamlnet-3.7.3/doc/html-main/Netsys_mem.html
If you look at the stub behind e.g. Unix.read, you'll see that the data
is first read into an internal unaligned buffer, and then copied to the
string buffer. This means usually two copies of the data: one from the
kernel buffer to the internal buffer, and one from there to the string.
If you use a Bigarray instead the internal buffer becomes superfluous:
Bigarrays are malloc'ed memory, and cannot be moved by the GC. Hence,
you can invoke the read() syscall directly with the Bigarray as buffer.
If you additionally ensure that the Bigarray is page-aligned, the kernel
can sometimes even avoid copying at all (though only some OS seem to
implement such a strategy, as changing the page mapping or doing some
direct I/O can be more costly than copying).
Another advantage here is that you can freely choose the size of the
buffer (Unix.read et al use fixed-size 64K for the internal buffer).
Also you can allocate the buffer in a shared area.
Ocamlnet now prefers Bigarrays as primary buffers where reasonable, and
where a speedup (or lower CPU consumption) can be expected. E.g. The
HTTP client first reads data into a bigarray, splits the header there
into lines (which are then normal strings again), and gathers the data
chunks from the HTTP body (which can be strings or Bigarrays, at the
user's choice).
Gerd
--
------------------------------------------------------------
Gerd Stolpmann, Darmstadt, Germany gerd@gerd-stolpmann.de
My OCaml site: http://www.camlcity.org
Contact details: http://www.camlcity.org/contact.html
Company homepage: http://www.gerd-stolpmann.de
------------------------------------------------------------
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 490 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Caml-list] IPv6 packet parsing
2013-10-18 13:52 ` Gerd Stolpmann
@ 2013-10-18 14:13 ` rixed
2013-10-18 14:40 ` Stéphane Glondu
0 siblings, 1 reply; 18+ messages in thread
From: rixed @ 2013-10-18 14:13 UTC (permalink / raw)
To: OCaml Mailing List
-[ Fri, Oct 18, 2013 at 03:52:07PM +0200, Gerd Stolpmann ]----
> > > Or do you mean using Bigarray.map_file on the whole
> > > raw ring buffer and handle it without pcap help?
>
> Without knowing details: maybe no copy is required at all? The pcap ring
> buffer could be directly wrapped as Bigarray.
Yes that's what I was trying to say.
Unfortunately, such implementation would be hard to maintain
(Linux capture ring buffer have already several implementations, to say
nothing about supporting other OSes).
Ideally, I'd like to pass const pointers to packet bytes provided by
libpcap to user callbacks up to bitstring through OCaml (say via a
sufficiently large int type?).
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Caml-list] IPv6 packet parsing
2013-10-18 12:20 ` Anil Madhavapeddy
2013-10-18 12:26 ` rixed
2013-10-18 13:52 ` Gerd Stolpmann
@ 2013-10-18 14:20 ` Markus Mottl
2013-10-18 17:23 ` Paul Pelzl
2013-10-18 17:52 ` rixed
2013-11-06 13:57 ` Richard W.M. Jones
3 siblings, 2 replies; 18+ messages in thread
From: Markus Mottl @ 2013-10-18 14:20 UTC (permalink / raw)
To: Anil Madhavapeddy; +Cc: Chapi Chapo, OCaml Mailing List
On Fri, Oct 18, 2013 at 8:20 AM, Anil Madhavapeddy <anil@recoil.org> wrote:
> We have a number of use-cases that run OCaml in kernel mode, directly
> operating on packets read from a network driver that's also written in
> OCaml. Bigarrays are used as the mechanism for passing around externally
> allocated memory (i.e. network card buffers) directly, whereas inspecting
> them with a string-based Bigarray requires an expensive data copy.
>
> See: http://anil.recoil.org/papers/2013-asplos-mirage.pdf
> or http://www.openmirage.org
It may also be of interest to investigate the use of "splice" and
"tee" on Linux to avoid frequent switches between user- and kernel
mode and to duplicate I/O streams without having to copy the data
(zero copy). There is an implementation in
Core_extended.Extended_Linux in the core_extended OPAM package.
Regards,
Markus
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Caml-list] IPv6 packet parsing
2013-10-18 14:20 ` Markus Mottl
@ 2013-10-18 17:23 ` Paul Pelzl
2013-10-18 17:52 ` rixed
1 sibling, 0 replies; 18+ messages in thread
From: Paul Pelzl @ 2013-10-18 17:23 UTC (permalink / raw)
To: OCaml Mailing List
On Fri, Oct 18, 2013 at 9:20 AM, Markus Mottl <markus.mottl@gmail.com> wrote:
> It may also be of interest to investigate the use of "splice" and
> "tee" on Linux to avoid frequent switches between user- and kernel
> mode and to duplicate I/O streams without having to copy the data
> (zero copy). There is an implementation in
> Core_extended.Extended_Linux in the core_extended OPAM package.
Before investigating too far and getting disappointed by mediocre
performance, it's worth pointing out that SPLICE_F_MOVE support got
pulled from the kernel quite some time ago [1] and still hasn't
returned [2]. It does look like someone is actively working on it,
however [3].
Paul
[1] https://lkml.org/lkml/2008/9/23/115
[2] http://lxr.free-electrons.com/ident?v=3.11;i=SPLICE_F_MOVE
[3] https://lkml.org/lkml/2013/10/7/532
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Caml-list] IPv6 packet parsing
2013-10-18 14:20 ` Markus Mottl
2013-10-18 17:23 ` Paul Pelzl
@ 2013-10-18 17:52 ` rixed
2013-10-18 20:09 ` Markus Mottl
1 sibling, 1 reply; 18+ messages in thread
From: rixed @ 2013-10-18 17:52 UTC (permalink / raw)
To: OCaml Mailing List
-[ Fri, Oct 18, 2013 at 10:20:19AM -0400, Markus Mottl ]----
> It may also be of interest to investigate the use of "splice" and
> "tee" on Linux to avoid frequent switches between user- and kernel
> mode and to duplicate I/O streams without having to copy the data
> (zero copy).
I feel sorry to be clueless twice in a row, but how are these new syscalls
supposed to help sniffing & parsing packets?
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Caml-list] IPv6 packet parsing
2013-10-18 17:52 ` rixed
@ 2013-10-18 20:09 ` Markus Mottl
0 siblings, 0 replies; 18+ messages in thread
From: Markus Mottl @ 2013-10-18 20:09 UTC (permalink / raw)
To: Chapi Chapo; +Cc: OCaml Mailing List
On Fri, Oct 18, 2013 at 1:52 PM, <rixed@happyleptic.org> wrote:
> I feel sorry to be clueless twice in a row, but how are these new syscalls
> supposed to help sniffing & parsing packets?
There are several ways in which they could be beneficial. In fact,
you'll usually have to combine them. Packet sniffers often need to
analyze some things in realtime while also logging all sniffed data to
a file for later reference.
"splice" allows you to move data from one descriptor to another
without ever having to copy it to user land in between. "tee" allows
you to cheaply duplicate the content of one pipe to another. E.g. you
can thus parse packets from one pipe for realtime analysis while
splicing duplicated (via "tee") data to a file.
The packet sniffer might also have another, non-verbatim output stream
that contains only filtered parts of the communication. You could use
"vmsplice" to send data contained in your user buffers to a pipe
(without copying!) and then splice the data to a file while e.g. also
duplicating it to a socket, etc.
Anyway, I agree with Paul that such performance tweaks may often
disappoint. Other things (e.g. batching of I/O transactions to
minimize system calls) are usually much more important.
Regards,
Markus
--
Markus Mottl http://www.ocaml.info markus.mottl@gmail.com
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [Caml-list] IPv6 packet parsing
2013-10-18 12:20 ` Anil Madhavapeddy
` (2 preceding siblings ...)
2013-10-18 14:20 ` Markus Mottl
@ 2013-11-06 13:57 ` Richard W.M. Jones
3 siblings, 0 replies; 18+ messages in thread
From: Richard W.M. Jones @ 2013-11-06 13:57 UTC (permalink / raw)
To: Anil Madhavapeddy; +Cc: rixed, OCaml Mailing List
On Fri, Oct 18, 2013 at 01:20:18PM +0100, Anil Madhavapeddy wrote:
> On Fri, Oct 18, 2013 at 02:16:12PM +0200, rixed@happyleptic.org wrote:
> > -[ Fri, Oct 18, 2013 at 12:59:55PM +0100, Anil Madhavapeddy ]----
> > > One feature I'd really like to see in Bitstring is support for Bigarray,
> > > since that avoids a copy into the OCaml heap and lets us do quite high
> > > performance parsing. If I remember right, there was a patch on the
> > > Bitstring issue tracker, but it wasn't parameterised (so it's either
> > > Bistring+string or Bitstring+bigarray, which isn't ideal).
> >
> > Pardon my lack of familiarity with bigarrays, but I can't see what's the
> > difference between copying packets from pcap ring buffer into a bigarray
> > or into a string. Or do you mean using Bigarray.map_file on the whole
> > raw ring buffer and handle it without pcap help?
>
> We have a number of use-cases that run OCaml in kernel mode, directly
> operating on packets read from a network driver that's also written in
> OCaml. Bigarrays are used as the mechanism for passing around externally
> allocated memory (i.e. network card buffers) directly, whereas inspecting
> them with a string-based Bigarray requires an expensive data copy.
I'm late to the party here, but could you just Obj.magic the bitstring
string pointer (ie. the first in the 3-tuple) to point at the Bigarray
data?
Note that bitstring doesn't really move the string around (well, I
don't want to claim that never happens, but it shouldn't happen in the
common cases AFAIK).
Rich.
--
Richard Jones
Red Hat
^ permalink raw reply [flat|nested] 18+ messages in thread