RFE: .so filters

[Jason at zx2c4.com (Jason A. Donenfeld)]

On Thu, Jan 9, 2014 at 11:58 PM, John Keeping <john at keeping.me.uk> wrote:
>
> That interface doesn't really match the way the current filters work.

Yes, hence the post.

>  We could swap out write with a function pointer
> into the filter, but I don't think we guarantee that all of the data is
> written in one go which makes life harder for filter writers (although
> for simple cases like author info we probably could guarantee to write
> it all at once).

I'm actually contemplating doing just this; it would be the only sane
way to reap the benefits of shared objects. Stdio will ensure that the
writes occur once per line, since stdout has line buffering enabled.
This is reasonable enough behavior.

> If we allow filters to act incrementally, then we can just leave the
> filter running and swap it in or out when required.  That would require
> a single dup2 to make it work the same way that the filters currently
> work.

So the other thing I've been considering is doing something like this,
but without shared objects. The idea is -- we exec the filter once.
The first N lines (or instead of \n we use \0 - same difference)
contain the arguments. Then the filter runs, using those arguments,
and does its thing per usual. At the end, however, it does not exit.
Instead of waitpid()ing on it in close filter, we SIGSTOP it, put the
fds back in place, etc. Then the next time that filter is called, we
SIGCONT it, it reads the first N lines as arguments again, and so
forth. I'm most tempted to go with this approach at the moment.

> I think scripts using stdin+stdout
> are a much simpler interface and provides a much lower barrier to entry,
> not least because the range of languages that can be used to implement
> the filters is so much greater.

Yea. I'm leaning toward the SIGSTOP solution with normal scripts... We
could do a megafast shared object plugin architecture, but it seems
like that's needlessly complicated right now...