From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/3654 Path: news.gmane.org!not-for-mail From: LM Newsgroups: gmane.linux.lib.musl.general Subject: regex libs (was Re: [musl] embedded newbies site.) Date: Tue, 16 Jul 2013 11:10:34 -0400 Message-ID: Reply-To: musl@lists.openwall.com NNTP-Posting-Host: plane.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary=089e01681640f6ebb904e1a260fa X-Trace: ger.gmane.org 1373987444 19581 80.91.229.3 (16 Jul 2013 15:10:44 GMT) X-Complaints-To: usenet@ger.gmane.org NNTP-Posting-Date: Tue, 16 Jul 2013 15:10:44 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: musl-return-3658-gllmg-musl=m.gmane.org@lists.openwall.com Tue Jul 16 17:10:48 2013 Return-path: Envelope-to: gllmg-musl@plane.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by plane.gmane.org with smtp (Exim 4.69) (envelope-from ) id 1Uz6tq-0003IF-W5 for gllmg-musl@plane.gmane.org; Tue, 16 Jul 2013 17:10:47 +0200 Original-Received: (qmail 28398 invoked by uid 550); 16 Jul 2013 15:10:46 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: Original-Received: (qmail 28375 invoked from network); 16 Jul 2013 15:10:46 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:date:message-id:subject:from:to:content-type; bh=EZsktqL3LFiFEhO458gJKTp0DMuEcvkQAHYn623Krns=; b=XA3il/oZsj728dwnIvv+46eUgM+AibseLLXupYW9gqautfzV8LJ/NUEz+oUnWtljZE p6AG86fh5dz20DkyASZIhyK/dPXoldzKn9bE/Ot6KqYHafrjPXPBkiqTfCH5l+le3qAi 5Og0japdvzv+esTt8ufddeNH/OhISB6g8ZRsKnuXYIXurm2gZTkz52ksFRtCWOT+OTYK SPq7olMlCxk3P1HPWKiqJYzJU819TYztG148jslvTkxjHaXooa6JBpWJKN2sDL5kPtlY a97sCT+Ylfcl3DKQDvDUq14scmm3faES81pdFaH9TolT4MlpCUAB6g1tUQmnpHixV5gZ sElg== X-Received: by 10.15.98.3 with SMTP id bi3mr1935101eeb.124.1373987434576; Tue, 16 Jul 2013 08:10:34 -0700 (PDT) Xref: news.gmane.org gmane.linux.lib.musl.general:3654 Archived-At: --089e01681640f6ebb904e1a260fa Content-Type: text/plain; charset=ISO-8859-1 On Tue, Jul 16, 2013 at 10:00 AM, Rich Felker wrote: > The whole concept of regular expressions is that they're regular, > meaning they're matchable in O(n) time with O(1) space. PCRE (the > implementation) uses backtracking for everything, giving it > exponentially-bad performance (JIT cannot fix this), and PCRE (the > language) has a lot of features that are fundamentally not regular and > thus can't be implemented efficiently. Also, the behavior of some of > the features (e.g. greedy vs non-greedy matching) were not designed > intentionally but just arose out of the backtracking implementation, > and thus don't make a lot of sense unless you think from the > standpoint of such an implementation. > Went back and rechecked the documentation ( http://www.pcre.org/readme.txt). You're both right, PCRE is offering the Perl regular expressions implementation even when one uses the pcreposix interface. Would have been nice if they offered actual regular expressions handling if you only want to use the POSIX compatible part of the interface. So what are some good regex library solutions? I'm also wondering if there are some good cross-platform portable library solutions (or if PCRE is the best pattern matching solution from a portability standpoint even if it's not strictly regex compatible). There's http://code.google.com/p/re2/ , but I've read some issues with its performance in a few web articles and didn't have much luck with portability to non-Linux platforms. There's the glibc solution: http://sourceforge.net/p/mingw/regex/ci/master/tree/ There's TRE ( https://github.com/laurikari/tre/), which some BSD systems want to use to create their grep ( https://wiki.freebsd.org/BSDgrep ). There's the Oniguruma library ( http://www.geocities.jp/kosako3/oniguruma/ ). There's Henry Spencer's regex ( http://www.arglist.com/regex ). That looks promising for portability. There's http://re2c.org/ Further searching also turns up http://tiny-rex.sourceforge.net/ which may have the same issues as PCRE. ICU seems to offer regex code ( http://userguide.icu-project.org/strings/regexp), probably same issue as PCRE. (Just my opinion, but ICU seems to do a lot of stuff for just one library.) There's BOOST's regex ( http://www.boost.org/doc/libs/1_54_0/libs/regex/doc/ ) which a lot of web sites recommend, but I've just never been a fan of the BOOST libraries. The Heirloom Project has regex code ( http://heirloom.sourceforge.net/ ). Have I missed any other interesting solutions? Sounds like I need to better clarify between regex pattern matching libraries and pattern matching libraries on the musl wiki's alternative library page. If you recommend any of the above libraries or possibly others and think certain implementations would be useful to others, let me know and I'll add the links to the wiki. I haven't really added anything for pattern-matching libraries beyond benchmark information. Thanks. Sincerely, Laura http://www.distasis.com --089e01681640f6ebb904e1a260fa Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
On Tue, Jul 16, 2013 at 10:00 AM, Rich Felker <= dalias@aerifal.cx> wrote:
The whole concept of regular expressions is that they're regular,
meaning they're matchable in O(n) time with O(1) space. PCRE (the
implementation) uses backtracking for everything, giving it
exponentially-bad performance (JIT cannot fix this), and PCRE (the
language) has a lot of features that are fundamentally not regular and
thus can't be implemented efficiently. Also, the behavior of some of the features (e.g. greedy vs non-greedy matching) were not designed
intentionally but just arose out of the backtracking implementation,
and thus don't make a lot of sense unless you think from the
standpoint of such an implementation.

Went back a= nd rechecked the documentation ( http://www.pcre.org/readme.txt ).=A0 You're both= right, PCRE is offering the Perl regular expressions implementation even w= hen one uses the pcreposix interface.=A0 Would have been nice if they offer= ed actual regular expressions handling if you only want to use the POSIX co= mpatible part of the interface.

So what are some good regex library solutions?=A0 I'm also wonderin= g if there are some good cross-platform portable library solutions (or if P= CRE is the best pattern matching solution from a portability standpoint eve= n if it's not strictly regex compatible).

There's http://code.google.com/p/re2/ , but I've read some issues with its= performance in a few web articles and didn't have much luck with porta= bility to non-Linux platforms.=A0 There's the glibc solution:=A0 http://sourcefor= ge.net/p/mingw/regex/ci/master/tree/=A0 There's TRE ( https://github.com/laurikari/tre/), wh= ich some BSD systems want to use to create their grep ( https://wiki.freebsd.org/BSDgrep ).=A0 There&= #39;s the Oniguruma library ( http://www.geocities.jp/kosako3/oniguruma/ ). There's Hen= ry Spencer's regex ( http://ww= w.arglist.com/regex ).=A0 That looks promising for portability.=A0 Ther= e's http://re2c.org/ Further searching= also turns up http://tiny-rex= .sourceforge.net/ which may have the same issues as PCRE.=A0 ICU seems = to offer regex code (http://userguide.icu-project.org/strings/regexp), probably same i= ssue as PCRE.=A0 (Just my opinion, but ICU seems to do a lot of stuff for j= ust one library.)=A0 There's BOOST's regex ( http://www.boost.org/doc/libs/1_= 54_0/libs/regex/doc/ ) which a lot of web sites recommend, but I've= just never been a fan of the BOOST libraries.=A0 The Heirloom Project has = regex code ( http://heirloom.s= ourceforge.net/ ).=A0 Have I missed any other interesting solutions?
Sounds like I need to better clarify between regex pattern matching lib= raries and pattern matching libraries on the musl wiki's alternative li= brary page.=A0 If you recommend any of the above libraries or possibly othe= rs and think certain implementations would be useful to others, let me know= and I'll add the links to the wiki.=A0 I haven't really added anyt= hing for pattern-matching libraries beyond benchmark information.

Thanks.

Sincerely,
Laura
http://www.distasis.com

--089e01681640f6ebb904e1a260fa--