cgit segfaults

cgit segfaults

[rworkman]

We're running cgit-1.1 with git-2.10.4 at https://git.slackbuilds.org and are seeing
some reproducible segfaults.

root at git:/var/log# dmesg -T
[Wed Aug 16 01:14:23 2017] traps: cgit.cgi[2210] general protection ip:4515bd sp:7ffd787a9470 error:0 in cgit.cgi[400000+103000]

This can be reliably triggered (i.e. every time) with at least one particular link (I'll share it 
privately with cgit devs, but since I don't know if there's any security impact, I'm not going
to put it out on the list as yet).

I've applied 1b4ef6783a71962f8b5da3a23f283 and c699866699411346c5dba4064575
from git master since they appeared to address some segfaults, but apparently they were
unrelated to whatever it is that we're seeing. 

Aside from (obviously) sharing the reproducer, any tips on debugging this? We of course
have a strong preference for debugging tips that don't impact services on the machine,
but if needed, we'll do what we have to do...

-RW
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20170816/e4d52b6c/attachment.asc>

cgit segfaults

[john]

On Wed, Aug 16, 2017 at 01:26:52AM -0500, Robby Workman wrote:
> We're running cgit-1.1 with git-2.10.4 at https://git.slackbuilds.org and are seeing
> some reproducible segfaults.
> 
> root at git:/var/log# dmesg -T
> [Wed Aug 16 01:14:23 2017] traps: cgit.cgi[2210] general protection ip:4515bd sp:7ffd787a9470 error:0 in cgit.cgi[400000+103000]
> 
> This can be reliably triggered (i.e. every time) with at least one particular link (I'll share it 
> privately with cgit devs, but since I don't know if there's any security impact, I'm not going
> to put it out on the list as yet).
> 
> I've applied 1b4ef6783a71962f8b5da3a23f283 and c699866699411346c5dba4064575
> from git master since they appeared to address some segfaults, but apparently they were
> unrelated to whatever it is that we're seeing. 
> 
> Aside from (obviously) sharing the reproducer, any tips on debugging this? We of course
> have a strong preference for debugging tips that don't impact services on the machine,
> but if needed, we'll do what we have to do...

You can run cgit from the command line with your config and the URL
using something like:

	CGIT_CONFIG=/path/to/cgitrc QUERY_STRING=url=cgit/repo/... cgit

This is what the tests do in tests/setup.sh::cgit_url().

That should allow you to build a debug binary and reproduce under that
without a webserver involved, which means you can run under gdb or
valgrind.

cgit segfaults

[rworkman]

On Wed, 16 Aug 2017 09:36:28 +0100
John Keeping <john at keeping.me.uk> wrote:

> On Wed, Aug 16, 2017 at 01:26:52AM -0500, Robby Workman wrote:
> > We're running cgit-1.1 with git-2.10.4 at
> > https://git.slackbuilds.org and are seeing some reproducible
> > segfaults.
> > 
> > root at git:/var/log# dmesg -T
> > [Wed Aug 16 01:14:23 2017] traps: cgit.cgi[2210] general protection
> > ip:4515bd sp:7ffd787a9470 error:0 in cgit.cgi[400000+103000]
> > 
> > This can be reliably triggered (i.e. every time) with at least one
> > particular link (I'll share it privately with cgit devs, but since
> > I don't know if there's any security impact, I'm not going to put
> > it out on the list as yet).
> > 
> > I've applied 1b4ef6783a71962f8b5da3a23f283 and
> > c699866699411346c5dba4064575 from git master since they appeared to
> > address some segfaults, but apparently they were unrelated to
> > whatever it is that we're seeing. 
> > 
> > Aside from (obviously) sharing the reproducer, any tips on
> > debugging this? We of course have a strong preference for debugging
> > tips that don't impact services on the machine, but if needed,
> > we'll do what we have to do...  
> 
> You can run cgit from the command line with your config and the URL
> using something like:
> 
> 	CGIT_CONFIG=/path/to/cgitrc QUERY_STRING=url=cgit/repo/...
> cgit
> 
> This is what the tests do in tests/setup.sh::cgit_url().
> 
> That should allow you to build a debug binary and reproduce under that
> without a webserver involved, which means you can run under gdb or
> valgrind.


Okay, that's helpful - thanks! I've got something that seems to point
at git's pathspec.c (we're building with (and using on the machine)
git-2.10.4 currently), but I have no idea where to go from here. 
This is the gdb output:

(gdb) run
Starting program: /var/www/cgi-bin/cgit.cgi 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
Content-Type: text/plain; charset=UTF-8
Content-Disposition: inline; filename="82746b4b48cec68acdbb5b7a5ad841b1a21872af..65131f01e212203fbde61d3074640651a02cb6e0.patch"
Last-Modified: Thu, 24 Aug 2017 06:08:13 GMT
Expires: Thu, 24 Aug 2017 06:13:13 GMT


Program received signal SIGSEGV, Segmentation fault.
0x00000000004515bd in prefix_pathspec (elt=0x6234623634373238 <error: Cannot access memory at address 0x6234623634373238>, prefixlen=0, prefix=0x0, flags=0, 
    raw=0x77a138, p_short_magic=<synthetic pointer>, item=0x77a808) at pathspec.c:149
149		if (elt[0] != ':' || literal_global ||
(gdb) 

-RW
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20170824/77b7aadc/attachment.asc>

cgit segfaults

[john]

On Thu, Aug 24, 2017 at 01:18:20AM -0500, Robby Workman wrote:
> On Wed, 16 Aug 2017 09:36:28 +0100
> John Keeping <john at keeping.me.uk> wrote:
> 
> > On Wed, Aug 16, 2017 at 01:26:52AM -0500, Robby Workman wrote:
> > > We're running cgit-1.1 with git-2.10.4 at
> > > https://git.slackbuilds.org and are seeing some reproducible
> > > segfaults.
> > > 
> > > root at git:/var/log# dmesg -T
> > > [Wed Aug 16 01:14:23 2017] traps: cgit.cgi[2210] general protection
> > > ip:4515bd sp:7ffd787a9470 error:0 in cgit.cgi[400000+103000]
> > > 
> > > This can be reliably triggered (i.e. every time) with at least one
> > > particular link (I'll share it privately with cgit devs, but since
> > > I don't know if there's any security impact, I'm not going to put
> > > it out on the list as yet).
> > > 
> > > I've applied 1b4ef6783a71962f8b5da3a23f283 and
> > > c699866699411346c5dba4064575 from git master since they appeared to
> > > address some segfaults, but apparently they were unrelated to
> > > whatever it is that we're seeing. 
> > > 
> > > Aside from (obviously) sharing the reproducer, any tips on
> > > debugging this? We of course have a strong preference for debugging
> > > tips that don't impact services on the machine, but if needed,
> > > we'll do what we have to do...  
> > 
> > You can run cgit from the command line with your config and the URL
> > using something like:
> > 
> > 	CGIT_CONFIG=/path/to/cgitrc QUERY_STRING=url=cgit/repo/...
> > cgit
> > 
> > This is what the tests do in tests/setup.sh::cgit_url().
> > 
> > That should allow you to build a debug binary and reproduce under that
> > without a webserver involved, which means you can run under gdb or
> > valgrind.
> 
> 
> Okay, that's helpful - thanks! I've got something that seems to point
> at git's pathspec.c (we're building with (and using on the machine)
> git-2.10.4 currently), but I have no idea where to go from here. 
> This is the gdb output:
> 
> (gdb) run
> Starting program: /var/www/cgi-bin/cgit.cgi 
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library "/lib64/libthread_db.so.1".
> Content-Type: text/plain; charset=UTF-8
> Content-Disposition: inline; filename="82746b4b48cec68acdbb5b7a5ad841b1a21872af..65131f01e212203fbde61d3074640651a02cb6e0.patch"
> Last-Modified: Thu, 24 Aug 2017 06:08:13 GMT
> Expires: Thu, 24 Aug 2017 06:13:13 GMT
> 
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0x00000000004515bd in prefix_pathspec (elt=0x6234623634373238 <error: Cannot access memory at address 0x6234623634373238>, prefixlen=0, prefix=0x0, flags=0, 
>     raw=0x77a138, p_short_magic=<synthetic pointer>, item=0x77a808) at pathspec.c:149
> 149		if (elt[0] != ':' || literal_global ||
> (gdb) 

What version of CGit are you using?  It looks like you could be missing
commit be39d22 (ui-patch: fix crash when using path limit, 2016-11-24)
and using a version affected by the problem that patch fixes.

cgit segfaults

[rworkman]

On Thu, 24 Aug 2017 09:12:02 +0100
John Keeping <john at keeping.me.uk> wrote:

> On Thu, Aug 24, 2017 at 01:18:20AM -0500, Robby Workman wrote:
> > On Wed, 16 Aug 2017 09:36:28 +0100
> > John Keeping <john at keeping.me.uk> wrote:
> >   
> > > On Wed, Aug 16, 2017 at 01:26:52AM -0500, Robby Workman wrote:  
> > > > We're running cgit-1.1 with git-2.10.4 at
> > > > https://git.slackbuilds.org and are seeing some reproducible
> > > > segfaults.
> > > > 
> > > > root at git:/var/log# dmesg -T
> > > > [Wed Aug 16 01:14:23 2017] traps: cgit.cgi[2210] general
> > > > protection ip:4515bd sp:7ffd787a9470 error:0 in
> > > > cgit.cgi[400000+103000]
> > > > 
> > > > This can be reliably triggered (i.e. every time) with at least
> > > > one particular link (I'll share it privately with cgit devs,
> > > > but since I don't know if there's any security impact, I'm not
> > > > going to put it out on the list as yet).
> > > > 
> > > > I've applied 1b4ef6783a71962f8b5da3a23f283 and
> > > > c699866699411346c5dba4064575 from git master since they
> > > > appeared to address some segfaults, but apparently they were
> > > > unrelated to whatever it is that we're seeing. 
> > > > 
> > > > Aside from (obviously) sharing the reproducer, any tips on
> > > > debugging this? We of course have a strong preference for
> > > > debugging tips that don't impact services on the machine, but
> > > > if needed, we'll do what we have to do...    
> > > 
> > > You can run cgit from the command line with your config and the
> > > URL using something like:
> > > 
> > > 	CGIT_CONFIG=/path/to/cgitrc QUERY_STRING=url=cgit/repo/...
> > > cgit
> > > 
> > > This is what the tests do in tests/setup.sh::cgit_url().
> > > 
> > > That should allow you to build a debug binary and reproduce under
> > > that without a webserver involved, which means you can run under
> > > gdb or valgrind.  
> > 
> > 
> > Okay, that's helpful - thanks! I've got something that seems to
> > point at git's pathspec.c (we're building with (and using on the
> > machine) git-2.10.4 currently), but I have no idea where to go from
> > here. This is the gdb output:
> > 
> > (gdb) run
> > Starting program: /var/www/cgi-bin/cgit.cgi 
> > [Thread debugging using libthread_db enabled]
> > Using host libthread_db library "/lib64/libthread_db.so.1".
> > Content-Type: text/plain; charset=UTF-8
> > Content-Disposition: inline;
> > filename="82746b4b48cec68acdbb5b7a5ad841b1a21872af..65131f01e212203fbde61d3074640651a02cb6e0.patch"
> > Last-Modified: Thu, 24 Aug 2017 06:08:13 GMT Expires: Thu, 24 Aug
> > 2017 06:13:13 GMT
> > 
> > 
> > Program received signal SIGSEGV, Segmentation fault.
> > 0x00000000004515bd in prefix_pathspec (elt=0x6234623634373238
> > <error: Cannot access memory at address 0x6234623634373238>,
> > prefixlen=0, prefix=0x0, flags=0, raw=0x77a138,
> > p_short_magic=<synthetic pointer>, item=0x77a808) at pathspec.c:149
> > 149		if (elt[0] != ':' || literal_global || (gdb)   
> 
> What version of CGit are you using?  It looks like you could be
> missing commit be39d22 (ui-patch: fix crash when using path limit,
> 2016-11-24) and using a version affected by the problem that patch
> fixes.


We are using the v1.1 release, which indeed does not include that
commit. I'll look into fixing that this evening and will report 
back with results. 

Perhaps a cgit 1.2 release would be good... :-)

-RW
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20170824/a5fdb92b/attachment.asc>

cgit segfaults

[rworkman]

On Thu, 24 Aug 2017 06:39:23 -0500
Robby Workman <rworkman at slackbuilds.org> wrote:

> On Thu, 24 Aug 2017 09:12:02 +0100
> John Keeping <john at keeping.me.uk> wrote:
> 
> > On Thu, Aug 24, 2017 at 01:18:20AM -0500, Robby Workman wrote:  
> > > On Wed, 16 Aug 2017 09:36:28 +0100
> > > John Keeping <john at keeping.me.uk> wrote:
> > >     
> > > > On Wed, Aug 16, 2017 at 01:26:52AM -0500, Robby Workman
> > > > wrote:    
> > > > > We're running cgit-1.1 with git-2.10.4 at
> > > > > https://git.slackbuilds.org and are seeing some reproducible
> > > > > segfaults.
> > > > > 
> > > > > root at git:/var/log# dmesg -T
> > > > > [Wed Aug 16 01:14:23 2017] traps: cgit.cgi[2210] general
> > > > > protection ip:4515bd sp:7ffd787a9470 error:0 in
> > > > > cgit.cgi[400000+103000]
> > > > > 
> > > > > This can be reliably triggered (i.e. every time) with at least
> > > > > one particular link (I'll share it privately with cgit devs,
> > > > > but since I don't know if there's any security impact, I'm not
> > > > > going to put it out on the list as yet).
> > > > > 
> > > > > I've applied 1b4ef6783a71962f8b5da3a23f283 and
> > > > > c699866699411346c5dba4064575 from git master since they
> > > > > appeared to address some segfaults, but apparently they were
> > > > > unrelated to whatever it is that we're seeing. 
> > > > > 
> > > > > Aside from (obviously) sharing the reproducer, any tips on
> > > > > debugging this? We of course have a strong preference for
> > > > > debugging tips that don't impact services on the machine, but
> > > > > if needed, we'll do what we have to do...      
> > > > 
> > > > You can run cgit from the command line with your config and the
> > > > URL using something like:
> > > > 
> > > > 	CGIT_CONFIG=/path/to/cgitrc
> > > > QUERY_STRING=url=cgit/repo/... cgit
> > > > 
> > > > This is what the tests do in tests/setup.sh::cgit_url().
> > > > 
> > > > That should allow you to build a debug binary and reproduce
> > > > under that without a webserver involved, which means you can
> > > > run under gdb or valgrind.    
> > > 
> > > 
> > > Okay, that's helpful - thanks! I've got something that seems to
> > > point at git's pathspec.c (we're building with (and using on the
> > > machine) git-2.10.4 currently), but I have no idea where to go
> > > from here. This is the gdb output:
> > > 
> > > (gdb) run
> > > Starting program: /var/www/cgi-bin/cgit.cgi 
> > > [Thread debugging using libthread_db enabled]
> > > Using host libthread_db library "/lib64/libthread_db.so.1".
> > > Content-Type: text/plain; charset=UTF-8
> > > Content-Disposition: inline;
> > > filename="82746b4b48cec68acdbb5b7a5ad841b1a21872af..65131f01e212203fbde61d3074640651a02cb6e0.patch"
> > > Last-Modified: Thu, 24 Aug 2017 06:08:13 GMT Expires: Thu, 24 Aug
> > > 2017 06:13:13 GMT
> > > 
> > > 
> > > Program received signal SIGSEGV, Segmentation fault.
> > > 0x00000000004515bd in prefix_pathspec (elt=0x6234623634373238
> > > <error: Cannot access memory at address 0x6234623634373238>,
> > > prefixlen=0, prefix=0x0, flags=0, raw=0x77a138,
> > > p_short_magic=<synthetic pointer>, item=0x77a808) at
> > > pathspec.c:149 149		if (elt[0] != ':' ||
> > > literal_global || (gdb)     
> > 
> > What version of CGit are you using?  It looks like you could be
> > missing commit be39d22 (ui-patch: fix crash when using path limit,
> > 2016-11-24) and using a version affected by the problem that patch
> > fixes.  
> 
> 
> We are using the v1.1 release, which indeed does not include that
> commit. I'll look into fixing that this evening and will report 
> back with results. 


Yep, that fixed it (well, I applied all of the other non-submodule
commits since 1.1) on both machines I was seeing the problem.
Thanks for the help - it's much appreciated!

-RW
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 195 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20170824/78b23332/attachment.asc>