Bug: SIGSEGV in OPENSSL_cleanse

Bug: SIGSEGV in OPENSSL_cleanse

[johny]

Hi all.

I am, or rather was, running an instance of cgit on an ARM box:

> uname -srm
Linux 4.9.28-2-ARCH armv6l

I run ArchLinuxARM and they recently had an update to openssl:

openssl		1.1.0.e-1
openssl-1.0	1.0.2.k-3

So I currently have 2 versions of openssl on that box.

After running cgit for awhile I noticed it now SIGSEGVs on certain requests:

           PID: 12517 (cgit.cgi)
           UID: 33 (http)
           GID: 33 (http)
        Signal: 11 (SEGV)
     Timestamp: Sun 2017-05-21 13:26:35 CEST (1 day 6h ago)
  Command Line: /usr/lib/cgit/cgit.cgi
    Executable: /usr/lib/cgit/cgit.cgi
 Control Group: /system.slice/system-uwsgi.slice/uwsgi at cgit.service
          Unit: uwsgi at cgit.service
         Slice: system-uwsgi.slice
       Boot ID: 93dadbde0e144f3ab346f1e21ac7ee5d
    Machine ID: 4bd17fc498ad478094fa58c3a7782769
      Hostname: Neuromancer
       Storage:
/var/lib/systemd/coredump/core.cgit\x2ecgi.33.93dadbde0e144f3ab346f1e21ac7ee5d.12517.1495365995000000000000.lz4
       Message: Process 12517 (cgit.cgi) of user 33 dumped core.

                Stack trace of thread 12517:
                #0  0x000000007678e7d8 OPENSSL_cleanse
(/usr/lib/libcrypto.so.1.0.0)


A bit more of the stack is shown when running the dump through gdb:

#0  0x765c37d8 in OPENSSL_cleanse () from /usr/lib/libcrypto.so.1.0.0
#1  0x7664243c in EVP_MD_CTX_cleanup () from /usr/lib/libcrypto.so.1.0.0
#2  0x76642774 in EVP_MD_CTX_destroy () from /usr/lib/libcrypto.so.1.0.0
Backtrace stopped: previous frame identical to this frame (corrupt stack?)


Investigating more, some weird behavior is shown, while ldd says
cgit.cgi will run with /usr/lib/libcrypto.so.1.0.0 and even the coredump
confirms that, when actually debugging the coredump, `info sharedlib` says:

warning: Corrupted shared library list: 0x76ffa8b0 != 0x0

Bug: SIGSEGV in OPENSSL_cleanse

[john]

On Mon, May 22, 2017 at 07:54:53PM +0200, Jan Jancar wrote:
> I am, or rather was, running an instance of cgit on an ARM box:
> 
> > uname -srm
> Linux 4.9.28-2-ARCH armv6l
> 
> I run ArchLinuxARM and they recently had an update to openssl:
> 
> openssl		1.1.0.e-1
> openssl-1.0	1.0.2.k-3
> 
> So I currently have 2 versions of openssl on that box.
> 
> After running cgit for awhile I noticed it now SIGSEGVs on certain requests:
> 
>            PID: 12517 (cgit.cgi)
>            UID: 33 (http)
>            GID: 33 (http)
>         Signal: 11 (SEGV)
>      Timestamp: Sun 2017-05-21 13:26:35 CEST (1 day 6h ago)
>   Command Line: /usr/lib/cgit/cgit.cgi
>     Executable: /usr/lib/cgit/cgit.cgi
>  Control Group: /system.slice/system-uwsgi.slice/uwsgi at cgit.service
>           Unit: uwsgi at cgit.service
>          Slice: system-uwsgi.slice
>        Boot ID: 93dadbde0e144f3ab346f1e21ac7ee5d
>     Machine ID: 4bd17fc498ad478094fa58c3a7782769
>       Hostname: Neuromancer
>        Storage:
> /var/lib/systemd/coredump/core.cgit\x2ecgi.33.93dadbde0e144f3ab346f1e21ac7ee5d.12517.1495365995000000000000.lz4
>        Message: Process 12517 (cgit.cgi) of user 33 dumped core.
> 
>                 Stack trace of thread 12517:
>                 #0  0x000000007678e7d8 OPENSSL_cleanse
> (/usr/lib/libcrypto.so.1.0.0)
> 
> 
> A bit more of the stack is shown when running the dump through gdb:
> 
> #0  0x765c37d8 in OPENSSL_cleanse () from /usr/lib/libcrypto.so.1.0.0
> #1  0x7664243c in EVP_MD_CTX_cleanup () from /usr/lib/libcrypto.so.1.0.0
> #2  0x76642774 in EVP_MD_CTX_destroy () from /usr/lib/libcrypto.so.1.0.0
> Backtrace stopped: previous frame identical to this frame (corrupt stack?)
> 
> 
> Investigating more, some weird behavior is shown, while ldd says
> cgit.cgi will run with /usr/lib/libcrypto.so.1.0.0 and even the coredump
> confirms that, when actually debugging the coredump, `info sharedlib` says:
> 
> warning: Corrupted shared library list: 0x76ffa8b0 != 0x0
> From        To          Syms Read   Shared Object Library
> 0x76fcf8f0  0x76feaaf0  Yes (*)     /lib/ld-linux-armhf.so.3
> 0x76e47000  0x76f6f794  Yes (*)     /usr/lib/libcrypto.so.1.1
> 0x76dcdb88  0x76dda8a0  Yes (*)     /usr/lib/libz.so.1
> 0x76da5190  0x76db6ce4  Yes (*)     /usr/lib/libpthread.so.0
> 0x76d8b7d0  0x76d8f418  Yes (*)     /usr/lib/librt.so.1
> 0x76d20580  0x76d74288  Yes (*)     /usr/lib/libluajit-5.1.so.2
> 0x76d09a30  0x76d0a944  Yes (*)     /usr/lib/libdl.so.2
> 0x76bd41f0  0x76cd4920  Yes (*)     /usr/lib/libc.so.6
> 0x76b3c680  0x76b776f0  Yes (*)     /usr/lib/libm.so.6
> 0x76b18180  0x76b26930  Yes         /usr/lib/libgcc_s.so.1
> 
> After that, setting LD_PRELOAD=/usr/lib/libcrypto.so.1.0
> actually fixes the problem. So is cgit incompatible with openssl 1.1
> or is there some other bug hiding?

Did you compile CGit yourself or are you using a pre-built package?
What version of libssl-dev is installed?

I wouldn't be surprised if compiling against openssl-1.0 headers but
linking with openssl-1.1 produces the behaviour you describe above.

Bug: SIGSEGV in OPENSSL_cleanse

[johny]

On 05/22/2017 08:51 PM, John Keeping wrote:
> Did you compile CGit yourself or are you using a pre-built package?
> What version of libssl-dev is installed?
> 
> I wouldn't be surprised if compiling against openssl-1.0 headers but
> linking with openssl-1.1 produces the behaviour you describe above.
> 

I am using the distro-provided version:

https://archlinuxarm.org/packages/armv6h/cgit

Name            : cgit
Version         : 1.1-2
Description     : A web interface for git written in plain C
Architecture    : armv6h
URL             : http://git.zx2c4.com/cgit/
Licenses        : GPL2
Groups          : None
Provides        : None
Depends On      : openssl  luajit
Optional Deps   : python-pygments: syntax highlighting support
                  python-markdown: about page formated with markdown
                  mime-types: serve file with correct content-type
header [installed]
Required By     : None
Optional For    : None
Conflicts With  : None
Replaces        : None
Installed Size  : 975.00 KiB
Packager        : Arch Linux ARM Build System <builder+xu6 at archlinuxarm.org>
Build Date      : Wed 26 Apr 2017 01:39:23 AM CEST
Install Date    : Fri 05 May 2017 08:21:33 PM CEST
Install Reason  : Explicitly installed
Install Script  : Yes
Validated By    : Signature


Arch linux doesn't package -dev packages, the headers are part of the
openssl or openssl-1.0 package. So they live in
"/usr/include/openssl/*.h" and "/usr/include/openssl-1.0/openssl/*.h"
respectively.

Currently rebuilding cgit, but that might take a while since it's just a
RaspberryPi so in the meantime, since I run cgit through uWSGI behind
nginx, I fixed this by adding LD_PRELOAD=/usr/lib/libcrypto.so.1.0.0 to
the uWSGI cgit environment. Still don't know how this happened and why
is the coredump reporting a SEGV in the wrong lib(with which in a
LD_PRELOAD it actually works).

Thanks! In the end this is really just a packaging / build bug so not an
upstream problem.

Cheers,
-- 
Jan
______________________________________________________
   /\  # PGP: 362056ADA8F2F4E421565EF87F4A448FE68F329D
  /__\  # https://neuromancer.sk
 /\  /\  # Eastern Seaboard Phishing Authority
/__\/__\  #

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 862 bytes
Desc: OpenPGP digital signature
URL: <http://lists.zx2c4.com/pipermail/cgit/attachments/20170522/59f07c3a/attachment.asc>