zsh-workers
 help / color / mirror / code / Atom feed
* segfault via completion menu
@ 2019-04-09 14:30 Wesley Schwengle
  2019-05-20 20:55 ` Oliver Kiddle
  0 siblings, 1 reply; 9+ messages in thread
From: Wesley Schwengle @ 2019-04-09 14:30 UTC (permalink / raw)
  To: zsh-workers

[-- Attachment #1: Type: text/plain, Size: 752 bytes --]

Hello all,

today I've experiences a segfault running on an update to date zsh
from git (6f35d6c0d0eeb80c0145e5226285a8a45ffb5f25)

I can trigger this in a fairly "large" git repo such as the one I have
at work, or using the repository of git itself. I've included the
output of gdb

I can trigger the bug with zsh -f and the attached zshrc:

* cd ~code/git # the git repo from git@github.com:git/git.git
* zsh -f
* source the zshrc provided in this e-mail
* vi Zaaksysteem::Bar::voo::vooo::voo<tab> <nowwaitafewsecs> <ctrl-c>
* You now get a message: Killed by signal in compadd after 0s
* vi <tab> # yields a segfault

Cheers,
Wesley

-- 
Wesley Schwengle, Developer
Mintlab B.V., https://www.zaaksysteem.nl
E: wesley@mintlab.nl
T:  +31 20 737 00 05

[-- Attachment #2: cmpadd --]
[-- Type: application/octet-stream, Size: 13467 bytes --]

#0  0x000055781597c760 in charrefinc (x=0x7ffd36155dd0, y=0x7f27d302985c <error: Cannot access memory at address 0x7f27d302985c>, 
    z=0x7ffd36155dc8) at pattern.c:1953
        wc = 32551 L'缧'
        ret = 0
#1  0x000055781597df8b in patmatch (prog=0x5578170839c8) at pattern.c:2746
        savpatinput = 0x7f27d30b92f0 "automenu-unambiguous"
        savchrop = 0x7f27d3029829 <error: Cannot access memory at address 0x7f27d3029829>
        chin = 97
        chpa = 361850322
        badin = 0
        badpa = 0
        scan = 0x5578170839d0
        next = 0x5578170839e8
        opnd = 0x55781597c970 <patmungestring+137>
        start = 0x7ffd361564c0 "\200e\025\066"
        save = 0x557817083990 "8"
        chrop = 0x7f27d3029829 <error: Cannot access memory at address 0x7f27d3029829>
        chrend = 0x7f27d302985c <error: Cannot access memory at address 0x7f27d302985c>
        compend = 0x14ffffffff <error: Cannot access memory at address 0x14ffffffff>
        savglobflags = 0
        op = 32551
        no = 0
        min = 1660976935
        fail = 0
        saverrsfound = 0
        from = 140725510823984
        to = 139809021202888
        comp = 93974246705666
        nextch = 32765
        q = 1
#2  0x000055781597d281 in pattryrefs (prog=0x557817083990, string=0x7f27d30b92f0 "automenu-unambiguous", stringlen=20, unmetalenin=-1, 
    patstralloc=0x7ffd36156030, patoffset=0, nump=0x0, begp=0x0, endp=0x0) at pattern.c:2466
        i = 0
        maxnpos = 0
        ret = 1
        origlen = 20
        sp = 0x7ffd36156100
        ep = 0x0
        ptr = 0x7ffd361562e8 "\360\222\v\323'\177"
        progstr = 0x5578170839c8 " \005"
        patstralloc_struct = {unmetalen = 20, unmetalenp = 0, alloced = 0x0, progstrunmeta = 0x0, progstrunmetalen = 0}
#3  0x000055781597cc23 in pattry (prog=0x557817083990, string=0x7f27d30b92f0 "automenu-unambiguous") at pattern.c:2197
No locals.
#4  0x00005578159165d2 in evalcond (state=0x7ffd36157960, fromtest=0x0) at cond.c:322
        test = 907371328
        npat = 2
        pprog = 0x557817083990
        st = 0x7ffd361562c0
        left = 0x7f27d30b92f0 "automenu-unambiguous"
        right = 0x0
        overridename = 0x0
        overridebuf = "\000\000\000\000\000\377\377\377\377\000\000\000"
        pcode = 0x5578172193e0
        code = 114
        ctype = 3
        htok = 1
        ret = 1
#5  0x0000557815924f69 in execcond (state=0x7ffd36157960, do_exec=0) at exec.c:5056
        stat = 1
#6  0x000055781591a2d2 in execsimple (state=0x7ffd36157960) at exec.c:1250
        q = 1
        code = 18
        lv = 1
        otj = 2
#7  0x000055781591a6ad in execlist (state=0x7ffd36157960, dont_change_job=1, exiting=0) at exec.c:1378
        donedebug = 0
        this_donetrap = 0
        donetrap = 0
        next = 0x5578172193f0
        code = 11841
        ret = 0
        cj = 2
        csp = 2
        ltype = 370
        old_pline_level = 2
        old_list_pipe = 0
        old_list_pipe_job = 1
        old_list_pipe_text = 0x0
        oldlineno = 70
        oldnoerrexit = 3
#8  0x0000557815951eea in execif (state=0x7ffd36157960, do_exec=0) at loop.c:555
        end = 0x5578172194a4
        next = 0x5578172194a4
        code = 6577
        olderrexit = 0
        s = 0
        run = 0
#9  0x0000557815922066 in execcmd_exec (state=0x7ffd36157960, eparams=0x7ffd36156c30, input=0, output=0, how=2, last1=2, 
    close_if_forked=-1) at exec.c:3912
        q = 2
        hn = 0x0
        filelist = 0x0
        node = 0x0
        fn = 0x55781723f980
        mfds = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}
        text = 0x0
        save = {-2, -2, -2, -2, -2, -2, -2, -2, -2, -2}
        fil = 32765
        dfil = 907373536
        is_cursh = 1
        do_exec = 0
        redir_err = 0
        i = 10
        nullexec = 0
        magic_assign = 0
        forked = 0
        old_lastval = 0
        is_shfunc = 0
        is_builtin = 0
        is_exec = 0
        use_defpath = 0
        cflags = 0
        orig_cflags = 0
        checked = 0
        oautocont = -1
        oxtrerr = 0x7f27d3830680 <_IO_2_1_stderr_>
        newxtrerr = 0x0
        args = 0x0
        redir = 0x0
        varspc = 0x0
        type = 17
        preargs = 0x0
#10 0x000055781591c823 in execpline2 (state=0x7ffd36157960, pcode=4547, how=2, input=0, output=0, last1=0) at exec.c:1929
        eparams = {args = 0x0, redir = 0x0, beg = 0x5578172193d0, varspc = 0x0, assignspc = 0x0, type = 17, postassigns = 0, htok = 0}
#11 0x000055781591b49e in execpline (state=0x7ffd36157960, slcode=55298, how=2, last1=0) at exec.c:1660
        ipipe = {0, 0}
        opipe = {0, 0}
        pj = 1
        newjob = 2
        old_simple_pline = 1
        slflags = 0
        code = 4547
        lastwj = 0
        lpforked = 0
#12 0x000055781591a841 in execlist (state=0x7ffd36157960, dont_change_job=1, exiting=0) at exec.c:1415
        isend = 1
        donedebug = 0
        this_donetrap = 0
        donetrap = 0
        next = 0x5578172194a4
        code = 55298
        ret = 1
        cj = 1
        csp = 1
        ltype = 2
        old_pline_level = 1
        old_list_pipe = 0
        old_list_pipe_job = 1
        old_list_pipe_text = 0x0
        oldlineno = 56
        oldnoerrexit = 0
#13 0x00005578159527f5 in exectry (state=0x7ffd36157960, do_exec=0) at loop.c:743
        end = 0x55781721aba4
        always = 0x55781721aad0
        endval = 32551
        save_retflag = 32551
        save_breaks = -754216832
        save_contflag = 0
        save_try_errflag = 139809029031552
        save_try_tryflag = 0
        save_try_interrupt = 139809021202664
#14 0x0000557815922066 in execcmd_exec (state=0x7ffd36157960, eparams=0x7ffd36157580, input=0, output=0, how=2, last1=2, 
    close_if_forked=-1) at exec.c:3912
        q = 6
        hn = 0x0
        filelist = 0x0
        node = 0x7ffd36157540
        fn = 0x8000000000
        mfds = {0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0}
        text = 0x0
        save = {-2, -2, -2, -2, -2, -2, -2, -2, -2, -2}
        fil = 32765
        dfil = 907375792
        is_cursh = 1
        do_exec = 0
        redir_err = 0
        i = 10
        nullexec = 0
        magic_assign = 0
        forked = 0
        old_lastval = 0
        is_shfunc = 0
        is_builtin = 0
        is_exec = 0
        use_defpath = 0
        cflags = 0
        orig_cflags = 0
        checked = 0
        oautocont = -1
        oxtrerr = 0x7f27d3830680 <_IO_2_1_stderr_>
        newxtrerr = 0x0
        args = 0x0
        redir = 0x0
        varspc = 0x0
        type = 21
        preargs = 0x0
#15 0x000055781591c823 in execpline2 (state=0x7ffd36157960, pcode=3651, how=2, input=0, output=0, last1=0) at exec.c:1929
        eparams = {args = 0x0, redir = 0x0, beg = 0x5578172192c4, varspc = 0x0, assignspc = 0x0, type = 21, postassigns = 0, htok = 0}
#16 0x000055781591b49e in execpline (state=0x7ffd36157960, slcode=1631234, how=2, last1=0) at exec.c:1660
        ipipe = {0, 0}
        opipe = {0, 0}
        pj = 0
        newjob = 1
        old_simple_pline = 0
        slflags = 0
        code = 3651
        lastwj = 0
        lpforked = 0
#17 0x000055781591a841 in execlist (state=0x7ffd36157960, dont_change_job=1, exiting=0) at exec.c:1415
        isend = 1
        donedebug = 0
        this_donetrap = 0
        donetrap = 0
        next = 0x55781721aba4
        code = 1631234
        ret = 0
        cj = 0
        csp = 0
        ltype = 2
        old_pline_level = 0
        old_list_pipe = 0
        old_list_pipe_job = 0
        old_list_pipe_text = 0x0
        oldlineno = 2
        oldnoerrexit = 0
#18 0x0000557815919fb7 in execode (p=0x5578170782c0, dont_change_job=1, exiting=0, context=0x5578159abeb2 "shfunc") at exec.c:1194
        s = {prog = 0x5578170782c0, pc = 0x5578172193f0, strs = 0x55781721ad1c "local"}
        zsh_eval_context_len = 32
        alen = 0
#19 0x0000557815927882 in runshfunc (prog=0x5578170782c0, wrap=0x0, name=0x7f27d3a31178 "_main_complete") at exec.c:5976
        cont = 21880
        ouu = 23
        ou = 0x55781720f290 "/home/wesleys/code/git"
#20 0x00007f27d30e4182 in comp_wrapper (prog=0x5578170782c0, w=0x0, name=0x7f27d3a31178 "_main_complete") at complete.c:1524
        opre = 0x55781720f960 ""
        oipre = 0x55781721f660 ""
        oqipre = 0x55781722c680 ""
        oqs = 0x557817233320 "\001"
        m = 0
        pp = 0x5578172be908
        osuf = 0x55781720f900 ""
        oisuf = 0x55781720f980 ""
        oqisuf = 0x55781722c620 ""
        ocur = 2
        orest = 0x0
        owords = 0x5578172332c0
        oq = 0x55781723edc0 ""
        oqi = 0x557817232df0 ""
        oaq = 0x0
        kunset = 0
        sm = 512
        oredirs = 0x55781720f2f0
        runset = 0
#21 0x000055781592766e in runshfunc (prog=0x5578170782c0, wrap=0x7f27d3102dc0 <wrapper>, name=0x7f27d3a31178 "_main_complete")
    at exec.c:5960
        cont = 21880
        ouu = 23
        ou = 0x55781720f4f0 "/home/wesleys/code/git"
#22 0x0000557815926ebc in doshfunc (shfunc=0x557817103190, doshargs=0x0, noreturnval=1) at exec.c:5826
        funcsave = 0x7f27d3a31020
        _switch_oldheaps = 0x7f27d3a35000
        pptab = 0x55781705dc40
        x = 0x7f27d3a39000
        ret = 21880
        name = 0x557817103240 "_main_complete"
        flags = 1187840
        fname = 0x7f27d3a35020 "_main_complete"
        prog = 0x5578170782c0
        oflags = 1187840
        funcdepth = 1
        funcheap = 0x7f27d3a35000
#23 0x00007f27d30e6bcb in callcompfunc (s=0x7f27d3a3a228 "", fn=0x5578170a2330 "_main_complete") at compcore.c:833
        largs = 0x0
        _switch_oldheaps = 0x7f27d3a39000
        p = 0x557817082200
        aadd = 0
        rset = 1023
        kset = 63959491
        ockpms = 0x0
        tmp = 0x7f27d3a3a238 ""
        icf = 0
        ocrpms = 0x0
        usea = 1
        osc = 0
        shfunc = 0x557817103190
        lv = 130
        buf = "\000\000\000\000\200\000\000\000\000\243\032\372\246\301ܟpj\006\027"
#24 0x00007f27d30e74ec in makecomplist (s=0x7f27d3a3a228 "", incmd=0, lst=0) at compcore.c:988
        os = 0x557817078ea0 ""
        onm = 0
        odm = 0
        osi = 11
        p = 0x0
        owb = 3
        owe = 3
        ooffs = 0
#25 0x00007f27d30e509b in do_completion (dummy=0x7f27d315fd88 <zlehooks+40>, dat=0x7ffd36158070) at compcore.c:343
        ret = 0
        lst = 0
        incmd = 0
        osl = 0
        s = 0x557817078ea0 ""
        opm = 0x5578172ae1c0 ""
        n = 0x557817078ea0
#26 0x0000557815959fa0 in runhookdef (h=0x7f27d315fd88 <zlehooks+40>, d=0x7ffd36158070) at module.c:1007
No locals.
#27 0x00007f27d31426f6 in docompletion (s=0x557817078ea0 "", lst=0, incmd=0) at zle_tricky.c:2340
        dat = {s = 0x557817078ea0 "", lst = 0, incmd = 0}
#28 0x00007f27d313df26 in docomplete (lst=0) at zle_tricky.c:868
        active = 1
        s = 0x557817078ea0 ""
        ol = 0x0
        olst = 4
        chl = 0
        ne = 0
        ocs = 3
        ret = 0
        dat = {-747745960, 32551}
#29 0x00007f27d313c721 in expandorcomplete (args=0x7f27d3160468 <zlenoargs>) at zle_tricky.c:314
        ret = 32551
#30 0x00007f27d313c303 in completecall (args=0x7f27d3160468 <zlenoargs>) at zle_tricky.c:207
No locals.
#31 0x00007f27d312841a in execzlefunc (func=0x7f27d315c7f0 <thingies+2000>, args=0x7f27d3160468 <zlenoargs>, set_bindk=0)
    at zle_main.c:1459
        atcurhist = 1
        inuse = 0
        wflags = 518
        r = 0
        ret = 0
        remetafy = 0
        nestedvichg = 1
        isrepeat = 0
        w = 0x55781711e670
        save_bindk = 0x7f27d315c7f0 <thingies+2000>
#32 0x00007f27d31274df in zlecore () at zle_main.c:1139
        km = 0x7ffd361584f0
#33 0x00007f27d3127e99 in zleread (lp=0x5578159db488 <prompt>, rp=0x0, flags=3, context=0, init=0x7f27d3150320 "zle-line-init", 
    finish=0x7f27d3150310 "zle-line-finish") at zle_main.c:1346
        s = 0x0
        bracket = 0x557817082280
        old_errno = 0
        tmout = 0
#34 0x00007f27d312ad4c in zle_main_entry (cmd=1, ap=0x7ffd36158580) at zle_main.c:2109
        lp = 0x5578159db488 <prompt>
        rp = 0x0
        flags = 3
        context = 0
#35 0x0000557815942812 in zleentry (cmd=1) at init.c:1604
        ret = 0x0
        ap = {{gp_offset = 40, fp_offset = 48, overflow_arg_area = 0x7ffd36158680, reg_save_area = 0x7ffd361585c0}}
#36 0x00005578159437a3 in inputline () at input.c:295
        flags = 3
        ingetcline = 0x557815986623 <install_handler+107> "\220\311\303UH\211\345\017\266\005-M\005"
        ingetcpmptl = 0x5578159db488 <prompt>
        ingetcpmptr = 0x0
        context = 0
#37 0x00005578159435f6 in ingetc () at input.c:228
        lastc = 32
#38 0x0000557815936540 in ihgetc () at hist.c:408
        c = 0
#39 0x000055781594d01a in gettok () at lex.c:611
        c = 0
        d = 0
        peekfd = -1
        peek = NULLTOK
#40 0x000055781594c70b in zshlex () at lex.c:275
No locals.
#41 0x000055781596feb1 in parse_event (endtok=37) at parse.c:581
No locals.
#42 0x000055781593eb76 in loop (toplevel=1, justonce=0) at init.c:147
        prog = 0x0
        err = 0
        non_empty = 1
#43 0x0000557815942cae in zsh_main (argc=1, argv=0x7ffd36158a58) at init.c:1758
        errexit = 0
        t = 0x7ffd36158a60
        runscript = 0x0
        zsh_name = 0x7ffd3615a4ab "zsh"
        cmd = 0x0
        t0 = 162
#44 0x00005578158f9d55 in main (argc=1, argv=0x7ffd36158a58) at ./main.c:93
No locals.

[-- Attachment #3: minimal-zshrc --]
[-- Type: application/octet-stream, Size: 815 bytes --]

# Speed up tips for ZSH
# See https://htr3n.github.io/2018/07/faster-zsh/ for more information
#zmodload zsh/zprof      # Enable this when you want to profile your zshrc

ZDOTDIR=$HOME/.zsh

autoload -Uz compinit

zstyle ':completion:*' completer _complete _match _approximate
zstyle ':completion:*:match:*' original only
zstyle ':completion:*:approximate:*' max-errors 1 numeric
zstyle ':completion:*' menu select

zstyle -e ':completion:*:approximate:*' \
    max-errors 'reply=($((($#PREFIX+$#SUFFIX)/3))numeric)'

zstyle ':completion:*' squeeze-slashes true

_ZCOMP=${ZDOTDIR:-$HOME}/.zcompdump
today=$(date --date '00:00 today' +%s)
if [[ ! -e $_ZCOMP || $today -gt $(stat --format %Y $_ZCOMP) ]];
then
    compinit -i
    touch ${_ZCOMP}
else
    compinit -C;
fi
unset _ZCOMP


# vim: filetype=zsh syntax=zsh

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: segfault via completion menu
  2019-04-09 14:30 segfault via completion menu Wesley Schwengle
@ 2019-05-20 20:55 ` Oliver Kiddle
  2019-05-21 21:58   ` Daniel Shahaf
  0 siblings, 1 reply; 9+ messages in thread
From: Oliver Kiddle @ 2019-05-20 20:55 UTC (permalink / raw)
  To: Wesley Schwengle; +Cc: zsh-workers

On 9 Apr, Wesley Schwengle wrote:
> today I've experiences a segfault running on an update to date zsh
> from git (6f35d6c0d0eeb80c0145e5226285a8a45ffb5f25)

Thanks for sending this to us.

> I can trigger this in a fairly "large" git repo such as the one I have
> at work, or using the repository of git itself. I've included the
> output of gdb
>
> I can trigger the bug with zsh -f and the attached zshrc:
>
> * cd ~code/git # the git repo from git@github.com:git/git.git
> * zsh -f
> * source the zshrc provided in this e-mail
> * vi Zaaksysteem::Bar::voo::vooo::voo<tab> <nowwaitafewsecs> <ctrl-c>
> * You now get a message: Killed by signal in compadd after 0s
> * vi <tab> # yields a segfault

This seems to be due to interrupting of pattern matching. I was able to
cut this down to something that doesn't involve completion:

  zsh -f
  setopt extendedglob
  () {
    TRAPINT() {
      return 1
    }
    : **/*~(#a10)Zaaksysteem::Bar::voo::vooo::voo
  }

  interrupt the function with Ctrl-C and now do something that involves
  pattern matching, e.g:

  [[ a = :*: ]]

The glob can be varied, it just needs to take long enough to give you
time to catch it with Ctrl-C so pick a big enough directory.

Older versions of zsh didn't have the problem so I've been able to
bisect it down to the change that introduced it:

[827d36077641ca87d1796b9c5cb05e7c44b01919] 36853: replace pushheap/popheap by NEWHEAPS/OLDHEAPS in doshfunc() to optimize memory management

Backing that out on top of master appears to fix the issue. As it was an
optimisation, that might be an option. From reading comments in mem.c,
it's not especially clear to me what newheaps/oldheaps do. There's only
the one other use.

Oliver

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: segfault via completion menu
  2019-05-20 20:55 ` Oliver Kiddle
@ 2019-05-21 21:58   ` Daniel Shahaf
  2019-05-21 22:19     ` Bart Schaefer
  0 siblings, 1 reply; 9+ messages in thread
From: Daniel Shahaf @ 2019-05-21 21:58 UTC (permalink / raw)
  To: Oliver Kiddle, Wesley Schwengle; +Cc: zsh-workers

Oliver Kiddle wrote on Mon, 20 May 2019 20:57 +00:00:
> Backing that out on top of master appears to fix the issue. As it was an
> optimisation, that might be an option. From reading comments in mem.c,
> it's not especially clear to me what newheaps/oldheaps do. There's only
> the one other use.

It's not too clear to me either.

The defining property of a heap, IIUC, is that freeheap() and popheap()
release all heap-allocated memory allocated since the last pushheap().
In this respect they're analogous to APR pools, which I'm familiar
with¹.

Then, what NEWHEAPS() seems to do is put away the entire stack
of heaps, create a new stack of heaps, and then OLDHEAPS() frees the
entire new stack of heaps and reverts to the old one.  How is this
better than simply doing pushheap() and popheap()?  

- Code between NEWHEAPS() and OLDHEAPS() doesn't have to be careful to
  match pushheap() and popheap() calls exactly, because OLDHEAPS() will
  clean up everything anyway.
  
- Anything allocated on heap between NEWHEAPS() and OLDHEAPS() becomes
  invalid once OLDHEAPS() is called.

- There might be some considerations about maximum depth of the stack or
  total number of bytes allocated by heaps in a single stack?

- (Anything else?)

HTH,

Daniel

¹ https://subversion.apache.org/docs/community-guide/conventions.html#apr-pools
  pushheap() ≈ { p = apr_pool_create(p); }
  popheap() ≈ { tmp = p->parent; apr_pool_destroy(p); p = tmp; }

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: segfault via completion menu
  2019-05-21 21:58   ` Daniel Shahaf
@ 2019-05-21 22:19     ` Bart Schaefer
  2019-05-22  8:49       ` Peter Stephenson
  0 siblings, 1 reply; 9+ messages in thread
From: Bart Schaefer @ 2019-05-21 22:19 UTC (permalink / raw)
  To: Daniel Shahaf; +Cc: Oliver Kiddle, Wesley Schwengle, zsh-workers

On Tue, May 21, 2019 at 2:59 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
>
> Oliver Kiddle wrote on Mon, 20 May 2019 20:57 +00:00:
> > Backing that out on top of master appears to fix the issue. As it was an
> > optimisation, that might be an option. From reading comments in mem.c,
> > it's not especially clear to me what newheaps/oldheaps do. There's only
> > the one other use.
>
> It's not too clear to me either.

Operationally, the difference between pushheap() and NEWHEAPS() is
that pushing scans the entire list of existing heap blocks and moves
them out of the way in favor of new empty blocks.  This is
time-consuming and eats a lot of memory if you're in a tightly
recursive function (read back through the articles leading up to the
change).  In contrast, NEWHEAPS() just starts an entirely new chain of
heap blocks without reference to the existing chain.

If an error occurs as a result of NEWHEAPS()/OLDHEAPS() in the context
of workers/36853, it ought to be traceable to something leaking
heap-allocated storage across boundaries, and probably means there was
a memory leak when interrupting a pattern match before, which this has
turned into an error by freeing the previously leaked space.

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: segfault via completion menu
  2019-05-21 22:19     ` Bart Schaefer
@ 2019-05-22  8:49       ` Peter Stephenson
  2019-05-23 16:34         ` Peter Stephenson
  0 siblings, 1 reply; 9+ messages in thread
From: Peter Stephenson @ 2019-05-22  8:49 UTC (permalink / raw)
  To: zsh-workers

On Tue, 2019-05-21 at 15:19 -0700, Bart Schaefer wrote:
> On Tue, May 21, 2019 at 2:59 PM Daniel Shahaf <d.s@daniel.shahaf.name> wrote:
> If an error occurs as a result of NEWHEAPS()/OLDHEAPS() in the context
> of workers/36853, it ought to be traceable to something leaking
> heap-allocated storage across boundaries, and probably means there was
> a memory leak when interrupting a pattern match before, which this has
> turned into an error by freeing the previously leaked space.

This is really just saying the same thing a different way, but ---
the original crash was in patmatch() when looking at the pattern code,
which is set up when we compile a pattern in a different
function.  So there's probably some path where it's possible not to
recompile a pattern or reuse a pattern without recompiling, or simply
hang on to it too long.

It'll be something in the prog passed into pattry() from evalcond() and
I'm guesing in this case the pprog in that function came from
stat->prog->pats[npat] so was fished out of the existing programme
rather than compiled locally.

In general (and fairly obviously), anything stored long term in compiled
code is in permanent (i.e. explicitly freed) storage and if the value is
on the heap it should only have been parsed or copied from permanent
storage for immediate use.

pws


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: segfault via completion menu
  2019-05-22  8:49       ` Peter Stephenson
@ 2019-05-23 16:34         ` Peter Stephenson
  2019-05-24 22:34           ` Peter Stephenson
  0 siblings, 1 reply; 9+ messages in thread
From: Peter Stephenson @ 2019-05-23 16:34 UTC (permalink / raw)
  To: zsh-workers

On Wed, 2019-05-22 at 09:49 +0100, Peter Stephenson wrote:
> On Tue, 2019-05-21 at 15:19 -0700, Bart Schaefer wrote:
> It'll be something in the prog passed into pattry() from evalcond() and
> I'm guesing in this case the pprog in that function came from
> stat->prog->pats[npat] so was fished out of the existing programme
> rather than compiled locally.

If so, something in the following assumptions is being violated, but I
can't see what from looking at the code.

- Whenever we initialise or copy pats[*] we set it to point to a dummy
pattern, ensuring we recompile the pattern next time.

- If we compile the pattern, we only save it if the code it's
being saved into is not on the heap.  (Which it shouldn't be
here, as this is a shell function.)

- If we save it we always use permanent memory.

Possibly adding some debug flags to check the heapiness of patterns at
various points might show something.  (Guess we should check it
is coming from a previously saved version rather than a newly
compiled version, though I don't see how the latter could cause
a crash.)

pws


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: segfault via completion menu
  2019-05-23 16:34         ` Peter Stephenson
@ 2019-05-24 22:34           ` Peter Stephenson
  2019-05-31 12:00             ` Wesley Schwengle
  0 siblings, 1 reply; 9+ messages in thread
From: Peter Stephenson @ 2019-05-24 22:34 UTC (permalink / raw)
  To: zsh-workers

On Thu, 2019-05-23 at 17:34 +0100, Peter Stephenson wrote:
> On Wed, 2019-05-22 at 09:49 +0100, Peter Stephenson wrote:
> > On Tue, 2019-05-21 at 15:19 -0700, Bart Schaefer wrote:
> > It'll be something in the prog passed into pattry() from evalcond() and
> > I'm guesing in this case the pprog in that function came from
> > stat->prog->pats[npat] so was fished out of the existing programme
> > rather than compiled locally.
> 
> If so, something in the following assumptions is being violated, but I
> can't see what from looking at the code.

Alternatively, there could be some internal state in the pattern
matching left over in the case of an error return --- there are
variables that are intended to be used in recursive matching but
currently aren't explicity initialised at the start of matching.  It's
not clear how that could happen, but it would be safest anyway to
sanitise always on entry.

pws

diff --git a/Src/pattern.c b/Src/pattern.c
index 737f5cdcb..3d30b013c 100644
--- a/Src/pattern.c
+++ b/Src/pattern.c
@@ -2030,6 +2030,16 @@ int errsfound;				/* Total error count so far */
 /**/
 int forceerrs;				/* Forced maximum error count */
 
+/*
+ * exactpos is used to remember how far down an exact string we have
+ * matched, if we are doing approximation and can therefore redo from
+ * the same point; we never need to otherwise.
+ *
+ * exactend is a pointer to the end of the string, which isn't
+ * null-terminated.
+ */
+static char *exactpos, *exactend;
+
 /**/
 void
 pattrystart(void)
@@ -2463,6 +2473,8 @@ pattryrefs(Patprog prog, char *string, int stringlen, int unmetalenin,
 
 	patinput = patinstart;
 
+	exactpos = exactend = NULL;
+	/* The only external call to patmatch --- all others are recursive */
 	if (patmatch((Upat)progstr)) {
 	    /*
 	     * we were lazy and didn't save the globflags if an exclusion
@@ -2652,16 +2664,6 @@ patmatchlen(void)
 #define CHARMATCH_EXPR(expr, chpa) \
 	(charmatch_cache = (expr), CHARMATCH(charmatch_cache, chpa))
 
-/*
- * exactpos is used to remember how far down an exact string we have
- * matched, if we are doing approximation and can therefore redo from
- * the same point; we never need to otherwise.
- *
- * exactend is a pointer to the end of the string, which isn't
- * null-terminated.
- */
-static char *exactpos, *exactend;
-
 /*
  * Main matching routine.
  *


^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: segfault via completion menu
  2019-05-24 22:34           ` Peter Stephenson
@ 2019-05-31 12:00             ` Wesley Schwengle
  2019-05-31 13:29               ` Peter Stephenson
  0 siblings, 1 reply; 9+ messages in thread
From: Wesley Schwengle @ 2019-05-31 12:00 UTC (permalink / raw)
  To: zsh-workers

Op za 25 mei 2019 om 00:35 schreef Peter Stephenson
<p.w.stephenson@ntlworld.com>:

> [snip]

Hello, I'm back from my month vacation and see there has been progress
on this issue. I believe the bug is fixed now by commit
4b85edface379a3575273a2b712d80bd9420d4c9.

Thanks for the help!

Cheers,
Wesley

-- 
Wesley Schwengle, Developer
Mintlab B.V., https://www.zaaksysteem.nl
E: wesley@mintlab.nl
T:  +31 20 737 00 05

^ permalink raw reply	[flat|nested] 9+ messages in thread

* Re: segfault via completion menu
  2019-05-31 12:00             ` Wesley Schwengle
@ 2019-05-31 13:29               ` Peter Stephenson
  0 siblings, 0 replies; 9+ messages in thread
From: Peter Stephenson @ 2019-05-31 13:29 UTC (permalink / raw)
  To: zsh-workers

On Fri, 2019-05-31 at 14:00 +0200, Wesley Schwengle wrote:
> Op za 25 mei 2019 om 00:35 schreef Peter Stephenson
> <p.w.stephenson@ntlworld.com>:
> 
> > 
> > [snip]
> Hello, I'm back from my month vacation and see there has been progress
> on this issue. I believe the bug is fixed now by commit
> 4b85edface379a3575273a2b712d80bd9420d4c9.
> 
> Thanks for the help!

OK, that's good --- it looked plausible it was somewhere there but I
didn't know for sure so I'm glad to hear.

Cheers
pws


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2019-05-31 13:30 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-04-09 14:30 segfault via completion menu Wesley Schwengle
2019-05-20 20:55 ` Oliver Kiddle
2019-05-21 21:58   ` Daniel Shahaf
2019-05-21 22:19     ` Bart Schaefer
2019-05-22  8:49       ` Peter Stephenson
2019-05-23 16:34         ` Peter Stephenson
2019-05-24 22:34           ` Peter Stephenson
2019-05-31 12:00             ` Wesley Schwengle
2019-05-31 13:29               ` Peter Stephenson

Code repositories for project(s) associated with this public inbox

	https://git.vuxu.org/mirror/zsh/

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).