[BUG] String equal when compared, processed differently via //

[BUG] String equal when compared, processed differently via //

[Sebastian Gniazdowski]

Hello,
I get an output from git log. It has $'\1' (Ctrl-A), $'\2' (Ctrl-B),
etc. embedded.

Here is how I process the output:

# hash     \1 message  \2 repo info \3 date     \4 author
# match[1]    match[2]    match[3]     match[4]    match[5]
gitout=( "${gitout[@]//(#b)([^$'\1']#)$'\1'([^$'\2']#)$'\2'([^$'\3']#)$'\3'([^$'\4']#)$'\4'(*)/${CMDR_GREEN}${match[1]}${CMDR_GREEN_E}
${(q-)match[2]}} >${match[3]}<
${CMDR_MAGENTA}${(q-)match[4]}${CMDR_MAGENTA_E} ${match[5]}" )

On input, there is:

15f1acd^AUpdated README.md^B (HEAD -> master, origin/master)^C4 days
ago^DSebastian Gniazdowski

On output, there should be:

^C15f1acd^Y 'Updated README.md' > (HEAD -> master, origin/master)<
^E'4 days ago'^Y Sebastian Gniazdowski

THE THING IS: output string gets truncated after ^B in the input
string. Output is:

^C15f1acd^Y 'Updated README.md'


While it should be:

^C15f1acd^Y 'Updated README.md' > (HEAD -> master, origin/master)<
^E'4 days ago'^Y Sebastian Gniazdowski

I.E. (HEAD -> master, ...) is something that stops output. However, if
I assign identical input to an array "a" - BY HAND:

a=( "15f1acd"$'\1'"Updated README.md"$'\2'" (HEAD -> master,
origin/master)"$'\3'"4 days ago"$'\4'"Sebastian Gniazdowski" )

and then copy from Git output:

a[2]="${gitout[1]}"

and even compare:

[ "$a[1]" = "$a[2]" ] && echo equal >> /tmp/reply || echo "not equal"
>> /tmp/reply

I get:
- "equal" – compare shows strings are equal
- output:

^C15f1acd^Y 'Updated README.md'
^C15f1acd^Y 'Updated README.md' > (HEAD -> master, origin/master)<
^E'4 days ago'^Y Sebastian Gniazdowski

I.E. Identical input is once interrupted before the "( HEAD -> master,
... )", once processed correctly.


PS. The git command is:

gitcmd=( git -C "${ZCMDR[current_project]}" log --max-count=3
--pretty=format:'%h%x01%s%x02%d%x03%cr%x04%an' --abbrev-commit )
gitout=( "${(@f)"$( "${gitcmd[@]}" )"}" )

Best regards,
Sebastian Gniazdowski

Re: [BUG] String equal when compared, processed differently via //

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 1657 bytes --]

It's easy to test as a minimal test case, actually. If one will get
along with the complicated git format – %x01 is just $'\1'.

I attach minimal.zsh. It reveals bug when extendedglob is added to
emulate -LR zsh. Just to be run in any git directory, hoping that any
log message will have "(HEAD -> master, origin/master)" at top.
Interestingly, --max-count=1 instead of 2 stops the bug.

Example runs in two different repositories:

15f1acdUpdated README.md (HEAD -> master, origin/master)4 days
agoSebastian Gniazdowski
0280384Information on source of *foo*~^*bar*4 days agoSebastian Gniazdowski
15f1acd 'Updated README.md'
0280384 'Information on source of *foo*~^*bar*' >< '4 days ago'
Sebastian Gniazdowski

e6d9a0eA stub for uizcm, with binding (^O^U) (HEAD -> master)3 hours
agoSebastian Gniazdowski
feb4624myctags: Message about recompilation request (origin/master)3
hours agoSebastian Gniazdowski
e6d9a0e 'A stub for uizcm, with binding (^O^U)'
feb4624 'myctags: Message about recompilation request' >
(origin/master)< '3 hours ago' Sebastian Gniazdowski

It can be seen that third line doesn't have (HEAD -> master) etc.

Attached code:

====================================

#!/usr/bin/env zsh

emulate -LR zsh
setopt extendedglob

gitcmd=( git log --max-count=2
--pretty=format:'%h%x01%s%x02%d%x03%cr%x04%an' --abbrev-commit )
gitout=( "${(@f)"$( "${gitcmd[@]}" )"}" )

print -rl -- "${gitout[@]}"

gitout=( "${gitout[@]//(#b)([^$'\1']#)$'\1'([^$'\2']#)$'\2'([^$'\3']#)$'\3'([^$'\4']#)$'\4'(*)/${match[1]}
${(q-)match[2]}} >${match[3]}< ${(q-)match[4]} ${match[5]}" )

print -rl -- "${gitout[@]}"

[-- Attachment #2: minimal.zsh --]
[-- Type: application/octet-stream, Size: 423 bytes --]

#!/usr/bin/env zsh

emulate -LR zsh
setopt extendedglob

gitcmd=( git log --max-count=2 --pretty=format:'%h%x01%s%x02%d%x03%cr%x04%an' --abbrev-commit )
gitout=( "${(@f)"$( "${gitcmd[@]}" )"}" )

print -rl -- "${gitout[@]}"

gitout=( "${gitout[@]//(#b)([^$'\1']#)$'\1'([^$'\2']#)$'\2'([^$'\3']#)$'\3'([^$'\4']#)$'\4'(*)/${match[1]} ${(q-)match[2]}} >${match[3]}< ${(q-)match[4]} ${match[5]}" )

print -rl -- "${gitout[@]}"

Re: [BUG] String equal when compared, processed differently via //

[Sebastian Gniazdowski]

BTW. Any ideas for alternate field separation for git output that I could use?

Best regards,
Sebastian Gniazdowski

Re: [BUG] String equal when compared, processed differently via //

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 1689 bytes --]

Even this doesn't work:

gitcmd=( git log --max-count=2
--pretty=format:'<<<<<%h>>>>><<<<<%s>>>>><<<<<%d>>>>><<<<<%cr>>>>><<<<<%an>>>>>'
--abbrev-commit )
gitout=( "${gitout[@]//(#b)<<<<<(*)>>>>><<<<<(*)>>>>><<<<<(*)>>>>><<<<<(*)>>>>><<<<<(*)>>>>>/${match[1]}
${(q-)match[2]}} >${match[3]}< ${(q-)match[4]} ${match[5]}" )


Example outputs (the same repos):
<<<<<15f1acd>>>>><<<<<Updated README.md>>>>><<<<< (HEAD -> master,
origin/master)>>>>><<<<<4 days ago>>>>><<<<<Sebastian Gniazdowski>>>>>
<<<<<0280384>>>>><<<<<Information on source of
*foo*~^*bar*>>>>><<<<<>>>>><<<<<4 days ago>>>>><<<<<Sebastian
Gniazdowski>>>>>
15f1acd 'Updated README.md'
0280384 'Information on source of *foo*~^*bar*' >< '4 days ago'
Sebastian Gniazdowski

<<<<<e6d9a0e>>>>><<<<<A stub for uizcm, with binding (^O^U)>>>>><<<<<
(HEAD -> master)>>>>><<<<<3 hours ago>>>>><<<<<Sebastian
Gniazdowski>>>>>
<<<<<feb4624>>>>><<<<<myctags: Message about recompilation
request>>>>><<<<< (origin/master)>>>>><<<<<3 hours
ago>>>>><<<<<Sebastian Gniazdowski>>>>>
e6d9a0e 'A stub for uizcm, with binding (^O^U)'
feb4624 'myctags: Message about recompilation request' >
(origin/master)< '3 hours ago' Sebastian Gniazdowski


=====================================

#!/usr/bin/env zsh-5.2

emulate -LR zsh
setopt extendedglob

gitcmd=( git log --max-count=2
--pretty=format:'<<<<<%h>>>>><<<<<%s>>>>><<<<<%d>>>>><<<<<%cr>>>>><<<<<%an>>>>>'
--abbrev-commit )
gitout=( "${(@f)"$( "${gitcmd[@]}" )"}" )

print -rl -- "${gitout[@]}"

gitout=( "${gitout[@]//(#b)<<<<<(*)>>>>><<<<<(*)>>>>><<<<<(*)>>>>><<<<<(*)>>>>><<<<<(*)>>>>>/${match[1]}
${(q-)match[2]}} >${match[3]}< ${(q-)match[4]} ${match[5]}" )

print -rl -- "${gitout[@]}"

[-- Attachment #2: minimal2.zsh --]
[-- Type: application/octet-stream, Size: 459 bytes --]

#!/usr/bin/env zsh-5.2

emulate -LR zsh
setopt extendedglob

gitcmd=( git log --max-count=2 --pretty=format:'<<<<<%h>>>>><<<<<%s>>>>><<<<<%d>>>>><<<<<%cr>>>>><<<<<%an>>>>>' --abbrev-commit )
gitout=( "${(@f)"$( "${gitcmd[@]}" )"}" )

print -rl -- "${gitout[@]}"

gitout=( "${gitout[@]//(#b)<<<<<(*)>>>>><<<<<(*)>>>>><<<<<(*)>>>>><<<<<(*)>>>>><<<<<(*)>>>>>/${match[1]} ${(q-)match[2]}} >${match[3]}< ${(q-)match[4]} ${match[5]}" )

print -rl -- "${gitout[@]}"

Re: [BUG] String equal when compared, processed differently via //

[Sebastian Gniazdowski]

[-- Attachment #1: Type: text/plain, Size: 1449 bytes --]

Trying to find any workaround to this (and nothing worked, e.g.:

gitout=( "${(@f)"$( "${gitcmd[@]}" )"}" "${(@f)"$( "${gitcmd[@]}" )"}" )
gitout=( "${gitout[@]//...  ---> gitout=( "${(@)gitout[1,-1]//...
gitout=( "Test" "${gitout[@]}" )
adding (#s) and (#e) to pattern
Using (@f) without quotes

)

I stumbled upon using (I:2:) that helps:

gitout=( "${(I:2:)gitout[@]//(#b)([^$'\1']#)$'\1'([^$'\2']#)$'\2'([^$'\3']#)$'\3'([^$'\4']#)$'\4'(*)/${match[1]}
${(q-)match[2]}} >${match[3]}< ${(q-)match[4]} ${match[5]}" )

Attached source with this. (I:1:) doesn't help.

Now I get the desired output:

e6d9a0eA stub for uizcm, with binding (^O^U) (HEAD -> master)4 hours
agoSebastian Gniazdowski
feb4624myctags: Message about recompilation request (origin/master)4
hours agoSebastian Gniazdowski

e6d9a0eA stub for uizcm, with binding (^O^U) (HEAD -> master)4 hours
agoSebastian Gniazdowski
feb4624myctags: Message about recompilation request (origin/master)4
hours agoSebastian Gniazdowski > (origin/master)< '4 hours ago'
Sebastian Gniazdowski

(Instead of:

e6d9a0eA stub for uizcm, with binding (^O^U) (HEAD -> master)4 hours
agoSebastian Gniazdowski
feb4624myctags: Message about recompilation request (origin/master)4
hours agoSebastian Gniazdowski

e6d9a0e 'A stub for uizcm, with binding (^O^U)'
feb4624 'myctags: Message about recompilation request' >
(origin/master)< '4 hours ago' Sebastian Gniazdowski
)

Best regards,
Sebastian Gniazdowski

[-- Attachment #2: minimal4.zsh --]
[-- Type: application/octet-stream, Size: 431 bytes --]

#!/usr/bin/env zsh

emulate -LR zsh
setopt extendedglob

gitcmd=( git log --max-count=2 --pretty=format:'%h%x01%s%x02%d%x03%cr%x04%an' --abbrev-commit )
gitout=( "${(@f)"$( "${gitcmd[@]}" )"}" )

print -rl -- "${gitout[@]}" ""
gitout=( "${(I:2:)gitout[@]//(#b)([^$'\1']#)$'\1'([^$'\2']#)$'\2'([^$'\3']#)$'\3'([^$'\4']#)$'\4'(*)/${match[1]} ${(q-)match[2]}} >${match[3]}< ${(q-)match[4]} ${match[5]}" )

print -rl -- "${gitout[@]}"

Re: [BUG] String equal when compared, processed differently via //

[Sebastian Gniazdowski]

Ah no it doesn't help, it simply doesn't match at all and repeat output ...

Sorry

Re: [BUG] String equal when compared, processed differently via //

[Sebastian Gniazdowski]

Had to commit a workaround version. I iterate over each entry returned
from git log – manually. There I split using ${=entry}. Switched to
git returning $'\1' after each field, so IFS=$'\1' works fine. So, I
obtain second array gitout2, and go away from the problematic
(unmetafication?) gitout.

https://github.com/psprint/zcommodore/blob/d728ad3460be50c39b24a64698f23d19d5f8eafb/uizcm#L28-L40

Best regards,
Sebastian Gniazdowski

Re: [BUG] String equal when compared, processed differently via //

[Bart Schaefer]

On Sep 21, 10:57am, Sebastian Gniazdowski wrote:
} Subject: [BUG] String equal when compared, processed differently via //
}
} # hash     \1 message  \2 repo info \3 date     \4 author
} # match[1]    match[2]    match[3]     match[4]    match[5]
} gitout=( "${gitout[@]//(#b)([^$'\1']#)$'\1'([^$'\2']#)$'\2'([^$'\3']#)$'\3'([^$'\4']#)$'\4'(*)/${CMDR_GREEN}${match[1]}${CMDR_GREEN_E}
} ${(q-)match[2]}} >${match[3]}<
} ${CMDR_MAGENTA}${(q-)match[4]}${CMDR_MAGENTA_E} ${match[5]}" )
} 
} On input, there is:
} 
} 15f1acd^AUpdated README.md^B (HEAD -> master, origin/master)^C4 days
} ago^DSebastian Gniazdowski
} 
} On output, there should be:
} 
} ^C15f1acd^Y 'Updated README.md' > (HEAD -> master, origin/master)<
} ^E'4 days ago'^Y Sebastian Gniazdowski
} 
} THE THING IS: output string gets truncated after ^B in the input
} string.

If I replace your single gitout=( ... ) assignment with this loop --

integer i
for ((i=$#gitout; i ; i--)) {
    gitout[i]="${gitout[i]//(#b)([^$'\1']#)$'\1'([^$'\2']#)$'\2'([^$'\3']#)$'\3'([^$'\4']#)$'\4'(*)(#B)/${match[1]} ${(q-)match[2]}} >${match[3]}< ${(q-)match[4]} ${match[5]}"
}

-- then I get the output you expect (at least, on a zsh git tree).

I suspect it is because you are referencing $match[3] through $match[5]
outside the closing brace of the ${...//.../...} expression.  If I move
the 2nd closing brace from after ${(q-)match[2]} to place it after
${match[5]} instead, then to the best of my understanding I also get
the output you expect.

I looked at all the subsequent "minimal" examples from your follow-up
email and all of them have the 2nd closing brace after $match[2], so
they would all have suffered from this same problem.

That (I:2:) changes anything is mostly coincidental.

-- 
Barton E. Schaefer

Re: [BUG] String equal when compared, processed differently via //

[Bart Schaefer]

On Sep 21, 12:37pm, Sebastian Gniazdowski wrote:
}
} Ah no it doesn't help, it simply doesn't match at all and repeat output ...

I suspect we've mis-communicated somehow.

I made up the little test script appended at the end of this message.
It copies you gitcmd from workers/39403 but modified a little to put
markers around the five bits of output [1|one][2|two][3|three]...

Then it applies first your all-in-one substitution from workers/39403
(with double close brace after $match[2]) my suggested all-in-one
(with double close brace after $match[5]) from workers/39404 and my
for-loop (pattern tweaked to more closely match 39403, before I was
using the pattern from minimal4.zsh that you sent earlier).

What specifically is wrong ("doesn't match at all") with the latter two?

--- 8< -- snip -- 8< ---
#!/usr/bin/env zsh

emulate -LR zsh
setopt extendedglob

gitcmd=( git log --max-count=3 --pretty=format:'[1|%h]%x01[2|%s]%x02[3|%d]%x03[4|%cr]%x04[5|%an]' --abbrev-commit )
gitout=( "${(@f)"$( "${gitcmd[@]}" )"}" )

CMDR_GREEN='{G>'
CMDR_GREEN_E='<G}'
CMDR_MAGENTA='{M>'
CMDR_MAGENTA_E='<M}'

print -rl -- INPUT: "${gitout[@]}" ""

sebastian_out=(
"${gitout[@]//(#b)([^$'\1']#)$'\1'([^$'\2']#)$'\2'([^$'\3']#)$'\3'([^$'\4']#)$'\4'(*)/${CMDR_GREEN}${match[1]}${CMDR_GREEN_E}
${(q-)match[2]}} >${match[3]}<
${CMDR_MAGENTA}${(q-)match[4]}${CMDR_MAGENTA_E} ${match[5]}" )

print -rl -- +++++++++++++
print -rl -- workers/39403: "${sebastian_out[@]}"

bart_out=(
"${gitout[@]//(#b)([^$'\1']#)$'\1'([^$'\2']#)$'\2'([^$'\3']#)$'\3'([^$'\4']#)$'\4'(*)/${CMDR_GREEN}${match[1]}${CMDR_GREEN_E}
${(q-)match[2]} >${match[3]}<
${CMDR_MAGENTA}${(q-)match[4]}${CMDR_MAGENTA_E} ${match[5]}}" )

print -rl -- +++++++++++++
print -rl -- workers/39404: "${bart_out[@]}"

integer i
for ((i=$#gitout; i ; i--)) {
  gitout[i]="${gitout[i]//(#b)([^$'\1']#)$'\1'([^$'\2']#)$'\2'([^$'\3']#)$'\3'([^$'\4']#)$'\4'(*)/${CMDR_GREEN}${match[1]}${CMDR_GREEN_E}
${(q-)match[2]} >${match[3]}<
${CMDR_MAGENTA}${(q-)match[4]}${CMDR_MAGENTA_E} ${match[5]}}"
}

print -rl -- +++++++++++++
print -rl -- for-loop: "${gitout[@]}"
--- 8< -- snip -- 8< ---

Re: [BUG] String equal when compared, processed differently via //

[Sebastian Gniazdowski]

On 21 September 2016 at 23:13, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sep 21, 12:37pm, Sebastian Gniazdowski wrote:
> }
> } Ah no it doesn't help, it simply doesn't match at all and repeat output ...
>
> What specifically is wrong ("doesn't match at all") with the latter two?

I was referring to the mine (I:2:). It wasn't helping at all, just did
zero-match and repeated input, and terminal ate ^A, ^B, etc. so I
didn't notice.

Sorry for the mistake. Lost much time for this. Really glad there is
no such serious (@f) related bug. I've indeed put "}" in wrong place.

Thanks,
Sebastian Gniazdowski

Re: [BUG] String equal when compared, processed differently via //

[Sebastian Gniazdowski]

On 21 September 2016 at 23:13, Bart Schaefer <schaefer@brasslantern.com> wrote:
> On Sep 21, 12:37pm, Sebastian Gniazdowski wrote:
> }
> } Ah no it doesn't help, it simply doesn't match at all and repeat output ...
>
> I suspect we've mis-communicated somehow.
>
> I made up the little test script appended at the end of this message.
> It copies you gitcmd from workers/39403 but modified a little to put
> markers around the five bits of output [1|one][2|two][3|three]...

PS. I've tested the script and results are as expected.

Best regards,
Sebastian Gniazdowski