* Callgrind run
@ 2016-11-10 10:37 ` Sebastian Gniazdowski
2016-11-10 12:31 ` Peter Stephenson
2016-11-10 13:47 ` multibyte optimisations Peter Stephenson
0 siblings, 2 replies; 5+ messages in thread
From: Sebastian Gniazdowski @ 2016-11-10 10:37 UTC (permalink / raw)
To: zsh-workers
[-- Attachment #1: Type: text/plain, Size: 2353 bytes --]
Hello
I've run callgrind on Zsh, when executing syntax-highlighting code that
parses 823 lines of code:
2,269,560,047 ???:mb_metacharlenconv_r [/usr/local/bin/zsh-debug-opt]
1,698,947,505 ???:remnulargs [/usr/local/bin/zsh-debug-opt]
1,677,804,272 ???:_UTF8_mbrtowc [/usr/lib/system/libsystem_c.dylib]
1,425,973,736 ???:mbrtowc [/usr/lib/system/libsystem_c.dylib]
1,177,994,701 ???:untokenize [/usr/local/bin/zsh-debug-opt]
1,048,181,974 ???:mb_metacharlenconv [/usr/local/bin/zsh-debug-opt]
1,036,055,574 ???:getindex'2 [/usr/local/bin/zsh-debug-opt]
793,202,632 ???:haswilds [/usr/local/bin/zsh-debug-opt]
578,630,988 ???:mb_metastrlenend [/usr/local/bin/zsh-debug-opt]
483,051,992 ???:szone_free_definite_size
[/usr/lib/system/libsystem_malloc.dylib]
436,411,797 ???:ztrsub [/usr/local/bin/zsh-debug-opt]
364,444,476 ???:tiny_malloc_from_free_list
[/usr/lib/system/libsystem_malloc.dylib]
353,826,375 ???:pattrylen'2 [/usr/local/bin/zsh-debug-opt]
280,090,072 ???:tiny_free_list_add_ptr
[/usr/lib/system/libsystem_malloc.dylib]
258,502,596 ???:strlen [/usr/lib/dyld]
234,273,918 ???:pattrylen [/usr/local/bin/zsh-debug-opt]
209,835,520 ???:szone_size [/usr/lib/system/libsystem_malloc.dylib]
To repeat the run clone
https://github.com/psprint/history-search-multi-word/ and add "valgrind
--tool=callgrind" before "zsh" (after exec) in parse.zsh, then run
./parse.zsh ./to-parse.zsh. I think this is a very good real world test.
Seems that Zsh execution could be greatly optimized if functions:
remnulargs, untokenize, haswilds could be optimized. Not sure if the
results are reasonable, as haswilds just iterates over a string and does
quite basic switch. The other two functions have nested loops, so they
look more likely as being time consuming. Maybe the nested loop can be
changed to something else?
Other pointed functions seem to be very valid / expected – multibyte
functions. They can be optimized if a courageous decision will be made –
to do what charnext / pattern.c does:
if (!(patglobflags & GF_MULTIBYTE) || !(STOUC(*x) & 0x80))
return x + 1;
I.e. to optimize for ASCII as subset of UTF-8 also when calling
MB_METACHARLEN, not only for MB_METASTRLEN (recent change).
--
Sebastian Gniazdowski
psprint@fastmail.com
[-- Attachment #2: callgrind_annotate.txt --]
[-- Type: text/plain, Size: 7036 bytes --]
--------------------------------------------------------------------------------
Profile data file 'callgrind.out.11879' (creator: callgrind-3.12.0)
--------------------------------------------------------------------------------
I1 cache:
D1 cache:
LL cache:
Timerange: Basic block 0 - 2995164135
Trigger: Program termination
Profiled target: zsh-debug-opt -f -c source "./testparse.zsh" "./to-parse.zsh" "changes.out" "" (PID 11879, part 1)
Events recorded: Ir
Events shown: Ir
Event sort order: Ir
Thresholds: 99
Include dirs:
User annotated:
Auto-annotation: off
--------------------------------------------------------------------------------
Ir
--------------------------------------------------------------------------------
16,735,388,538 PROGRAM TOTALS
--------------------------------------------------------------------------------
Ir file:function
--------------------------------------------------------------------------------
2,269,560,047 ???:mb_metacharlenconv_r [/usr/local/bin/zsh-debug-opt]
1,698,947,505 ???:remnulargs [/usr/local/bin/zsh-debug-opt]
1,677,804,272 ???:_UTF8_mbrtowc [/usr/lib/system/libsystem_c.dylib]
1,425,973,736 ???:mbrtowc [/usr/lib/system/libsystem_c.dylib]
1,177,994,701 ???:untokenize [/usr/local/bin/zsh-debug-opt]
1,048,181,974 ???:mb_metacharlenconv [/usr/local/bin/zsh-debug-opt]
1,036,055,574 ???:getindex'2 [/usr/local/bin/zsh-debug-opt]
793,202,632 ???:haswilds [/usr/local/bin/zsh-debug-opt]
578,630,988 ???:mb_metastrlenend [/usr/local/bin/zsh-debug-opt]
483,051,992 ???:szone_free_definite_size [/usr/lib/system/libsystem_malloc.dylib]
436,411,797 ???:ztrsub [/usr/local/bin/zsh-debug-opt]
364,444,476 ???:tiny_malloc_from_free_list [/usr/lib/system/libsystem_malloc.dylib]
353,826,375 ???:pattrylen'2 [/usr/local/bin/zsh-debug-opt]
280,090,072 ???:tiny_free_list_add_ptr [/usr/lib/system/libsystem_malloc.dylib]
258,502,596 ???:strlen [/usr/lib/dyld]
234,273,918 ???:pattrylen [/usr/local/bin/zsh-debug-opt]
209,835,520 ???:szone_size [/usr/lib/system/libsystem_malloc.dylib]
193,985,837 ???:tiny_free_list_remove_ptr [/usr/lib/system/libsystem_malloc.dylib]
169,580,182 ???:szone_malloc_should_clear [/usr/lib/system/libsystem_malloc.dylib]
143,109,122 ???:_platform_memmove$VARIANT$Nehalem [/usr/lib/system/libsystem_platform.dylib]
97,432,800 ???:free [/usr/lib/dyld]
97,335,179 ???:itype_end [/usr/local/bin/zsh-debug-opt]
95,353,820 ???:get_tiny_free_size [/usr/lib/system/libsystem_malloc.dylib]
83,934,500 ???:pthread_getspecific [/usr/lib/system/libsystem_pthread.dylib]
81,015,036 ???:filesub [/usr/local/bin/zsh-debug-opt]
68,738,845 ???:__strcpy_chk [/usr/lib/system/libsystem_c.dylib]
60,927,832 ???:malloc_zone_malloc [/usr/lib/system/libsystem_malloc.dylib]
57,698,352 ???:zalloc [/usr/local/bin/zsh-debug-opt]
55,196,289 ???:bin_log [/usr/local/bin/zsh-debug-opt]
54,517,015 ???:stpcpy [/usr/lib/system/libsystem_c.dylib]
51,545,105 ???:setarrvalue [/usr/local/bin/zsh-debug-opt]
49,052,650 ???:get_tiny_previous_free_msize [/usr/lib/system/libsystem_malloc.dylib]
48,122,314 ???:ztrdup [/usr/local/bin/zsh-debug-opt]
45,371,076 ???:mathevalarg'2 [/usr/local/bin/zsh-debug-opt]
44,923,221 ???:arrlen [/usr/local/bin/zsh-debug-opt]
44,888,769 ???:__vsnprintf_chk [/usr/lib/system/libsystem_c.dylib]
43,521,301 ???:malloc [/usr/lib/dyld]
33,548,312 ???:__chk_overlap [/usr/lib/system/libsystem_c.dylib]
33,378,378 ???:execlist'2 [/usr/local/bin/zsh-debug-opt]
32,027,315 ???:_platform_memset$VARIANT$Merom [/usr/lib/system/libsystem_platform.dylib]
29,584,698 ???:_platform_strchr$VARIANT$Generic [/usr/lib/system/libsystem_platform.dylib]
28,786,904 ???:hasher [/usr/local/bin/zsh-debug-opt]
25,459,319 ???:zhalloc [/usr/local/bin/zsh-debug-opt]
25,436,057 ???:modify [/usr/local/bin/zsh-debug-opt]
23,233,085 ???:patcompile'2 [/usr/local/bin/zsh-debug-opt]
23,114,835 ???:zsfree [/usr/local/bin/zsh-debug-opt]
21,720,915 ???:_os_lock_spin_lock [/usr/lib/system/libsystem_platform.dylib]
21,033,364 ???:execrestore'2 [/usr/local/bin/zsh-debug-opt]
21,029,416 ???:ingetc [/usr/local/bin/zsh-debug-opt]
20,619,575 ???:freearray [/usr/local/bin/zsh-debug-opt]
18,246,076 ???:fetchvalue [/usr/local/bin/zsh-debug-opt]
17,288,068 ???:isascii [/usr/lib/system/libsystem_c.dylib]
16,274,888 ???:filesub'2 [/usr/local/bin/zsh-debug-opt]
15,162,279 ???:haswilds'2 [/usr/local/bin/zsh-debug-opt]
12,590,562 ???:parsestrnoerr [/usr/local/bin/zsh-debug-opt]
10,881,530 ???:szone_malloc [/usr/lib/system/libsystem_malloc.dylib]
10,206,971 ???:zstrtol_underscore [/usr/local/bin/zsh-debug-opt]
9,997,508 ???:_pthread_mutex_unlock_slow [/usr/lib/system/libsystem_pthread.dylib]
9,639,468 ???:_platform_strcmp [/usr/lib/system/libsystem_platform.dylib]
9,404,226 ???:modify'2 [/usr/local/bin/zsh-debug-opt]
8,688,566 ???:os_lock_unlock [/usr/lib/system/libsystem_platform.dylib]
8,688,566 ???:os_lock_lock [/usr/lib/system/libsystem_platform.dylib]
8,688,366 ???:_os_lock_spin_unlock [/usr/lib/system/libsystem_platform.dylib]
8,497,692 ???:op [/usr/local/bin/zsh-debug-opt]
8,390,890 ???:prefork [/usr/local/bin/zsh-debug-opt]
8,223,809 ???:patcompstart [/usr/local/bin/zsh-debug-opt]
7,962,974 ???:gethashnode2 [/usr/local/bin/zsh-debug-opt]
7,766,705 ???:scanmatchtable [/usr/local/bin/zsh-debug-opt]
7,013,693 ???:parsestrnoerr'2 [/usr/local/bin/zsh-debug-opt]
6,917,568 ???:op'2 [/usr/local/bin/zsh-debug-opt]
6,909,521 ???:getindex [/usr/local/bin/zsh-debug-opt]
6,827,173 ???:_pthread_mutex_lock_slow [/usr/lib/system/libsystem_pthread.dylib]
6,691,178 ???:hasbraces [/usr/local/bin/zsh-debug-opt]
6,601,957 ???:mathevalarg [/usr/local/bin/zsh-debug-opt]
6,523,091 ???:get_node_from_uniquing_table [/usr/lib/system/libsystem_malloc.dylib]
6,465,840 ???:ecgetstr [/usr/local/bin/zsh-debug-opt]
6,193,899 ???:getstrvalue [/usr/local/bin/zsh-debug-opt]
6,178,975 ???:matheval'2 [/usr/local/bin/zsh-debug-opt]
6,012,923 ???:patcompile [/usr/local/bin/zsh-debug-opt]
5,721,631 ???:ImageLoaderMachOCompressed::trieWalk(unsigned char const*, unsigned char const*, char const*) [/usr/lib/dyld]
5,083,953 ???:add [/usr/local/bin/zsh-debug-opt]
4,864,901 ???:__vsnprintf_chk'2 [/usr/lib/system/libsystem_c.dylib]
4,728,297 ???:dupstring [/usr/local/bin/zsh-debug-opt]
4,331,738 ???:fetchvalue'2 [/usr/local/bin/zsh-debug-opt]
4,212,315 ???:newparamtable [/usr/local/bin/zsh-debug-opt]
4,034,794 ???:__vfprintf [/usr/lib/system/libsystem_c.dylib]
3,977,804 ???:pattryrefs [/usr/local/bin/zsh-debug-opt]
3,604,588 ???:assignstrvalue [/usr/local/bin/zsh-debug-opt]
3,520,423 ???:matheval [/usr/local/bin/zsh-debug-opt]
3,518,560 ???:mb_charinit [/usr/local/bin/zsh-debug-opt]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Callgrind run
2016-11-10 10:37 ` Callgrind run Sebastian Gniazdowski
@ 2016-11-10 12:31 ` Peter Stephenson
2016-11-10 14:07 ` Sebastian Gniazdowski
2016-11-10 13:47 ` multibyte optimisations Peter Stephenson
1 sibling, 1 reply; 5+ messages in thread
From: Peter Stephenson @ 2016-11-10 12:31 UTC (permalink / raw)
To: zsh-workers
On Thu, 10 Nov 2016 02:37:12 -0800
Sebastian Gniazdowski <psprint@fastmail.com> wrote:
> Seems that Zsh execution could be greatly optimized if functions:
> remnulargs, untokenize, haswilds could be optimized. Not sure if the
> results are reasonable, as haswilds just iterates over a string and does
> quite basic switch. The other two functions have nested loops, so they
> look more likely as being time consuming. Maybe the nested loop can be
> changed to something else?
The nested loops aren't "real" nested loops; the inner loop runs to
completion and then breaks if the outer loop detects a condition that
needs handling.
To do a good job optimising here, we really need state information
outside the functions --- in an experiment with my start up files, only
16% of calls to untokenize() actually had any effect. But recording the
state generally is a very big change.
Some possible optimisations are along the following lines, although a
bit of care it's needed as it's not necessarily the case on all
architectures that the bit test used by itok() is necessarily faster
than the range test the following replaces it with. It did seem faster
on this fairly standard Intel CPU.
I probably won't be committing this.
diff --git a/Src/exec.c b/Src/exec.c
index a01a633..a6b01a6 100644
--- a/Src/exec.c
+++ b/Src/exec.c
@@ -1953,26 +1953,24 @@ makecline(LinkList list)
mod_export void
untokenize(char *s)
{
- if (*s) {
+ if (*s) { /* "" may be a const string. Ick. */
int c;
- while ((c = *s++))
- if (itok(c)) {
+ while ((c = *s++)) {
+ if (c >= FIRST_TOK && c <= LAST_TOK) {
char *p = s - 1;
if (c != Nularg)
- *p++ = ztokens[c - Pound];
+ *p++ = ztoken_to_char[STOUC(c)];
while ((c = *s++)) {
- if (itok(c)) {
- if (c != Nularg)
- *p++ = ztokens[c - Pound];
- } else
- *p++ = c;
+ if (c != Nularg)
+ *p++ = ztoken_to_char[STOUC(c)];
}
*p = '\0';
break;
}
+ }
}
}
diff --git a/Src/glob.c b/Src/glob.c
index 50f6dce..4d3fc51 100644
--- a/Src/glob.c
+++ b/Src/glob.c
@@ -3570,7 +3570,7 @@ remnulargs(char *s)
if (*s) {
char *o = s, c;
- while ((c = *s++))
+ while ((c = *s++)) {
if (c == Bnullkeep) {
/*
* An active backslash that needs to be turned back into
@@ -3579,7 +3579,7 @@ remnulargs(char *s)
* pattern matching.
*/
continue;
- } else if (inull(c)) {
+ } else if (c >= FIRST_NULL && c <= LAST_NULL) {
char *t = s - 1;
while ((c = *s++)) {
@@ -3595,6 +3595,7 @@ remnulargs(char *s)
}
break;
}
+ }
}
}
diff --git a/Src/lex.c b/Src/lex.c
index 8896128..bfd6b11 100644
--- a/Src/lex.c
+++ b/Src/lex.c
@@ -37,6 +37,18 @@
/**/
mod_export char ztokens[] = "#$^*(())$=|{}[]`<>>?~`,-!'\"\\\\";
+/*
+ * Map a possibly tokenized unsigned char to a normal unsigned
+ * char, for use in untokenize().
+ *
+ * Tokens that need untokenizing (everything in ztokens except Nularg)
+ * map to a different character, everything else maps to itself.
+ * In particular, metafied characters are passed through unchanged
+ * (effectively escaping tokens) and do not need special handling.
+ */
+/**/
+mod_export char ztoken_to_char[256];
+
/* parts of the current token */
/**/
diff --git a/Src/utils.c b/Src/utils.c
index 3d535b8..9fa8a97 100644
--- a/Src/utils.c
+++ b/Src/utils.c
@@ -4012,6 +4012,18 @@ inittyptab(void)
for (s = PATCHARS; *s; s++)
typtab[STOUC(*s)] |= IPATTERN;
+ for (t0 = 0; t0 < 256; t0++)
+ {
+ if (itok(t0) && (char)t0 != Nularg)
+ {
+ ztoken_to_char[t0] = ztokens[t0 - STOUC(Pound)];
+ }
+ else
+ {
+ ztoken_to_char[t0] = (char)t0;
+ }
+ }
+
unqueue_signals();
}
diff --git a/Src/zsh.h b/Src/zsh.h
index a5d4455..5065a54 100644
--- a/Src/zsh.h
+++ b/Src/zsh.h
@@ -170,6 +170,7 @@ struct mathfunc {
* These should match the characters in ztokens, defined in lex.c
*/
#define Pound ((char) 0x84)
+#define FIRST_TOK Pound
#define String ((char) 0x85)
#define Hat ((char) 0x86)
#define Star ((char) 0x87)
@@ -204,6 +205,7 @@ struct mathfunc {
* and backslashes.
*/
#define Snull ((char) 0x9d)
+#define FIRST_NULL Snull
#define Dnull ((char) 0x9e)
#define Bnull ((char) 0x9f)
/*
@@ -217,6 +219,8 @@ struct mathfunc {
* is used to initialise the IMETA type in inittyptab().
*/
#define Nularg ((char) 0xa1)
+#define LAST_TOK Nularg
+#define LAST_NULL Nularg
/*
* Take care to update the use of IMETA appropriately when adding
^ permalink raw reply [flat|nested] 5+ messages in thread
* multibyte optimisations
2016-11-10 10:37 ` Callgrind run Sebastian Gniazdowski
2016-11-10 12:31 ` Peter Stephenson
@ 2016-11-10 13:47 ` Peter Stephenson
2016-11-10 14:57 ` Sebastian Gniazdowski
1 sibling, 1 reply; 5+ messages in thread
From: Peter Stephenson @ 2016-11-10 13:47 UTC (permalink / raw)
To: zsh-workers
On Thu, 10 Nov 2016 02:37:12 -0800
Sebastian Gniazdowski <psprint@fastmail.com> wrote:
> Other pointed functions seem to be very valid / expected – multibyte
> functions. They can be optimized if a courageous decision will be made –
> to do what charnext / pattern.c does:
>
> if (!(patglobflags & GF_MULTIBYTE) || !(STOUC(*x) & 0x80))
> return x + 1;
>
> I.e. to optimize for ASCII as subset of UTF-8 also when calling
> MB_METACHARLEN, not only for MB_METASTRLEN (recent change).
These look straightforward and along the same lines as what we already
do.
pws
diff --git a/Src/utils.c b/Src/utils.c
index 3d535b8..cceaf4c 100644
--- a/Src/utils.c
+++ b/Src/utils.c
@@ -84,7 +84,15 @@ set_widearray(char *mb_array, Widechar_array wca)
mb_charinit();
while (*mb_array) {
- int mblen = mb_metacharlenconv(mb_array, &wci);
+ int mblen;
+
+ if (STOUC(*mb_array) <= 0x7f) {
+ mb_array++;
+ *wcptr++ = (wchar_t)*mb_array;
+ continue;
+ }
+
+ mblen = mb_metacharlenconv(mb_array, &wci);
if (!mblen)
break;
@@ -5249,6 +5257,12 @@ mb_metacharlenconv_r(const char *s, wint_t *wcp, mbstate_t *mbsp)
const char *ptr;
wchar_t wc;
+ if (STOUC(*s) <= 0x7f) {
+ if (wcp)
+ *wcp = (wint_t)*s;
+ return 1;
+ }
+
for (ptr = s; *ptr; ) {
if (*ptr == Meta) {
inchar = *++ptr ^ 32;
@@ -5301,7 +5315,7 @@ mb_metacharlenconv_r(const char *s, wint_t *wcp, mbstate_t *mbsp)
mod_export int
mb_metacharlenconv(const char *s, wint_t *wcp)
{
- if (!isset(MULTIBYTE)) {
+ if (!isset(MULTIBYTE) || STOUC(*s) <= 0x7f) {
/* treat as single byte, possibly metafied */
if (wcp)
*wcp = (wint_t)(*s == Meta ? s[1] ^ 32 : *s);
@@ -5442,6 +5456,12 @@ mb_charlenconv_r(const char *s, int slen, wint_t *wcp, mbstate_t *mbsp)
const char *ptr;
wchar_t wc;
+ if (slen && STOUC(*s) <= 0x7f) {
+ if (wcp)
+ *wcp = (wint_t)*s;
+ return 1;
+ }
+
for (ptr = s; slen; ) {
inchar = *ptr;
ptr++;
@@ -5477,7 +5497,7 @@ mb_charlenconv_r(const char *s, int slen, wint_t *wcp, mbstate_t *mbsp)
mod_export int
mb_charlenconv(const char *s, int slen, wint_t *wcp)
{
- if (!isset(MULTIBYTE)) {
+ if (!isset(MULTIBYTE) || STOUC(*s) <= 0x7f) {
if (wcp)
*wcp = (wint_t)*s;
return 1;
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Callgrind run
2016-11-10 12:31 ` Peter Stephenson
@ 2016-11-10 14:07 ` Sebastian Gniazdowski
0 siblings, 0 replies; 5+ messages in thread
From: Sebastian Gniazdowski @ 2016-11-10 14:07 UTC (permalink / raw)
To: zsh-workers
[-- Attachment #1: Type: text/plain, Size: 1388 bytes --]
On Thu, Nov 10, 2016, at 04:31 AM, Peter Stephenson wrote:
> To do a good job optimising here, we really need state information
> outside the functions --- in an experiment with my start up files, only
> 16% of calls to untokenize() actually had any effect. But recording the
> state generally is a very big change.
>
> Some possible optimisations are along the following lines, although a
> bit of care it's needed as it's not necessarily the case on all
> architectures that the bit test used by itok() is necessarily faster
> than the range test the following replaces it with. It did seem faster
> on this fairly standard Intel CPU.
Tested this and no big change, maybe 14 ms – running times are 2135 vs
2149, but that can be just instability. However callgrind reports
851,712,174 instructions instead of 1,177,994,701 for untokenize, while
other instruction counts are kept the same so the test seems valid.
My motivation is parsing of long Zsh code – would be a cool thing to
iterate long (z)-splitted input in say 400 ms instead of 2 seconds – a
dreamed result, maybe actually impossible, as disabling multibyte yields
1560 ms. State recording might seem bad but at least there is room for
improvement condensed in apparently few places, better than counting
cycles along whole Zsh code.
--
Sebastian Gniazdowski
psprint@fastmail.com
[-- Attachment #2: callgrind_annotate3.txt --]
[-- Type: text/plain, Size: 7316 bytes --]
--------------------------------------------------------------------------------
Profile data file 'callgrind.out.37869' (creator: callgrind-3.12.0)
--------------------------------------------------------------------------------
I1 cache:
D1 cache:
LL cache:
Timerange: Basic block 0 - 3061023589
Trigger: Program termination
Profiled target: zsh-ps-debug-opt -f -c source "./testparse.zsh" "./to-parse.zsh" "changes.out" "" (PID 37869, part 1)
Events recorded: Ir
Events shown: Ir
Event sort order: Ir
Thresholds: 99
Include dirs:
User annotated:
Auto-annotation: off
--------------------------------------------------------------------------------
Ir
--------------------------------------------------------------------------------
16,408,775,049 PROGRAM TOTALS
--------------------------------------------------------------------------------
Ir file:function
--------------------------------------------------------------------------------
2,269,560,047 ???:mb_metacharlenconv_r [/usr/local/bin/zsh-ps-debug-opt]
1,697,840,717 ???:remnulargs [/usr/local/bin/zsh-ps-debug-opt]
1,677,804,272 ???:_UTF8_mbrtowc [/usr/lib/system/libsystem_c.dylib]
1,425,973,736 ???:mbrtowc [/usr/lib/system/libsystem_c.dylib]
1,048,181,974 ???:mb_metacharlenconv [/usr/local/bin/zsh-ps-debug-opt]
1,036,055,574 ???:getindex'2 [/usr/local/bin/zsh-ps-debug-opt]
851,712,174 ???:untokenize [/usr/local/bin/zsh-ps-debug-opt]
793,202,632 ???:haswilds [/usr/local/bin/zsh-ps-debug-opt]
578,630,988 ???:mb_metastrlenend [/usr/local/bin/zsh-ps-debug-opt]
482,828,373 ???:szone_free_definite_size [/usr/lib/system/libsystem_malloc.dylib]
436,411,797 ???:ztrsub [/usr/local/bin/zsh-ps-debug-opt]
363,212,196 ???:tiny_malloc_from_free_list [/usr/lib/system/libsystem_malloc.dylib]
353,826,375 ???:pattrylen'2 [/usr/local/bin/zsh-ps-debug-opt]
282,357,130 ???:tiny_free_list_add_ptr [/usr/lib/system/libsystem_malloc.dylib]
258,502,798 ???:strlen [/usr/lib/dyld]
234,273,918 ???:pattrylen [/usr/local/bin/zsh-ps-debug-opt]
209,831,892 ???:szone_size [/usr/lib/system/libsystem_malloc.dylib]
193,951,431 ???:tiny_free_list_remove_ptr [/usr/lib/system/libsystem_malloc.dylib]
169,581,080 ???:szone_malloc_should_clear [/usr/lib/system/libsystem_malloc.dylib]
143,108,999 ???:_platform_memmove$VARIANT$Nehalem [/usr/lib/system/libsystem_platform.dylib]
97,432,800 ???:free [/usr/lib/system/libsystem_malloc.dylib]
97,335,179 ???:itype_end [/usr/local/bin/zsh-ps-debug-opt]
95,268,036 ???:get_tiny_free_size [/usr/lib/system/libsystem_malloc.dylib]
83,934,500 ???:pthread_getspecific [/usr/lib/system/libsystem_pthread.dylib]
81,015,036 ???:filesub [/usr/local/bin/zsh-ps-debug-opt]
68,739,019 ???:__strcpy_chk [/usr/lib/system/libsystem_c.dylib]
60,928,000 ???:malloc_zone_malloc [/usr/lib/system/libsystem_malloc.dylib]
57,698,433 ???:zalloc [/usr/local/bin/zsh-ps-debug-opt]
55,196,334 ???:bin_log [/usr/local/bin/zsh-ps-debug-opt]
54,517,153 ???:stpcpy [/usr/lib/system/libsystem_c.dylib]
51,545,105 ???:setarrvalue [/usr/local/bin/zsh-ps-debug-opt]
49,058,372 ???:get_tiny_previous_free_msize [/usr/lib/system/libsystem_malloc.dylib]
48,122,383 ???:ztrdup [/usr/local/bin/zsh-ps-debug-opt]
45,371,076 ???:mathevalarg'2 [/usr/local/bin/zsh-ps-debug-opt]
44,923,221 ???:arrlen [/usr/local/bin/zsh-ps-debug-opt]
44,888,797 ???:__vsnprintf_chk [/usr/lib/system/libsystem_c.dylib]
43,521,421 ???:malloc [/usr/lib/system/libsystem_malloc.dylib]
33,548,396 ???:__chk_overlap [/usr/lib/system/libsystem_c.dylib]
33,378,378 ???:execlist'2 [/usr/local/bin/zsh-ps-debug-opt]
32,027,396 ???:_platform_memset$VARIANT$Merom [/usr/lib/system/libsystem_platform.dylib]
29,584,698 ???:_platform_strchr$VARIANT$Generic [/usr/lib/system/libsystem_platform.dylib]
28,788,128 ???:hasher [/usr/local/bin/zsh-ps-debug-opt]
25,459,319 ???:zhalloc [/usr/local/bin/zsh-ps-debug-opt]
25,436,057 ???:modify [/usr/local/bin/zsh-ps-debug-opt]
23,233,085 ???:patcompile'2 [/usr/local/bin/zsh-ps-debug-opt]
23,114,835 ???:zsfree [/usr/local/bin/zsh-ps-debug-opt]
21,720,950 ???:_os_lock_spin_lock [/usr/lib/system/libsystem_platform.dylib]
21,033,364 ???:execrestore'2 [/usr/local/bin/zsh-ps-debug-opt]
21,029,575 ???:ingetc [/usr/local/bin/zsh-ps-debug-opt]
20,619,575 ???:freearray [/usr/local/bin/zsh-ps-debug-opt]
18,246,076 ???:fetchvalue [/usr/local/bin/zsh-ps-debug-opt]
17,288,068 ???:isascii [/usr/lib/system/libsystem_c.dylib]
16,274,888 ???:filesub'2 [/usr/local/bin/zsh-ps-debug-opt]
15,162,279 ???:haswilds'2 [/usr/local/bin/zsh-ps-debug-opt]
12,590,562 ???:parsestrnoerr [/usr/local/bin/zsh-ps-debug-opt]
10,881,555 ???:szone_malloc [/usr/lib/system/libsystem_malloc.dylib]
10,206,971 ???:zstrtol_underscore [/usr/local/bin/zsh-ps-debug-opt]
9,997,820 ???:_pthread_mutex_unlock_slow [/usr/lib/system/libsystem_pthread.dylib]
9,639,594 ???:_platform_strcmp [/usr/lib/system/libsystem_platform.dylib]
9,404,226 ???:modify'2 [/usr/local/bin/zsh-ps-debug-opt]
8,688,580 ???:os_lock_unlock [/usr/lib/system/libsystem_platform.dylib]
8,688,580 ???:os_lock_lock [/usr/lib/system/libsystem_platform.dylib]
8,688,380 ???:_os_lock_spin_unlock [/usr/lib/system/libsystem_platform.dylib]
8,497,692 ???:op [/usr/local/bin/zsh-ps-debug-opt]
8,390,890 ???:prefork [/usr/local/bin/zsh-ps-debug-opt]
8,223,809 ???:patcompstart [/usr/local/bin/zsh-ps-debug-opt]
7,963,097 ???:gethashnode2 [/usr/local/bin/zsh-ps-debug-opt]
7,766,705 ???:scanmatchtable [/usr/local/bin/zsh-ps-debug-opt]
7,013,693 ???:parsestrnoerr'2 [/usr/local/bin/zsh-ps-debug-opt]
6,917,568 ???:op'2 [/usr/local/bin/zsh-ps-debug-opt]
6,909,521 ???:getindex [/usr/local/bin/zsh-ps-debug-opt]
6,827,386 ???:_pthread_mutex_lock_slow [/usr/lib/system/libsystem_pthread.dylib]
6,691,178 ???:hasbraces [/usr/local/bin/zsh-ps-debug-opt]
6,601,957 ???:mathevalarg [/usr/local/bin/zsh-ps-debug-opt]
6,523,105 ???:get_node_from_uniquing_table [/usr/lib/system/libsystem_malloc.dylib]
6,465,840 ???:ecgetstr [/usr/local/bin/zsh-ps-debug-opt]
6,193,899 ???:getstrvalue [/usr/local/bin/zsh-ps-debug-opt]
6,178,975 ???:matheval'2 [/usr/local/bin/zsh-ps-debug-opt]
6,012,923 ???:patcompile [/usr/local/bin/zsh-ps-debug-opt]
5,721,631 ???:ImageLoaderMachOCompressed::trieWalk(unsigned char const*, unsigned char const*, char const*) [/usr/lib/dyld]
5,084,013 ???:add [/usr/local/bin/zsh-ps-debug-opt]
4,864,913 ???:__vsnprintf_chk'2 [/usr/lib/system/libsystem_c.dylib]
4,728,297 ???:dupstring [/usr/local/bin/zsh-ps-debug-opt]
4,331,738 ???:fetchvalue'2 [/usr/local/bin/zsh-ps-debug-opt]
4,212,315 ???:newparamtable [/usr/local/bin/zsh-ps-debug-opt]
4,034,794 ???:__vfprintf [/usr/lib/system/libsystem_c.dylib]
3,977,804 ???:pattryrefs [/usr/local/bin/zsh-ps-debug-opt]
3,604,588 ???:assignstrvalue [/usr/local/bin/zsh-ps-debug-opt]
3,520,423 ???:matheval [/usr/local/bin/zsh-ps-debug-opt]
3,518,560 ???:mb_charinit [/usr/local/bin/zsh-ps-debug-opt]
3,487,929 ???:freeheap [/usr/local/bin/zsh-ps-debug-opt]
[-- Attachment #3: callgrind_annotate.txt --]
[-- Type: text/plain, Size: 7036 bytes --]
--------------------------------------------------------------------------------
Profile data file 'callgrind.out.11879' (creator: callgrind-3.12.0)
--------------------------------------------------------------------------------
I1 cache:
D1 cache:
LL cache:
Timerange: Basic block 0 - 2995164135
Trigger: Program termination
Profiled target: zsh-debug-opt -f -c source "./testparse.zsh" "./to-parse.zsh" "changes.out" "" (PID 11879, part 1)
Events recorded: Ir
Events shown: Ir
Event sort order: Ir
Thresholds: 99
Include dirs:
User annotated:
Auto-annotation: off
--------------------------------------------------------------------------------
Ir
--------------------------------------------------------------------------------
16,735,388,538 PROGRAM TOTALS
--------------------------------------------------------------------------------
Ir file:function
--------------------------------------------------------------------------------
2,269,560,047 ???:mb_metacharlenconv_r [/usr/local/bin/zsh-debug-opt]
1,698,947,505 ???:remnulargs [/usr/local/bin/zsh-debug-opt]
1,677,804,272 ???:_UTF8_mbrtowc [/usr/lib/system/libsystem_c.dylib]
1,425,973,736 ???:mbrtowc [/usr/lib/system/libsystem_c.dylib]
1,177,994,701 ???:untokenize [/usr/local/bin/zsh-debug-opt]
1,048,181,974 ???:mb_metacharlenconv [/usr/local/bin/zsh-debug-opt]
1,036,055,574 ???:getindex'2 [/usr/local/bin/zsh-debug-opt]
793,202,632 ???:haswilds [/usr/local/bin/zsh-debug-opt]
578,630,988 ???:mb_metastrlenend [/usr/local/bin/zsh-debug-opt]
483,051,992 ???:szone_free_definite_size [/usr/lib/system/libsystem_malloc.dylib]
436,411,797 ???:ztrsub [/usr/local/bin/zsh-debug-opt]
364,444,476 ???:tiny_malloc_from_free_list [/usr/lib/system/libsystem_malloc.dylib]
353,826,375 ???:pattrylen'2 [/usr/local/bin/zsh-debug-opt]
280,090,072 ???:tiny_free_list_add_ptr [/usr/lib/system/libsystem_malloc.dylib]
258,502,596 ???:strlen [/usr/lib/dyld]
234,273,918 ???:pattrylen [/usr/local/bin/zsh-debug-opt]
209,835,520 ???:szone_size [/usr/lib/system/libsystem_malloc.dylib]
193,985,837 ???:tiny_free_list_remove_ptr [/usr/lib/system/libsystem_malloc.dylib]
169,580,182 ???:szone_malloc_should_clear [/usr/lib/system/libsystem_malloc.dylib]
143,109,122 ???:_platform_memmove$VARIANT$Nehalem [/usr/lib/system/libsystem_platform.dylib]
97,432,800 ???:free [/usr/lib/dyld]
97,335,179 ???:itype_end [/usr/local/bin/zsh-debug-opt]
95,353,820 ???:get_tiny_free_size [/usr/lib/system/libsystem_malloc.dylib]
83,934,500 ???:pthread_getspecific [/usr/lib/system/libsystem_pthread.dylib]
81,015,036 ???:filesub [/usr/local/bin/zsh-debug-opt]
68,738,845 ???:__strcpy_chk [/usr/lib/system/libsystem_c.dylib]
60,927,832 ???:malloc_zone_malloc [/usr/lib/system/libsystem_malloc.dylib]
57,698,352 ???:zalloc [/usr/local/bin/zsh-debug-opt]
55,196,289 ???:bin_log [/usr/local/bin/zsh-debug-opt]
54,517,015 ???:stpcpy [/usr/lib/system/libsystem_c.dylib]
51,545,105 ???:setarrvalue [/usr/local/bin/zsh-debug-opt]
49,052,650 ???:get_tiny_previous_free_msize [/usr/lib/system/libsystem_malloc.dylib]
48,122,314 ???:ztrdup [/usr/local/bin/zsh-debug-opt]
45,371,076 ???:mathevalarg'2 [/usr/local/bin/zsh-debug-opt]
44,923,221 ???:arrlen [/usr/local/bin/zsh-debug-opt]
44,888,769 ???:__vsnprintf_chk [/usr/lib/system/libsystem_c.dylib]
43,521,301 ???:malloc [/usr/lib/dyld]
33,548,312 ???:__chk_overlap [/usr/lib/system/libsystem_c.dylib]
33,378,378 ???:execlist'2 [/usr/local/bin/zsh-debug-opt]
32,027,315 ???:_platform_memset$VARIANT$Merom [/usr/lib/system/libsystem_platform.dylib]
29,584,698 ???:_platform_strchr$VARIANT$Generic [/usr/lib/system/libsystem_platform.dylib]
28,786,904 ???:hasher [/usr/local/bin/zsh-debug-opt]
25,459,319 ???:zhalloc [/usr/local/bin/zsh-debug-opt]
25,436,057 ???:modify [/usr/local/bin/zsh-debug-opt]
23,233,085 ???:patcompile'2 [/usr/local/bin/zsh-debug-opt]
23,114,835 ???:zsfree [/usr/local/bin/zsh-debug-opt]
21,720,915 ???:_os_lock_spin_lock [/usr/lib/system/libsystem_platform.dylib]
21,033,364 ???:execrestore'2 [/usr/local/bin/zsh-debug-opt]
21,029,416 ???:ingetc [/usr/local/bin/zsh-debug-opt]
20,619,575 ???:freearray [/usr/local/bin/zsh-debug-opt]
18,246,076 ???:fetchvalue [/usr/local/bin/zsh-debug-opt]
17,288,068 ???:isascii [/usr/lib/system/libsystem_c.dylib]
16,274,888 ???:filesub'2 [/usr/local/bin/zsh-debug-opt]
15,162,279 ???:haswilds'2 [/usr/local/bin/zsh-debug-opt]
12,590,562 ???:parsestrnoerr [/usr/local/bin/zsh-debug-opt]
10,881,530 ???:szone_malloc [/usr/lib/system/libsystem_malloc.dylib]
10,206,971 ???:zstrtol_underscore [/usr/local/bin/zsh-debug-opt]
9,997,508 ???:_pthread_mutex_unlock_slow [/usr/lib/system/libsystem_pthread.dylib]
9,639,468 ???:_platform_strcmp [/usr/lib/system/libsystem_platform.dylib]
9,404,226 ???:modify'2 [/usr/local/bin/zsh-debug-opt]
8,688,566 ???:os_lock_unlock [/usr/lib/system/libsystem_platform.dylib]
8,688,566 ???:os_lock_lock [/usr/lib/system/libsystem_platform.dylib]
8,688,366 ???:_os_lock_spin_unlock [/usr/lib/system/libsystem_platform.dylib]
8,497,692 ???:op [/usr/local/bin/zsh-debug-opt]
8,390,890 ???:prefork [/usr/local/bin/zsh-debug-opt]
8,223,809 ???:patcompstart [/usr/local/bin/zsh-debug-opt]
7,962,974 ???:gethashnode2 [/usr/local/bin/zsh-debug-opt]
7,766,705 ???:scanmatchtable [/usr/local/bin/zsh-debug-opt]
7,013,693 ???:parsestrnoerr'2 [/usr/local/bin/zsh-debug-opt]
6,917,568 ???:op'2 [/usr/local/bin/zsh-debug-opt]
6,909,521 ???:getindex [/usr/local/bin/zsh-debug-opt]
6,827,173 ???:_pthread_mutex_lock_slow [/usr/lib/system/libsystem_pthread.dylib]
6,691,178 ???:hasbraces [/usr/local/bin/zsh-debug-opt]
6,601,957 ???:mathevalarg [/usr/local/bin/zsh-debug-opt]
6,523,091 ???:get_node_from_uniquing_table [/usr/lib/system/libsystem_malloc.dylib]
6,465,840 ???:ecgetstr [/usr/local/bin/zsh-debug-opt]
6,193,899 ???:getstrvalue [/usr/local/bin/zsh-debug-opt]
6,178,975 ???:matheval'2 [/usr/local/bin/zsh-debug-opt]
6,012,923 ???:patcompile [/usr/local/bin/zsh-debug-opt]
5,721,631 ???:ImageLoaderMachOCompressed::trieWalk(unsigned char const*, unsigned char const*, char const*) [/usr/lib/dyld]
5,083,953 ???:add [/usr/local/bin/zsh-debug-opt]
4,864,901 ???:__vsnprintf_chk'2 [/usr/lib/system/libsystem_c.dylib]
4,728,297 ???:dupstring [/usr/local/bin/zsh-debug-opt]
4,331,738 ???:fetchvalue'2 [/usr/local/bin/zsh-debug-opt]
4,212,315 ???:newparamtable [/usr/local/bin/zsh-debug-opt]
4,034,794 ???:__vfprintf [/usr/lib/system/libsystem_c.dylib]
3,977,804 ???:pattryrefs [/usr/local/bin/zsh-debug-opt]
3,604,588 ???:assignstrvalue [/usr/local/bin/zsh-debug-opt]
3,520,423 ???:matheval [/usr/local/bin/zsh-debug-opt]
3,518,560 ???:mb_charinit [/usr/local/bin/zsh-debug-opt]
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: multibyte optimisations
2016-11-10 13:47 ` multibyte optimisations Peter Stephenson
@ 2016-11-10 14:57 ` Sebastian Gniazdowski
0 siblings, 0 replies; 5+ messages in thread
From: Sebastian Gniazdowski @ 2016-11-10 14:57 UTC (permalink / raw)
To: zsh-workers
On Thu, Nov 10, 2016, at 05:47 AM, Peter Stephenson wrote:
> On Thu, 10 Nov 2016 02:37:12 -0800
> Sebastian Gniazdowski <psprint@fastmail.com> wrote:
> > Other pointed functions seem to be very valid / expected – multibyte
> > functions. They can be optimized if a courageous decision will be made –
> > to do what charnext / pattern.c does:
> >
> > if (!(patglobflags & GF_MULTIBYTE) || !(STOUC(*x) & 0x80))
> > return x + 1;
> >
> > I.e. to optimize for ASCII as subset of UTF-8 also when calling
> > MB_METACHARLEN, not only for MB_METASTRLEN (recent change).
>
> These look straightforward and along the same lines as what we already
> do.
Was worried that multibyte state can be not clear when requesting length
of character, but that cannot really happen, and if it would, then the
loop that advances char by char would have a problem, being in unclear
situation after recent advancement. With this patch the parser runs for
1493 ms instead of 2148 ms :)
--
Sebastian Gniazdowski
psprint@fastmail.com
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2016-11-10 14:57 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <CGME20161110103845epcas3p3e7cabeffae723219daafa8d3e6b32f12@epcas3p3.samsung.com>
2016-11-10 10:37 ` Callgrind run Sebastian Gniazdowski
2016-11-10 12:31 ` Peter Stephenson
2016-11-10 14:07 ` Sebastian Gniazdowski
2016-11-10 13:47 ` multibyte optimisations Peter Stephenson
2016-11-10 14:57 ` Sebastian Gniazdowski
Code repositories for project(s) associated with this public inbox
https://git.vuxu.org/mirror/zsh/
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).