From mboxrd@z Thu Jan 1 00:00:00 1970 X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on inbox.vuxu.org X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=MAILING_LIST_MULTI, RCVD_IN_MSPIKE_H2 autolearn=ham autolearn_force=no version=3.4.4 Received: (qmail 15891 invoked from network); 27 Jan 2023 06:21:22 -0000 Received: from second.openwall.net (193.110.157.125) by inbox.vuxu.org with ESMTPUTF8; 27 Jan 2023 06:21:22 -0000 Received: (qmail 11591 invoked by uid 550); 27 Jan 2023 06:21:19 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Reply-To: musl@lists.openwall.com Received: (qmail 11551 invoked from network); 27 Jan 2023 06:21:17 -0000 X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=nMxqkEQ2bH30Fa3+X2qxeG5OZqA+dezXYDErf1bU4zQ=; b=dV4weAj6zpmlFmMbc8n2YO0HHXuRsGM6xDldXJ8QD3Ubm2qvqpR/SXDm85zAv1G1XR 1Msf9GhIc+o1tON058N0aOw/Zws++INtB14VuQPA775Cr6LYHdC3X2LvdJ9IrK4ElJME 8U+06bQcuK31B3M3UTY8qEZjGQBT6nxlfqCmvQDMDoouzdOltl3zDRHdD7dv1KC6isr/ WKFI+vDjcZyth3RSYVa3hnU9JMfmAmtu6/XhZk0jwB7cPV9uVqpdGOQvO09Iyheenyxo c+XJplgEiIRsn/8XCVQLsy9l43nW6o3pDATpBFBpZOjAmMqe1kaLOfC7G4JQIm5NF99T yaDw== X-Gm-Message-State: AFqh2ko3TsIP+o4lwc04W7VqJB8baezrzniKUEoRGeDhZcLFjEj823bt elqxuMLgE+YOEdn9LAs+8LKmTDPQWadOYvuRd3TcgRAjrFLHwbZo5pg74VU39dUMbhlpaRTF2o0 gO+v1QVa2dRAhJjfCVHYXYw== X-Received: by 2002:a05:6a21:33a4:b0:b8:8208:a839 with SMTP id yy36-20020a056a2133a400b000b88208a839mr52458948pzb.20.1674800463538; Thu, 26 Jan 2023 22:21:03 -0800 (PST) X-Google-Smtp-Source: AMrXdXvD2kr3g526/0Bz/TFCg41THm5jyvF4bqELekJTZuYc/tNCtOklkPKOgErEELwyNfm3QRIjbw== X-Received: by 2002:a05:6a21:33a4:b0:b8:8208:a839 with SMTP id yy36-20020a056a2133a400b000b88208a839mr52458908pzb.20.1674800462845; Thu, 26 Jan 2023 22:21:02 -0800 (PST) Date: Fri, 27 Jan 2023 15:20:51 +0900 From: Dominique MARTINET To: Rich Felker Cc: musl@lists.openwall.com Message-ID: References: <20230124083747.GI4163@brightrain.aerifal.cx> <20230125055323.GK4163@brightrain.aerifal.cx> MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="sR1w1Z2nXlPD36o0" Content-Disposition: inline In-Reply-To: Subject: Re: [musl] infinite loop in mallocng's try_avail --sR1w1Z2nXlPD36o0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Dominique MARTINET wrote on Wed, Jan 25, 2023 at 03:48:37PM +0900: > I'll add a circular buffer to log things like the active[0] at entry and > its mask values, then set my board up to reproduce again, which will > probably bring us to next Monday. I've reproduced with that, it seems to confirm that we entered try_avail() with m->avail == 0 and the next element had freed == 0... (format: '__func__ (__LINE__): ', m is printed with %p, masks with %x -- lines moved due to the debug statements, I've attached both the patch and full log to this mail for history, however ugly the code is) In particular, m->next is logged as identical to m here, but when looking at gdb "almost immediately" after we can see that m->next isn't m anymore: ---- alloc_slot (324): 0x2436f40: avail 0, freed 0 try_avail (145): new m: 0x2436f88, avail 3ffffffe, freed 0 try_avail (171): mask 0, mem active_idx: 29, m/m->next 0x2436f88/0x2436f88 try_avail (178): BUGGED (gdb) p (*pm) $6 = (struct meta *) 0x2436f88 (gdb) p (*pm)->next $8 = (struct meta *) 0x2436ee0 ---- This is on a single core arm board (i.MX6 ULL), so there should be no room for cache problems, and there aren't any thread, but... openrc handles SIGCHLD, and I just confirmed it calls free() in its signal handler..... Since malloc/free aren't signal-safe, that explains everything we've seen and it's a bug I can now fix in openrc (also quite recomforting to confirm this isn't a musl bug) Thank you for your help! -- Dominique --sR1w1Z2nXlPD36o0 Content-Type: text/x-diff; charset=utf-8 Content-Description: 0001-mallocng-debug-statements.patch Content-Disposition: attachment; filename="0001-mallocng-debug-statements.patch" >From d95eba3d44b7e3154fc2a89755494f80d49e0e59 Mon Sep 17 00:00:00 2001 From: Dominique Martinet Date: Wed, 25 Jan 2023 16:19:37 +0900 Subject: [PATCH] mallocng debug statements --- src/malloc/mallocng/malloc.c | 43 ++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+) diff --git a/src/malloc/mallocng/malloc.c b/src/malloc/mallocng/malloc.c index d695ab8ec982..99855e7e0bd9 100644 --- a/src/malloc/mallocng/malloc.c +++ b/src/malloc/mallocng/malloc.c @@ -39,6 +39,24 @@ static const uint8_t med_cnt_tab[4] = { 28, 24, 20, 32 }; struct malloc_context ctx = { 0 }; +char dbg2_buf[1024*3] = { 0 }; +size_t dbg2_off = 0; + +#define dbg2(fmt, args...) do { \ + if (dbg2_off > sizeof(dbg2_buf) - 100) dbg2_off = 0; \ + dbg2_off += snprintf(dbg2_buf + dbg2_off, sizeof(dbg2_buf) - dbg2_off - 1, \ + "%s (%d): " fmt "\n", __func__, __LINE__, ##args); \ +} while (0) + +char dbg_buf[1024*100] = { 0 }; +size_t dbg_off = 0; + +#define dbg(fmt, args...) do { \ + if (dbg_off > sizeof(dbg_buf) - 200) dbg_off = 0; \ + dbg_off += snprintf(dbg_buf + dbg_off, sizeof(dbg_buf) - dbg_off - 1, \ + "%s (%d): " fmt "\n", __func__, __LINE__, ##args); \ +} while (0) + struct meta *alloc_meta(void) { struct meta *m; @@ -123,9 +141,13 @@ static uint32_t try_avail(struct meta **pm) dequeue(pm, m); m = *pm; if (!m) return 0; + if (m->sizeclass == 0) + dbg("new m: %p, avail %x, freed %x", m, m->avail_mask, m->freed_mask); } else { m = m->next; *pm = m; + if (m->sizeclass == 0) + dbg("new m: %p, avail %x, freed %x", m, m->avail_mask, m->freed_mask); } mask = m->freed_mask; @@ -136,6 +158,8 @@ static uint32_t try_avail(struct meta **pm) m = m->next; *pm = m; mask = m->freed_mask; + if (m->sizeclass == 0) + dbg("new m: %p, avail %x, freed %x", m, m->avail_mask, m->freed_mask); } // activate more slots in a not-fully-active group @@ -143,10 +167,19 @@ static uint32_t try_avail(struct meta **pm) // any other group with free slots. this avoids // touching & dirtying as-yet-unused pages. if (!(mask & ((2u<mem->active_idx)-1))) { + if (m->sizeclass == 0) { + dbg("mask %x, mem active_idx: %d, m/m->next %p/%p", mask, m->mem->active_idx, m, m->next); + } if (m->next != m) { m = m->next; *pm = m; } else { + if (m->sizeclass == 0) { + dbg("BUGGED"); + char msg[] = "\n\nSHOULD NEVER GET HERE!\n\n\n"; + write(2, msg, sizeof(msg)); + while(1); + } int cnt = m->mem->active_idx + 2; int size = size_classes[m->sizeclass]*UNIT; int span = UNIT + size*cnt; @@ -280,11 +313,18 @@ static struct meta *alloc_group(int sc, size_t req) m->last_idx = cnt-1; m->freeable = 1; m->sizeclass = sc; + dbg2("%p: sc %d idx %d", m, sc, active_idx); + dbg("%p: sc %d idx %d", m, sc, active_idx); return m; } static int alloc_slot(int sc, size_t req) { + if (sc == 0) { + dbg("%p: avail %x, freed %x", ctx.active[sc], + ctx.active[sc] ? ctx.active[sc]->avail_mask : 0, + ctx.active[sc] ? ctx.active[sc]->freed_mask : 0); + } uint32_t first = try_avail(&ctx.active[sc]); if (first) return a_ctz_32(first); @@ -293,6 +333,9 @@ static int alloc_slot(int sc, size_t req) g->avail_mask--; queue(&ctx.active[sc], g); + if (sc == 0) { + dbg("%p: avail %x", g, g->avail_mask); + } return 0; } -- 2.39.0 --sR1w1Z2nXlPD36o0 Content-Type: application/x-xz Content-Description: dbg.log.xz Content-Disposition: attachment; filename="dbg.log.xz" Content-Transfer-Encoding: base64 /Td6WFoAAATm1rRGAgAhAQwAAACPmEGc4M2wDNtdADCbCc9pAk7ZY322lIjR9kYTQZqA6dYv p+YBhXL3EsJLVpZg1rPBNrku/9pUWItUVM6nUBMTnyuqhxnXexMkbz3faL2GEybaqU4bnATD GlWWCepzy5X8UP1cBFEEN5dwrAkbr7PaEeanpgrrh6ePpgzvhNfeiijjrbs3lvlp9YyGGWbW 99Kr287GzQgdaXc2aMEQxgpPRYrPBveCXEEl8tjj/83yyZ7A7ueOcRT1v7Y6uCNZXztTGXvL joUf/3T+j81Dm4hlzfSitf0voJwdebizax38/qmU8aL1b0J9DnxN0o9BEmxch04eyf3svnQt Guk1++kOoxOrevgd/h5wwkb9+nxZXBYBGGTBSPxj2dr3mvD/Mq7/cqpYaoyghS4wyhsBAAxy ChfifIl193p1MyPhF0gIZKopMgSfYLVQ9n9ZxBBfwnamyiZJ311KV7MJbxE+8jt7Ka1yr6Be oqSCYPhXTZcOTQJjk0AfwjfLRuoHG7amTWoDD6+ZsIUkbKge4mLAna0HmEcl1qnLtXf4qD/V bCSwWDRX2oSwWCU6ekzrNXDh3LPt02r34nTLdB6xrwG42B/qPDbGScFZ7POoinCMmnNOYA9j xZW9Gl/G83gzJHBFSTlEVbU3kKKHFulPXd3Y3DntL+yhYqJWloqPKkJoAvidm8EjL0kptZZQ ynTjcmOPOqfSBKYrh9gidBNFdTsOSGzHOE+qhl/pNBIMqgIyJ21krp0rMKzyDb9Ub0Jz6c1O DXnftyvfk/fHfq2GMTu721YCaoZM3W1QulzPmIMfTMv/jeVCOTb/NaVROfivE8JAn/eXkFT+ zVe/kre/m/af0ea/aaZTZdZSe0/F9ZpHoPQVOz20NNfwwKVqMRVKGJWQw/T+7u1Jog07vccN uSraYexOxEpDbj0cKXiEp8Gt+BkHNjcE7PDtp7Im1bdbF2Ur4ibvffEFfyV0H44O6V6VpnJJ onhw7ZQLEu9YJNXFd+kdKZeCywZSdkV9/1hRDaACCArYfkPIQzLCaPvpt7vr/fSINge8B0xw jY/eX3diRUPCSVYgZC1DGrj841w/z9n+xpKXqUwJEEKXfXHpM1xV0efL1Qz5fQpEFawx+s3L eSgIcnKN7Fd2QT+rx+AyKQq8Gy/Z/riMxIjDWcWLeSyWFyacmuZhciA2IV9TrzZaFQyqqfpG b1+j+F0DGZqhhU3YdW0SMzcoXk2Jzs3YFr+vocuITTwz6VsPeDbCf3qQuQcMViNRx3Mq1xSV fOT3ONDRPgIvkfJaiBiH015f6se2Y3pw3FUjnt6BwzMplTHmRZdfmjzCAOUzoliOBrcpIPMx bLLOPSm6j3AGgFkEreaESnJYd6jUzvlb4KVdrqe3hlXDnhnzf5tT9Ik+6Ed4rp9kKIlCuXnm nU+ov98z4kuPd++JWGWj0TeWyqZS6D5mDuZ2g20N4Z9OvWxGMoX5pAF97qYzdcDOPn9cTuDv Pqv3Wk9COJBkajL7ZBRS9MyXogB8Iwm6/lkVL6tAMneX/8RILPoshj2oOYRnoHCBnIbwgNgk 7pjcJtb7VwGs+aqXkEZIaxR9bIuQNudzEHXpbMRJHtkI/t4eXzrdtJ877RPCxdZsWEg7GMYi 6UBUcRRQtA0l3hr4HmOwPjOrPj8ssr8jr1RTZ8F9YknU9UT84EILsfGbYUJWzaJeprOwJzJa XF6TuTs9C1QKVd+CAe5HXA8tJPAAeaur31hI3tjKryyZd2c2EXMlq4M/su/2uqO1J7u0TbEo e9vmDtl0eh+P/tsi7XN9T15bVXH+PlzSwBU5BINOLmp6ghcMbmRxFtygPXYhS4nsl7AYodx4 gwuIDwIdwR1lSkomaz9UU2FwbkPrg7wbM70HCy216SZR3jZqMA8S1iA/fMRtof+i1YDJg/kR FyIpGJvG/ilBBSHN78g7KddBVx/PjBbrVtHMRxAXWqcQhvBtlR7wTIZaH5A4XT8ILRDTmnVX LUrbqaxwA32iIxHcLSTfWnWspG7CX2kvYRra1Oj+9jrQ5mjkINOosc4nlxM5N0gvm3OzMYEe kI6S2m6uxvob0eproOmzdRKyrAowIpOBd5uHPHn2i81fIRdu8XchQA1b8rNq7wWnHxzJld2o TRJKbVg4JR7CR0cqX0JjYENbkWk82lPlymXN3vDsDNQv/dv0NhGSk9cICcrm+q8J34cCoDED YJ9RtRFqOW2Ul41BUkwVZweEfwvQLkEwu1tj65Pm15RRE1q/5KUyvYB8t/155tKBa6KAY6vz Zwll/0atZxVqC8yhPIW1xD7gnADeQV9eGeYxZi12PGtTc/k19PxcibojHa2S24KbgTekRoDU g0UZaSrXtOxXjhn7coFwzaheUNPvhj2zJsqCfYg9LmmROAGACvwL1EF66sbzbyczOGNq8x1D vjBC0B/CP2irGk47R+FPSBF38yczoYXTJdr8wJ5t/Q1xukj5m94Dkvgrj46NBz35RIaoV6Qc FsWV+1uVKWEvJn+5KWS+woqGXHU2RLqyvaeDXUNj1J86SiVKIHAPDYvxujGv9our0Oxeu6Uc lURxmgYXVAI5f0iORUnjxxciie3FkoPV5JTV3+2U+7S/m3dDfsKgrV4kI/3krq3e+JToirXP +rsjEsRzhEXx9EnLsa3NQrW9lYnZ68V8rvn1aCdruEfbu6hFGg9m6ACmMCLSX0KTcLvsrfIV ZPmB1i89oI7zj9jOuBE6bXEFd97uLJfhwWTpQSP4yZzitRiMngsD5Y6eRP8zscoYTIm4Zlkq +jeTFRHzsLOKfD+oFKghhPlrD7pA1Mh3+J7XLwJkz5NXyvfXCZusUxidgiVP19zMme4GeNYo gA+r9vKGP9UqQQ03gSQzdnsrs9n0I43ZkA+lDvZwhvsUbup9ob0v7kqbWKm4IHMwspvGCk/r NUh4elaiGso/6s9Ugfvu3lJu8p9RNCi7buu2RkG0O6vNNzhMbQIA1VZZLSFxqw7yB406WgW1 AK4Mrpab5hc1T+HkDunBavTfbqzX15KiJ89gBZpufMIO3RpR05NToW6EcJHxFt+uvvyZvbWC UxqbKIohRTBNHSI0pA3vhXHgsAz2BGKgIVr+Ak3XhmvXBy5+u8L6NZ318psCtcLeX5ctTnVu Hu66mb4Aa/tYzXR6ykB2VRy9XHLW/XZhaOD/19hP5AYrQgWzO803ck0113dmDAWqd7XJiax7 RqgmaZDeBf2VIHI9OVvvbFfKhmJxuavXnBmF58ncvZ0LCdQ1uFK8x1nn1dBZWaZ/O6LB0+ZS CL/YOr1bOlomBjDKnXmnomsKC+3b7TK09qQjQIf3xpuwZapuPhOFkjrqF0ufv/hS+ip8lgib +2XAmMExB3Heo4XWGQa656vB6ZMuZP4a4S9gplDrDwiQ70yjBKk6peOepv9V+tm1fnhHYDlO A8xxUC9sFPmzTdEe9MIlRKjVgFUFi+/SigM0I4r1XrknOx4ncYBIkm0W4ViM7ZAPPoN9k1AK x2Cw2oCEvAKlAvfiIDGoCdjfN8Ri7O6iMJMJsM2VN0QksRokQTiGwenqOW/7qSbyDWKx0GA/ 9bVm+9qmIIR89H5oHjyEtlkxcb7GQg6/aj1SHA3bo9ZhzEj3XcjIpLp0m29Jv+Qy9+kmUcvE KV3XjLVnoDwZBPYze3YL7MQqBzMmqRVngwOcq9K2IG1L53OtjR3VAo7IVPiGlqVQwZtSGVbv 856oTVYubSoKQJzJl1GkDo7MBSvfi8+9zkwhKKlVnqrpz3YRpCoCFLU3X5xR29SzjQH8Ryi1 7Ylfc9eBl5rYBd2JZLGRJ5BoP9n+Qfhtlrd6l+JpxV990EzIzu6n2fIy36FdrWslVFZ/zjaD BX6Zc7K2GYxdPO6kRbNM1ZCcz+ihGWBjMnFMveBrSnOTjy8Mh/Jgu2x7HjHbHXRKEi7cPJbG LIaK8zu++LmcSlCISJTAJ+Itu+kZqhZ9t1PeynWryADvp1AoXELSQQprY5fVAc8JFe4O16Je 581Kvjqy5uFnbQKFBcK38vAi4ntv4igDb4kgN8e2Q36XDyoc10Q5SlE8bYg53v/aO0zjPN9K D1mqq90eKQ+7UO220bZltODOMQ2n2r7uOGc+x0FOBXeEnVCAf969TIYk47LbVRBmtzxzvjLv 1KQ+evzDA2oC+XqUnhxIkxuaG75e0odaQCQIvvFAyZ7ePkhEWcoTd2d6wrAo+ZrxHlbjIliO GbmbApZR5RPX0SP4BRp70BosuKSEB7e4yaBp2AB42ta2CvEnIesfUFo/Z7oYR5eXdeU4akUr 25YePoM1JxDpNVtn1KG1Gn23T1KJvkL4lHP24ujDD7UYt+hvaF5KKwGCuY3P+Z53x0BDIAJP +zDHLc98ym/6pF/mmvtQ8PnRXL3AhX9npQHyAAAAXPpLEBurCyIAAfcZsZsDAKIll9mxxGf7 AgAAAAAEWVo= --sR1w1Z2nXlPD36o0--