From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/11596 Path: news.gmane.org!.POSTED!not-for-mail From: Tobias Koch Newsgroups: gmane.linux.lib.musl.general Subject: Re: pthread_getattr_np doing loads of mremaps on ARM, MIPS under QEMU user-mode Date: Sun, 25 Jun 2017 01:05:52 +0300 Message-ID: <29a099b1-8c83-28d2-c996-80d84dda8e7f@nonterra.com> References: <20170615141927.GN1627@brightrain.aerifal.cx> <95368764-4c3d-8f6e-4030-87872d8913ea@nonterra.com> <20170621004107.GZ1627@brightrain.aerifal.cx> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: blaine.gmane.org Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="------------07EACAEDB9FB96E0B826B1E6" X-Trace: blaine.gmane.org 1498341975 17043 195.159.176.226 (24 Jun 2017 22:06:15 GMT) X-Complaints-To: usenet@blaine.gmane.org NNTP-Posting-Date: Sat, 24 Jun 2017 22:06:15 +0000 (UTC) User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:52.0) Gecko/20100101 Thunderbird/52.2.0 To: musl@lists.openwall.com Original-X-From: musl-return-11609-gllmg-musl=m.gmane.org@lists.openwall.com Sun Jun 25 00:06:11 2017 Return-path: Envelope-to: gllmg-musl@m.gmane.org Original-Received: from mother.openwall.net ([195.42.179.200]) by blaine.gmane.org with smtp (Exim 4.84_2) (envelope-from ) id 1dOtBy-00048Q-8B for gllmg-musl@m.gmane.org; Sun, 25 Jun 2017 00:06:10 +0200 Original-Received: (qmail 5827 invoked by uid 550); 24 Jun 2017 22:06:13 -0000 Mailing-List: contact musl-help@lists.openwall.com; run by ezmlm Precedence: bulk List-Post: List-Help: List-Unsubscribe: List-Subscribe: List-ID: Original-Received: (qmail 5806 invoked from network); 24 Jun 2017 22:06:12 -0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nonterra01.onmicrosoft.com; s=selector1-nonterra-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=RXznuRC0+4YxX5BQeGG6VqUyFj5mjSiCsQKS1e9p2Os=; b=q8vqkUDaRPMzFWcXhnAeD9P56NBIuuia+fBeX8LFMpWXckCTiSLpJBV1XNBOgWZCapUVNG5Vygl3hjuC3sUzgsK3jPRjtjQnq12KTf8gcFWeXPvPQu86B+yiWyJonYPalpyg2d/ktumLNnlgQX6TnJg3dERSY432qP1ZPBXWcA8= Authentication-Results: lists.openwall.com; dkim=none (message not signed) header.d=none;lists.openwall.com; dmarc=none action=none header.from=nonterra.com; In-Reply-To: <20170621004107.GZ1627@brightrain.aerifal.cx> Content-Language: en-US X-Originating-IP: [176.54.184.193] X-ClientProxiedBy: HE1PR07CA0006.eurprd07.prod.outlook.com (2603:10a6:7:67::16) To AM3PR08MB0152.eurprd08.prod.outlook.com (2a01:111:e400:8853::12) X-MS-PublicTrafficType: Email X-MS-Office365-Filtering-Correlation-Id: f5172ebf-c786-4e68-74f4-08d4bb4d31f6 X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(300000503095)(300135400095)(201703131423075)(300000504095)(300135200095)(300000505095)(300135600095);SRVR:AM3PR08MB0152; X-Microsoft-Exchange-Diagnostics: 1;AM3PR08MB0152;3:XtXH/HHrCpPR0D4W1tEQeUnVwQvAnYLJLeqMXBOeDeyajhMvkRk/IUkAZ20pa238G30ios1vsL09q7Yw7tLs5rMSOt5WvGi3Cla1qsxjFlYI2nU8QdPXfPI5BLRAb4cHsTqepVIGimjKltDO2QtlyPFiuh5MA+TdRTw8heSInI/tY+DcS5r+j/1vuSD6XgiQ/c6Y78+euvzqigb4P+Sp+LZCuAzk2/W8GPeLTEBnxpsX6XOa1OIZKTtln67fHAxnbwCOvtBR0/nRHamesCkCXQyFgphvpTKnQEnnRr/TABKYBApFq1C63xUbpoR/ul8niO0Pr/7JdFSy3gpioZdZYprR4ZMgtdlfczfjN3Bcppj28Eq+rSTxHIiGM6jF70woIvURhoxbkBDSnJquUf7WGq5hYjudSo/GI/4Qt0f7MilKKpjRD4x7DCWzCz0oGd67uYIPP2JPrvMTRbzuIyC+Z5ty5BHBBRGYYJxFbuLN23eEBS8dfDjIS9EfBDgbnTFZjrOdNypOE23UjmyoWiMGVVbpS2sjdTRbIJF1dmjZDxi92fKwE+c3rVG/N308hvWbTn2ZdwDvR/3HfwfWpvt+BV6jmqVImuCc/oXtEHpx+yjhQpLd+/9KyTpiiYKQmYB0Tl6yakvG54muwQqkmDDUXw== X-MS-TrafficTypeDiagnostic: AM3PR08MB0152: X-Microsoft-Exchange-Diagnostics: 1;AM3PR08MB0152;25:R6xxiSA8pSeStyxntXtQV/VTH5ORYokTyCAcdL/kL3abWNl5wb/vY5bgqiHC3ApkPH0JCpwg2BD0GnytYPmvIasrSKsLHDbJU1ZCAz8+uz3TJCvu2015DhRHZQiFoNheB1y4Me5Qm4TLc0KDqnNacCXI1uzDETN3ZI4P63ckNTHU+9ATwE2z4RTFvNPnr1vQkdqnyNKIFTd7pTceNiCAE2PUBwozsWdTs/IfSTALPAwatpgCxgWFba2Z6w012xTZOmvluZmsurLKL6EnH8pzP6KKUNTz+cnw6DOZmKegqvLydVAnBjf5g5rYBb7KLyecjTnQNYJDLZIIMCyNgLOCS+I4H3KWHee44K+bQjeYpLai03FWftnZOkfwKv+CGhIWE/RzILyGC5oWvIttY9xdLu5nwADGq5QFuxquyPqVc1/aI2Gocoj+g9mo8fQ0VF79L67+S4mPEObUWLqNX4qahOl5rB46XnUSrtikFMsGU6F3Xa7Az9TYQci8LgPNIpHoAX8vnaoF1Tv3ltLWH59RXUIL9+l3H/wS7iaZ2sEGw713yCZx3uNAqUbQXWp0qF+US5AEh8YRdFldi9pTDmb3qzoe02FwIUbCynPxYC7vV5Z0mTjzMARd7Jg8f9jxH5SKb9Pgn1hU5bwVwPmWQhyLNC23TabGtCfDUA675RAJOPfHoCblIlAgmrXIipsORuMdOFNo1/nIE20UWZp5MnZO7UiMccL34n/cNb46x2nNTTCJLqo2jkTtLrnK9uHG7C5F9qYyzQZwGlb7 JsyTfvc/0VEFWHAirCXGgn1rHvR+NHqfK6PN6032uS11CSo1cb3RLgJ79ud6OLnRp X-Microsoft-Exchange-Diagnostics: 1;AM3PR08MB0152;31:2JLln7u8YTIBa8Sf1BMmKz6JFMh+vQ3qHoT/Z5UURfDsDUkcEb8lFcOBWVHsUb2C91VGa3Vrsi9OYNPv19nF+qdwiKf3ep3l5yqyPmJB/P8llAcWtYPhbaA6p9AAhE/rhTJwX7hQTMObKvyIYeKzKdsyy//MrlsvMLWAcPf5/QYJQyYlB0Lh1VMUnjuIJ3bA8XVdPzH1hQNWLWetH+K2+rgV5BM+sQo2mcrdSjhYbCEGqQ33pDT9sxz4+n2HxCHsWd/eK0oyVwbpfM45dNgEj3Bk4Cv3RWBXp2TDFJa4r8qWJ3F9PGzWHbYpescgSsMNMdoCihnZJRCWIwqHtbRv1ipvYgJNZCkNkWYAgzeIWIZ/ZVXeamgpRdZTEhoSlO7PAAXNkuLmCq9r3AjxkTg0yqJwz85VdwPoNsREahbIeDHA/W+nvxhy8X5Q8Nj/2okmeCOlM9NPZMReZAywRAIBMTtMxbUfUVTlfnH/pnbusDkB+0Mke/298psrmlHN7p9IhILF28LbN0bKuRKBhmHCUnUEf486FEWO0wWph0R+f0jb7xm5bPimcvty8pIrGW52cbwMEdwI/A1HYgGRlx1X/fTF3S0C2rDoPTUq157M32UtxytERMmiBbPnYWnfVzrDpIl9ZJhqVY8P6rqrZIIbN721suGArfBgb9P7zKKn6cmR4CNscyhb1XVp7/paXjkVvNjUoME1QFTiP7sYAuEAag== X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:(236129657087228)(148574349560750)(209349559609743)(50300203121483); X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(5005006)(8121501046)(93006095)(93001095)(3002001)(100000703101)(100105400095)(10201501046)(6041248)(20161123564025)(20161123560025)(2016111802025)(201703131423075)(201703061421075)(20161123555025)(20161123558100)(20161123562025)(6043046)(6072148)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:AM3PR08MB0152;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:AM3PR08MB0152; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;AM3PR08MB0152;4:HvW4nFD1ifGpF13ie7rD7STUUsQXKggzHYDeOVhGpC?= =?us-ascii?Q?YJi0T/jMtIDwIr+CYpfwHQoOr71yqk6oVdkX19ptqfIW3AYu5YIZjzqf1u6n?= =?us-ascii?Q?E8rMQdKOaZnQ/meSchbrr+4p3j9QarznBfK37cZZxhWjN+iHThLQD18wlU/S?= =?us-ascii?Q?th9mweEPOYSUJl3atPqSEJd2rgzCLKY7IB6uu04EpfVOCWFEPmnAm0E1YySj?= =?us-ascii?Q?VKdhtd9Gz7dSVgef3ObmsDwP/xI+hqd8n1EEtOU/O1M9JDiFIjKz/iVvMRre?= =?us-ascii?Q?0EcAC8ytGWutnQEf3weRKuu1R0EK0mq8dRcLQCOAoyC7AqR1pPMlk5m3krK8?= =?us-ascii?Q?xlv9/FHb5o7LP5tOiPZ6qXgYrdtN3qeZ+2PVIAGhE3zjC5dnIA+0XcaitsK2?= =?us-ascii?Q?db1+KKqw18M679Idf51Ef4wUCZJRV+PSS+AM6MCuyg0uqqhg7pvIKVUQydds?= =?us-ascii?Q?ZuW1JolcXVOxNXJ+rqEJyaAX74Luo47SqzKaVaK9hj3E4vw3TifoR53YKP4m?= =?us-ascii?Q?5kohgyD5fMMNDeaUL+Joay4PsDOOKnSY1tP1/+c+gBuWRxX4Pk4+gTvxhgGw?= =?us-ascii?Q?xdI30umURmeBt6oUUOjNeBvJ1/1RiUb9h+0mQUTzqwMLqY4EuG6HpoftQ+aM?= =?us-ascii?Q?BQSwlT X-Forefront-PRVS: 03484C0ABF X-Forefront-Antispam-Report: SFV:NSPM;SFS:(10009020)(4630300001)(6009001)(39400400002)(39450400003)(24454002)(31686004)(42186005)(6116002)(3846002)(53936002)(6486002)(84326002)(4001350100001)(508600001)(66066001)(65956001)(76176999)(50986999)(54356999)(110136004)(54896002)(2906002)(25786009)(65826007)(6512007)(270700001)(229853002)(6506006)(86362001)(93886004)(31696002)(2950100002)(33646002)(6666003)(189998001)(2361001)(53546010)(2351001)(7736002)(5660300001)(81166006)(36756003)(8676002)(6916009);DIR:OUT;SFP:1101;SCL:1;SRVR:AM3PR08MB0152;H:Tobiass-MacBook-Pro.local;FPR:;SPF:None;MLV:sfv;LANG:en; X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;AM3PR08MB0152;23:K9vG4ZDpLKaba8YlfVOXOTxL5uVTCYIMBy8xiCb38?= =?us-ascii?Q?lHMKPrSWTwkVm4DRqov9iuQjjertEFUDGvX+IGGgHfNKq/ljSmUudk/VAmgc?= =?us-ascii?Q?uqWOwTXs+44x2jhbRfFxxJNia9BGroneiqm56tu+MdU9G5oJe/Xwe8Dmwk33?= =?us-ascii?Q?DesNVTXgoSBMmWic5LLJrg066NrsJkt52u3zqD8K9HLfJB0APp1zOMlGosvg?= =?us-ascii?Q?sXlXcWHvrK/JUkP+dKAHDAPQLFc0o+ntbymW4oVCBKcOoCYSj0Hop/7y3yjX?= =?us-ascii?Q?6fmEU93geyLEjWMeu2oOdv+Gw0tXsa1+7dVu4lA4qyNqj1leCyOzhTFcil9m?= =?us-ascii?Q?kJLlCkhv1OTHD0H9LsDgVr5cU5yJ9j3XpVDBsfq7VZ171zIt9EHfh+kTx1fo?= =?us-ascii?Q?DTBEeahIukUwuylIak0k5SP7yRCF0KGenYMv/rIQA1AGdgdCc5tU0sqlpRWq?= =?us-ascii?Q?M73vKTRYjqXSlqm7CO3r76fbl09/kw+4VDM5roszbazhnMrrGweDjSKowsT5?= =?us-ascii?Q?uETVYr3chnXXApg5ACZX/GLbN2otapnmON+tq8FAc4mN8kZqtZ14nHaihUvO?= =?us-ascii?Q?LNsIndS4Vkyd8W45LeDrrdjp2KD/f2L/8nr5frU+jxtroeUJT00VQ7XO5QDy?= =?us-ascii?Q?FQSKcx X-Microsoft-Exchange-Diagnostics: =?us-ascii?Q?1;AM3PR08MB0152;6:D3w+1eETCmSlAwQGLYkycDZauvpnfq+JvgJVONgi0+?= =?us-ascii?Q?27N/6mClMDlOKVlZZ2pZhg1JwtwUXBa3L4aAabMTjJzyVnrkfycOyMoq82jf?= =?us-ascii?Q?TimT8mozi450LEq1lYTDxcK1kY8g6rn/tgZV+CaP5iu/xZh9rqAAkUtcBQk8?= =?us-ascii?Q?3+ihhLc8KVPTiCNSrvaHJMa9FAX8lTK2cy79BNmmt09Z7lMC1dqYw33at9/E?= =?us-ascii?Q?Ztg+NAU5pb3qg4dj7qAZZu4PDJ8R4o6Tez7kjhovzZ62Y1vukcSK5srreLT9?= =?us-ascii?Q?HYqv6t7qHC8Xiku0XvUKLnu2sghjOge3Oiik3deE6nXPP+m1n0l0R+tHZy8f?= =?us-ascii?Q?WzYO09PadtqJPBq2LRTWFNwQJr9gl2bGUy/TO+DuY2XDZ4wM/620UawnFZ/H?= =?us-ascii?Q?5qzFd+lZtiMNYMKUkGAxA60P0b0GDMBFSTYw4Ax1/d0048MTDkTDtaJV+iRl?= =?us-ascii?Q?y/hvBExRCI/cQeiMDKxwZ5H9ou24UfojW7GsnJ08q01GzJfybc9RlqnBNWGb?= =?us-ascii?Q?E4Da6wfMzzjoqOyZxLS9INrArVD/Ih35sEDaySYoLxALiUtWqHHa08NWwyCA?= =?us-ascii?Q?g4NArq36Tl2tZ4XVI2IXcJVP69XdAi9k+LtqmlO/N/md740Co2yk5u4TRAag?= =?us-ascii?Q?qCPCCH X-Microsoft-Exchange-Diagnostics: 1;AM3PR08MB0152;5:lmmE9mIB3KJCRXF4UR0oDRQE/ljKWxjR2malKUvFG5Nag4cseYPNlt3SK3hBCyfMvojAaUyIuuceF3zEgF4ZJ1l9cEz742ClV9i+iU3VHOPYIBVWuPrr0Nx7G0X7kdfHb1ODRHhLn5495fotZCAi2ic6B3rMoHSq4ws2LC8vZd5YhHWVgFl4OiSpVUkUNjU/F3MnIPLmlbu/bV3FjIoCngqb6qUVGyGf+ApL+fSZ83vMN8arDF6dYM5RneKL7b2QbMNG2O0ROYVf9dEZfYwKKBnGePuLiUtBzenCmcSkJmPfCloCaAJtJ81v5afsDcJ2h7vd+ZkElCNod3HCHX3dQJFHuKLYgSWd7cHCM67/y/Q0RDIIk9lV0qsHQfYk+OzbO+vEvcJIjdora66tFSVc2NnUP9i4Amu8RErHqy8cOlg+Jkbx31hOuOXbn6SqVTHS6WoF+JNYGcaIOOkguNUsF7p5T+Y6UmtXo8PGHU/5+pSsdpor0PLlkD0/tJbyodIm;24:NeeK9vBAYEyPikLnNwAbhTDfIis0OQSpYXV4NZGFbtR6OvC1680P5DH3xzbNqcsMc/ox82lw+0QzgQf8uU4VXCC0QyR2GWwJy7ZoAZeXnXc= SpamDiagnosticOutput: 1:99 SpamDiagnosticMetadata: NSPM X-Microsoft-Exchange-Diagnostics: 1;AM3PR08MB0152;7:BDyeNs+t9s9gNBifbEn5MJQe1YqWL/xecv7+ApfecqWDx1JuEIXJYIrVKDrhfJhltzj0L54FpUnsOmxnotc8Fbr5TZck+CJluEurUFTR2ZMn1+4agLCnrpQBRPjPtEe6bypW4g9YoPXZu3GlFFhY9yhTZUc9dH/ZJfMeKbMLAdXG01P4tnYuCbHuNcOC7cWl6Yj+p6IfWFdeUQSo8GAVtq+Ts5xYhMMD81epFMIa6JgK0G9gAA1n1epvmEQu+oIwCoi++PwX8sxOhEQdcgrNqmn6FinSALN7acV+pqPnG9bECVnpFcyomd/x6uYeeolROcZ5ktVHr+XW1U9dfSQrbI/jp55fKkZ3R6y67qdNLigbQqSnzJu9m8MgvYxGa2VrJf9McYT7fcg3dPdua1EYqXu9Hv1EKjsmv30QhFmcpaW1bOi+abUIRZdNOkW7XwByri7hQNbZ8J3riAhadac/BNwWgeLD1V/RkZbbJduqB9T3K2WODdFAl2xH6aRSOFGaaAw5X8fYd44JR+kcw1tHws1IlYexfW4Vgt3gpI5tP6ZJMvhXlNr1lD9Xcz45D6LlwQMdDbdsw6U1XD3wAkBvLfy8dLFXgO3+4yJfBYTSVpfIYBz6cLfAzxCSUDtIzWJ47nmTAd+pAVA+NKYnbW35n0i85F75KnNcgU5YSnTNOENpWJcnxIyM8sG2aENLydM80TJsiFSuGQIuL0dRU6njNim8B+qpY4QsMtumu8BWy+gVJmXZ7XoVqeecC1JqVlqcZoV0iSYWnk9N1 l+mkdqaHtfx+3FOFyM3Xh9ocoUfjm0= X-OriginatorOrg: nonterra.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jun 2017 22:05:58.6193 (UTC) X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted X-MS-Exchange-Transport-CrossTenantHeadersStamped: AM3PR08MB0152 Xref: news.gmane.org gmane.linux.lib.musl.general:11596 Archived-At: This is a multi-part message in MIME format. --------------07EACAEDB9FB96E0B826B1E6 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hi, thank you! And yes, you are absolutely right. QEMU always fails in mmap.c after looking at page flags in the extended memory range: int prot = 0; if (reserved_va && old_size < new_size) { abi_ulong addr; for (addr = old_addr + old_size; addr < old_addr + new_size; addr++) { prot |= page_get_flags(addr); } } if (prot == 0) { host_addr = mremap(g2h(old_addr), old_size, new_size, flags); if (host_addr != MAP_FAILED && reserved_va && old_size > new_size) { mmap_reserve(old_addr + old_size, new_size - old_size); } } else { /* musl's pthread_getattr_np always fails here doing mremaps: */ errno = ENOMEM; host_addr = MAP_FAILED; } I wonder why it cannot just try the remap directly and use the result of that, but that's clearly a question for the QEMU people. Tobias On 21/06/2017 03:41, Rich Felker wrote: > [...] > OK, qemu-user is just emulating mremap very poorly. The man page > documents that it fails with EFAULT when the address passed is not > valid, but qemu seems to be giving ENOMEM, which has a different > meaning. I remember we looked hard when this was first implemented to > find an acceptable/safe way to probe for valid pages, so I doubt we'll > find a better or equally good way that both works around the bug in > qemu-user and does the right thing on all real kernels. I think the > qemu bug should just be reported and fixed. > > Rich --------------07EACAEDB9FB96E0B826B1E6 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 8bit

Hi,

thank you! And yes, you are absolutely right. QEMU always fails in mmap.c after looking at page flags in the extended memory range:

        int prot = 0;
        if (reserved_va && old_size < new_size) {
            abi_ulong addr;
            for (addr = old_addr + old_size;
                 addr < old_addr + new_size;
                 addr++) {
                prot |= page_get_flags(addr);
            }
        }
        if (prot == 0) {
            host_addr = mremap(g2h(old_addr), old_size, new_size, flags);
            if (host_addr != MAP_FAILED && reserved_va && old_size > new_size) {
                mmap_reserve(old_addr + old_size, new_size - old_size);
            }
        } else {
	    /* musl's pthread_getattr_np always fails here doing mremaps: */
 	    errno = ENOMEM;
            host_addr = MAP_FAILED;
        }

I wonder why it cannot just try the remap directly and use the result of that, but that's clearly a question for the QEMU people.

Tobias


On 21/06/2017 03:41, Rich Felker wrote:
[...]
OK, qemu-user is just emulating mremap very poorly. The man page
documents that it fails with EFAULT when the address passed is not
valid, but qemu seems to be giving ENOMEM, which has a different
meaning. I remember we looked hard when this was first implemented to
find an acceptable/safe way to probe for valid pages, so I doubt we'll
find a better or equally good way that both works around the bug in
qemu-user and does the right thing on all real kernels. I think the
qemu bug should just be reported and fixed.

Rich

--------------07EACAEDB9FB96E0B826B1E6--