From mboxrd@z Thu Jan 1 00:00:00 1970 X-Msuck: nntp://news.gmane.org/gmane.linux.lib.musl.general/326 Path: news.gmane.org!not-for-mail From: Rich Felker Newsgroups: gmane.linux.lib.musl.general Subject: Re: FreeSec DES-based crypt(3) Date: Mon, 2 May 2011 15:01:35 -0400 Message-ID: <20110502190135.GL277@brightrain.aerifal.cx> References: <20110502134333.GB18325@openwall.com> Reply-To: musl@lists.openwall.com NNTP-Posting-Host: lo.gmane.org Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii X-Trace: dough.gmane.org 1312595725 11604 80.91.229.12 (6 Aug 2011 01:55:25 GMT) X-Complaints-To: usenet@dough.gmane.org NNTP-Posting-Date: Sat, 6 Aug 2011 01:55:25 +0000 (UTC) To: musl@lists.openwall.com Original-X-From: envelope-from@hidden Mon May 02 19:08:43 2011 Content-Disposition: inline In-Reply-To: <20110502180336.GA20095@openwall.com> User-Agent: Mutt/1.5.21 (2010-09-15) Xref: news.gmane.org gmane.linux.lib.musl.general:326 Archived-At: On Mon, May 02, 2011 at 10:03:36PM +0400, Solar Designer wrote: > On Mon, May 02, 2011 at 09:49:52AM -0400, Rich Felker wrote: > > I could consider using malloc to obtain > > a permanent des state, allocated and initialized on first use, with > > fallback to the stack if malloc fails. > > This makes sense. > > You might be over-estimating the cost of non-const static storage, > though. It will only consume address space, not actual memory, until a > process actually invokes crypt(3) for a DES-based hash, in which case > the cost will be the same as above. Well actually it's a little bit more complicated than that. What happens is that it can fragment the .data segment. For instance if all the .data being written to in most programs was previously in a single page, but the 35k des state got assigned an address between .data symbols which are commonly used, then with the des state, a minimal program would now take an extra dirty page. In the long term, I plan to isolate the modules which have significant amounts of unlikely-to-be-used static state together in the linking order to avoid this type of problem. There still are a couple issues with the malloc approach. One is that it does use additional memory. Using the stack, the memory is already reserved as commit charge (Linux reserves at least 128k commit charge for stack to each process, in my experience) and will be reused by other function calls immediately. Using malloc, the 35k is permanently reserved for DES use only. The other issue, which I don't care about but some people may, is that valgrind will show this as a memory leak. > way, in my opinion. I don't want to discuss another bikeshed, so I'd > rather not go into detail. Don't worry, I don't consider this a bikeshed. You're an expert on crypto and anti-bloat is my specialty and a major defining aspect of musl, so in a way these topics are closer to the nuclear reactor than the bikeshed. > yet), but I just thought that you'd want musl not to be 50x slower than > glibc at this. I would, but I want to balance it with other considerations and achieve all goals at the same time. :) I wonder... is there any way to cut down on the size of this data without affecting (or perhaps even improving) the performance of the code? Then making the data static const would be reasonable. In my experience giant tables suggest 1980s/early-90s style optimization that's often counterproductive today... Rich