Re: FreeSec DES-based crypt(3)

[Rich Felker <dalias@aerifal.cx>]

On Mon, May 02, 2011 at 10:03:36PM +0400, Solar Designer wrote:
> On Mon, May 02, 2011 at 09:49:52AM -0400, Rich Felker wrote:
> > I could consider using malloc to obtain
> > a permanent des state, allocated and initialized on first use, with
> > fallback to the stack if malloc fails.
> 
> This makes sense.
> 
> You might be over-estimating the cost of non-const static storage,
> though.  It will only consume address space, not actual memory, until a
> process actually invokes crypt(3) for a DES-based hash, in which case
> the cost will be the same as above.

Well actually it's a little bit more complicated than that. What
happens is that it can fragment the .data segment. For instance if all
the .data being written to in most programs was previously in a single
page, but the 35k des state got assigned an address between .data
symbols which are commonly used, then with the des state, a minimal
program would now take an extra dirty page. In the long term, I plan
to isolate the modules which have significant amounts of
unlikely-to-be-used static state together in the linking order to
avoid this type of problem.

There still are a couple issues with the malloc approach. One is that
it does use additional memory. Using the stack, the memory is already
reserved as commit charge (Linux reserves at least 128k commit charge
for stack to each process, in my experience) and will be reused by
other function calls immediately. Using malloc, the 35k is permanently
reserved for DES use only. The other issue, which I don't care about
but some people may, is that valgrind will show this as a memory leak.

> way, in my opinion.  I don't want to discuss another bikeshed, so I'd
> rather not go into detail.

Don't worry, I don't consider this a bikeshed. You're an expert on
crypto and anti-bloat is my specialty and a major defining aspect of
musl, so in a way these topics are closer to the nuclear reactor than
the bikeshed.

> yet), but I just thought that you'd want musl not to be 50x slower than
> glibc at this.

I would, but I want to balance it with other considerations and
achieve all goals at the same time. :)

I wonder... is there any way to cut down on the size of this data
without affecting (or perhaps even improving) the performance of the
code? Then making the data static const would be reasonable. In my
experience giant tables suggest 1980s/early-90s style optimization
that's often counterproductive today...

Rich