From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: To: 9fans@cse.psu.edu Subject: Re: [9fans] plan 9 overcommits memory? From: Charles Forsyth Date: Mon, 3 Sep 2007 22:16:17 +0100 In-Reply-To: <2c68bbb34242a932e4b3e9be554c6f18@quanstro.net> MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Topicbox-Message-UUID: b684d320-ead2-11e9-9d60-3106f5b1d025 > if we allow overcomitted memory, *any* access of brk'd memory might page > fault. this seems like a real step backwards in error recovery as most programs > assume that malloc either returns n bytes of valid memory or fails. since > this assumption is false, either we need to make it true or fix most programs. in the Inferno environment, i concluded that exceptions were the only way of handling that (you could use notes in Plan 9), and that deals with both explicit and implicit allocations. it's more obvious in Inferno because implicit allocations are common, because the run time system allocates memory dynamically, and not just for stack frames. the exception handlers are placed, optionally, fairly high up in the application processes, with further supervising sets towards the root of the system (eg, to encapsulate individual applications within the window system). an unhandled exception within an application process causes that process and perhaps others in its group to die, and the exception is propagated to a process that's the nominated process group leader. note that the process that incurs the exception is just the one that ran out of memory, not the one that `really' caused the problem. there needs to be some extra mechanism to ensure that important functions survive in any case. i looked at quota systems, but they are far too pessimistic for memory systems as they are for disc systems (for most embedded devices, which is where you typically care most about this). typically you end up with either over-committing (which is where we started), or poor utilisation (which also isn't great for small embedded systems). that left some form of allocation accounting, but we found that most programmers for one reason or another found quite hard the systems analysis that's needed to make allocation accounting work (although the degree of pessimism is typically much less than that of quotas, which are too coarse-grained). i used a variant that reserved a given amount of memory for use by distinguished processes critical to system operation or recovery. (perhaps this protected memory structure should have nested, but it seemed better to see if that would really be useful.) systems analysis at this level is much easier, though still neglected.