From mboxrd@z Thu Jan 1 00:00:00 1970 Message-ID: Date: Wed, 13 Mar 2013 02:51:02 +0000 From: mycroftiv@sphericalharmony.com To: 9fans@9fans.net MIME-Version: 1.0 Content-Type: text/plain; charset="US-ASCII" Content-Transfer-Encoding: 7bit Subject: Re: [9fans] A note about new software for Plan 9 Topicbox-Message-UUID: 283223ec-ead8-11e9-9d60-3106f5b1d025 Bakul Shah wrote: > The question really is, can we virtualize plan9 machines while > staying pretty much within the plan9 framework or by extending > it in a way consistent with its design? > > To fully virtualize plan9 you'd have to virtualize kernel > services as well. May be you'd've to dream up a ## service > that mediates access to hosts's #services! Thanks again for your comments. From your description of past projects, I see you have professional experience in related domains, so as a home user without a professional background, I'm glad to learn more. I will talk a little bit about what ANTS does right now in relation to full virtualization/jailing. As you point out, there is a big difference between per-process namespaces and fully virtualized kernel devices. The ANTS software makes the choice to focus on what can be done right now with the existing Plan 9 kernel with minimal changes. I think there has been previous discussion in years past of making multiple verions of the kernel #devices available, and also some ideas about letting the Plan 9 kernel run as a userspace process. If you had the ability to synthesize kernel devices on demand and also run multiple kernels as userspace processes, I think that would be pretty close to the full-scale technically sophisticated general approach to Plan 9 jails/pseudo VMs. I think a project to do those things would be interesting and it has been discussed before, but so far as I know, it hasn't been implemented in practice. In relation to that approach, ANTS is very different, and the reason for this is what you talked about in your first reply - what problems ANTS is trying to solve. In my use and testing, I discoved that solving the problems that were important to me didn't require those mechanisms. The problems I was working on turned out to be solvable just by using the existing Plan 9 namespace framework in a new way. ANTS is designed to help administer venti/fossil/cpu grids by creating an independent namespace that doesn't depend on those services. Doing this has the additional benefit that you can easily create independent sub-namespaces that can be used for some of the same things that VMs or jails can. Because I am a home user using Plan 9 on a grid of machines in my basement, the problems that I have been focused on solving are related to making a home grid easy to create and administer, and letting me blend different forms of Plan 9 like Labs and 9front and imported unix resources more freely. I think that in the larger and more professional context, the problems and solutions are different. Profesionally, you are very concerned with making sure that customer A's data doesn't get mixed up with customer B's data so the solution is to provide full isolation of the environments. On my home grid, customer A and customer B are both me, and I want to have access to everything at once. I want an independent namespace that I can use to administer my grid even if I need to reboot some nodes, and I want to use Labs, 9front, p9p, inferno all in a mixed environment. My problem and needed solution is how I create and manage all these different namespaces but also be able to move data between them and have as much sharing as I want, and as much separation as I need. What I have found - and what ANTS is based on - is that even limited to nothing but per-process namespaces and the existing Plan 9 framework, you get a huge amount of functionality with no additional complexity if you work to set up namespace carefully. The way ANTS solves its target problems has some features in common with virtual machines, but ANTS isn't intended as a replacement for VMs. There are problems VMs solve (full isolation/security) which are not the target problems of ANTS. Of course, there are also many features of the namespace tools which aren't conceptually similar to VMs at all, such as automated progressive backup and namespace rewriting, and these solve other problems. :) > While one VM has no access to resources of another VM, the > parent host or the "hypervisor" does need access to every VM > to manage and connect up their resources as needed (for > instance connect A and B's serial devices to each other). > [Ideally this host is also virtualizable] What you have described here is a major difference between ANTS and vms. Resource sharing between environments on the same machine (or on different machines) in ANTS is done mostly through a 9pfs called hubfs, which makes both shell execution and datapipes from other machines available in the file namespace. It connects machines in a way that is a little bit like screen or tmux, but it works directly at the level of the file descriptors (no TTY) and takes the form of a 9p fs to integrate naturally into Plan 9. So when the goal is sharing resources between environments, the extra layer of the VM is often imposes some additional complexity or performance penalty, in comparison to having the resources side by side as processes wtih different namespaces. The question "if you are trying to share everything, why are you creating separate enviornments?" might occur to the reader, but the answer is simple. Different namespace groups and user environments serve different purposes. The main "new namespace" in ANTS is the early boot ram/paq rooted namespace. The reason this namespace exists and has a different structure from the user namespace is that it has a different purpose. The service namespace exists to start and stop services and administer the machine. This is a different purpose than the user namespace, which is about running interactive applications, working with documents, etc. Because the job of administering the machine and connecting to the userspace fileserver is a different job than the job of the userspace processes running in Acme (or another program), it makes sense for their namespace to be structured in a way that matches their purpose. Thanks again for the chance to discuss with someone with a lot of practical experience working with larger groups of machines. Ben Kidwell "mycroftiv" - A minor note. There is a tiny example of a jail in ANTS in the form of the default profile for user none, which unmounts devices and rfork m to prevent new attaches. Sometimes I set up a telnet listener for none using it. In theory if you have a user profile like this and set up an entry into this namespace, you can do lightweight "jails" in the different ANTS namespaces. This isn't really ANTS specific although the fact that ANTS lets you create independent namespaces might make these kind of RFNOMNT jails more useful. I haven't actually played with these things too much because I don't need to put myself in a security jail when I'm the only user on my grid! If someone is interested in working on how ANTS could be used as part of a true jail/security framework I'd certainly be interested in helping.