-
-
Notifications
You must be signed in to change notification settings - Fork 605
Google Summer of Code 2014
This page contains project ideas for Google Summer of Code 2014! We have tried our best to come up with ideas that cover a broad range of OSv needs, from the kernel itself, to the management system we are building. You are also welcome to visit our issues list in github. They are usually smaller projects and not urgent bugfixes, but it could give you a better idea about where we are heading.
We are also building a new deployment tool centered on OSv called Capstan. It is in very early stage, and therefore we are not listing anything related to it in the ideas list. But if you have a really good and sound idea towards that, we would be happy to take it.
If you're interested in knowing more about OSv, gather details about other potential ideas or make a proposal based on these ideas, please send an e-mail to osv-dev@googlegroups.com. We would be happy to help.
Note: we are listing one mentor per idea in the idea list, but you should view it as a suggestion, not as a guarantee. Specifically, if many ideas here are shared by the same mentor and end up selected, we reserve the right to assign another mentor to overlook your work. This is to make sure the mentor will have all the necessary time to advise you.
Unlike Linux, OSv does not have any kind of fixed structure describing a page. That means some way of figuring out characteristics about the page / allocation is needed.
Currently, free(v) uses the offset of an object within a page (o = v & 0xfff) to determine which allocator is responsible for the page:
- if o == 0, it is a large object (> page size)
- if o == 8, it is a an early allocator object
- otherwise, it is a mempool object, with the pool page header residing at the beginning of the page (v - o)
this scheme has a number of issues:
- allocating objects that are exactly page sized waste two pages
- it is hard to align objects that are between half a page and a page
- the header for a large object is sized an entire page (the expected waste is half a page on average)
We can improve this by using information other than the offset within the page to determine the allocator. If we map physical memory multiple times (all aliases of each other), we can use virtual address bits 44:46 to determine the allocator.
The value of v[44:46] can be used as follows:
0 - mempool allocator 1 - page allocator (for objects between half a page and a page) 2 - half-page allocator (for objects exactly half a page, useful for network buffers) 3 - large objects 4-5 reserved 6 - early allocator objects 7 - debug allocator (using a non identity virtual:physical mapping)
The virtual memory map will now be
0000_0000_0000_0000 - 0000_7fff_ffff_ffff kernel code and application virtual memory
ffff_8000_0000_0000 - ffff_8fff_ffff_ffff mempool (sub-page) objects
ffff_9000_0000_0000 - ffff_9fff_ffff_ffff page objects
ffff_a000_0000_0000 - ffff_afff_ffff_ffff halfpage objects
ffff_b000_0000_0000 - ffff_bfff_ffff_ffff large objects (> 1 page)
ffff_e000_0000_0000 - ffff_efff_ffff_ffff early allocator objects
ffff_f000_0000_0000 - ffff_ffff_ffff_ffff debug allocator objects
More details an follow up discussions at in https://github.com/cloudius-systems/osv/issues/87
Difficulty: Medium
Required skills: Low-level memory management, C++
Mentor: Avi Kivity
There are a number of things we need to do to improve our ability to run runtimes other than the JVM. Part of it consists of enabling other runtimes to be executed inside OSv, part of it consists of improving our ability to run without a runtime at all. Currently, only the JVM and mruby are fully supported.
The student working on this will have to bootstrap the proper runtime and hook it into our module system, so that it becomes as easy to run other runtimes as it is to run java now.
- Go language runtime support
Although we are interested in expanding our runtime list to everything that is relevant, go is one of our top priorities. Our new management module, capstan (https://github.com/cloudius-systems/capstan) is written in go, and we expect that to gain relevance.
OSv is implementing the POSIX API layer on a need-to-do basis. The first task of porting a new runtime is identifying which calls are needed for go that are not implemented by OSv, and then of course, implementing them.
- Running unmodified binaries
Due to initial technical constraints, we do not have the ability to run an ELF binary. We have our own ELF loader and from there, we execute a shared object (.so). While this works, this means native compiled applications that wants to be ran on OSv need to be recompiled as a shared library.
It is technically possible to lift that limitation, by:
- Loading a full elf and reading its list of mappings
- If the kernel mapping overlaps the binary (it probably does), remap the kernel somewhere else, where the application left a hole unfilled
- Audit all the places in which the kernel symbols are used directly: those places will access the old addresses, and will need to be fixed.
Difficulty: Easy to Medium
Required skills: POSIX, C++
Mentor: Glauber Costa
Motivation: Virtualization impose overhead for IO. The main issue is the cost of guest to host exit caused by accessing virtual device registers. SRIOV relieves those exits when a virtual function of a physical device is assigned to a guest. The downside is that the whole guest memory needs to be pinned in the host, standard live migration isn't possible and the guest needs a driver to that hardware. Paravirtual device assignment is a new form of virtualization. In this case the host is responsible for setting up the control channels for RDMA or infiniband while the guest takes case for solely the data path.
The guest driver should be small as possible, concentrating on delivering IO and interrupts. Those interfaces should be well defined, allowing the hypervisor to provide an option rom to the guest and thus relieving the guest from knowing the hardware state. The solution should work with KVM and OSv.
Difficulty: Hard
Required skills: virtio, RDMA or infiniband
Mentor: Avi Kivity
Implement Nahanni guest driver for OSv and present performance that outperform virtio-net. Links: [1] https://gitorious.org/nahanni/pages/Home
[2] http://www.linux-kvm.org/wiki/images/c/cc/2011-forum-nahanni.v5.for.public.pdf
Difficulty: Hard
Required skills: virtio
Mentor: Asias He
Currently, our Xen support is restricted to HVM guests, which means hardware virtualization extensions are needed. However, that is not Xen's main mode of operation. That causes us to lag behind in Xen-centric deployments like Amazon.
There is a very basic pv code in OSv, that go as far as setting up the start-of-day layout and page tables (although the later is not merged)
The student tackling this project will need to implement all the relevant hypercalls that will enable OSv to boon on top of Xen, and also make sure his implementation is efficient
Difficulty: Medium
Required skills: x86 architecture, Xen.
Mentor: Glauber Costa
Cloud Foundry (CF) uses containers in order to run application bundle. Containers are abstracted by CF using the term 'Warden'. The reason for the abstraction is that CF supports more than a single type of containers (lxc on Linux, and there is a flavor for windows too). From the user perspective, the application is packaged as 'build pack' and from there on, the CF infrastructure is responsible to run it within the Warden container. Containers may be executed on top of a physical host or within a virtual machine (running on Softlayer/EC2/VMware). CF knows how to execute new virtual machines and run containers within them.
The project is about running unmodified build packs without any change using a virtual machine powered by OSv. The motivation is the following:
- General hypervisor advantages:
- Virtual machines provide better security than containers. The attack surface of containers is huge.
- Hypervisors provide better flexibility than containers - live migration is supported, better SLA per application, different kernel/OS,..
- Specific OSv advantages:
- Tiny foot print, up to 10MB per image!
- Lower overhead comparing to regular VMs
- Better JVM performance
Overall, OSv offers the best of all worlds. The porting efforts for CF isn't big as it already knows how to run VMs. It's mainly about running a single Warden in a VM for OSv. This is the CF architecture: http://docs.cloudfoundry.com/docs/running/architecture/ The DEA can run within the OSv instance. Instead of running a Warden it will just run the single application within the very same OSv instance. A notion 'stack' helps replacing the VM together with a different DEA: https://groups.google.com/a/cloudfoundry.org/forum/#!searchin/vcap-dev/tzach/vcap-dev/8aYckhjQuh4/4y21LH2tALsJ
Difficulty: Medium
Required skills: High level language code , C++
Mentor Pekka Enberg
ZFS, the default disk-based file system in OSv, supports symbolic links, but the respective functions (zfs_symlink and zfs_readlink) are not integrated into the ZFS vnode operations list (VNOPS).
The first step is to integrate the aforementioned functions into the ZFS VNOPS (all located at '.../zfs/zfs_vnops.c'), and add respective entries, i.e. vnop_symlink_t and vnop_readlink_t, into the VNOPS structure definition which is located at '/include/osv/vnode.h'. This latter one makes the VNOPS list from the Virtual File System (VFS) layer compatible with these new functions.
The final step is to implement the function symlink, and the system calls sys_symlink and sys_readlink (partially implemented), readlink is already implemented (need review though). The task is easier when you take a look at the neighbor functions, and get a basic understanding of how they were implemented.
Another important detail is that these functions must be POSIX compliant. It's really recommended to use manpages as they are very descriptive about the functionality and error codes.
Difficulty: Medium
Required skills: File systems, C/C++.
Mentor: Raphael Carvalho
There are multiple configuration management tools for automatic configuration. They all ways of helping DevOps configuring and controlling large data centers with thousands of machines.
The different configuration management tools differ in their API, how configuration is stored and the configuration DSL, they do all use a local client as a mediator between the central management and the configure host.
The goal of his project is to add a client for one of the configuration tools, they all have their pros and cons so any would do. The challenge arise from been a single process OS, on the other hand, OSv has an API for remote configuration that replaces much of code needed in a client on other OS.
Difficulty: Medium
Required skills: Depends on the client C++/Java/Ruby, ability to deal with a lot of material.
Mentor: Amnon Heiman