mcgrof

I'm announcing the release of kdevops which aims at making setting up and testing the Linux kernel for any project as easy as possible. Note that setting up testing for a subsystem and testing a subsystem are two separate operations, however we strive for both. This is not a new test framework, it allows you to use existing frameworks, and set those frameworks up as easily can humanly be possible. It relies on a series of modern hip devops frameworks, it relies on ansible, vagrant and terraform, ansible roles through the Ansible Galaxy, and terraform modules.

Three example demo projects are released which demo it's use:

• kdevops – skeleton generic example using linux-stable
• fw-kdevops – used for testing firmware loading using linux-next. This example demo was written in about one hour tops by forking kdevops, trimming it, adding a new ansible galaxy for selftests. You are expected to be able to fork it and add your respective kernel selftest fork in a minute
• oscheck – actively being used to test and advance the XFS filesystem for stable kernel releases. If you fork this to try to add support for testing a new filesystem under a new project, please let me know how long it took you to do that.

Fancy pictures in a nutshell

Of course you just want pictures and the ability to go home after seeing them. Should these be on instagram as well? Gosh.

A first run of kdevops

On a first run:

Running the bootlinux role on just one host

Example run of just running the ansible bootlinux role on just one host:

End of running the bootlinx ansible role on just one host

This shows what it looks like at the end of running the ansible bootlinux role after the host has booted into the new shiny kernel:

Logging into test test systems

Well, since we set up your ~/ssh/.config for you, all you gotta do now is just ssh in to the target host you want to test, it will already have the shiny new kernel installed and booted into it:

Motivations for kdevops

Below I'll document just a bit of the motivation behind this project. The documentation and demo projects should hopefully suffice for how to use all this.

Testing ain't easy, brah!

Getting contributors to your subsystem / driver in Linux is wonderful, however ensuring it doesn't break anything is a completely separate matter. It is my belief that testing a patch to ensure no testable regressions exist should be painless, and simple, however that has never been the case.

Testing frameworks ain't easy to setup, brah!

Linux kernel testing frameworks should also be really easy to set up. But that is typically never the case either. One example case of complexity in setting a test framework is fstests used to tests Linux kernel filesystems, and to ensure to the best of our ability that a new patch doesn't regress the kernel against a baseline. But wait, what is the baseline?

Setting up test systems ain't easy to ramp up, brah!

Another difficulty with testing the Linux kernel comes with the fact that you don't want to test the kernel on same kernel you're laptop is running on, otherwise you'd crash it, and if you're testing filesystems you may even end up corrupting your filesystem. So typically folks end up using virtualization technologies to setup virtual machines, boot into them, and then use the virtualized hosts as test vehicles. Another alternative is to use cloud service providers such as OpenStack, Azure, Amazon Web Services, Google Cloud Compute to create hosts on the cloud and use these instead. But I've heard complaints about how even setting up KVM can be complex, even from kernel developers! Even some kernel developers don't want to know how to set up a virtual environment to test things.

I hear ya, brah!

My litmus test for a full set up complexity is all the work required to setup fstests to test a Linux filesystem. If a solution for all the woes above were to ever be provided, I figured it'd have to allow to you easily setup fstests to test XFS without you doing much work.

I started looking into this effort first by trying to provide my own set of wrappers around KVM to let you easily setup KVM. Then I extended this effort to easily setup fstests. Both efforts were all shell hacks... It worked for me, but I was still not really happy with it all. It seemed hacky.

Ted Ts'o's xfstests-bld.git provided a cloud environment solution for using setting up fstests on Google Cloud Compute for ext filesystemes (ext2, ext3, ext4), however I was not satisfied with this given I wanted it easy to allow you to test any filesystem, and be Cloud provider agnostic.

ansible provides a proper replacement for shell hacks, in a distribution agnostic manner, and even OS agnostic manner. Vagrant lets me replace all those terrible original bash hacks to setup KVM with simple elegant descriptions of what I want a set of target set of hosts to look like. It also lets me support not only KVM but also Virtualbox, and even support Mac OS X. Terraform accomplishes the same but for cloud environments, and supports different providers.

Feedback and rants welcomed

So, give the repositories a shot, I welcome feedback and rants.

kdevops is intended to be used as the de-facto example for all of the ansible roles, and terraform modules.

fw-kdevops is intended to be forked by folks wanting a simple two host test setup where all you need is linux-next and to run selftests.

oscheck is already actively used to help advance XFS on the stable kernel releases, and is intended to be forked by folks who want to use fstests to test any filesystem on any kernel release.