CONTRIBUTING.md

Nautilus Contribution Guidelines

We are happy to have contributors to our project. If you'd like to contribute a feature, bug fix, or other component to Nautilus, we ask that you first read these guidelines.

Table of Contents

• How Can I Contribute?
• Development Model
• Testing

How Can I Contribute?

Reporting Bugs

You can report bugs in the issue tracker for the Github page. Make sure to label the issue appropriately, and provide a verbose description which outlines in detail everything necessary for us (or others) to reproduce the bug. This includes:

1. Your development environment (which version of gcc, what host are you compiling on?
2. How did you compile Nautilus? You can even attach your .config file (generated by running make menuconfig so we can know which exact configuration you're using.
3. Your runtime environment: if running with qemu, which flags did you provide. If you're running on bare hardware, what are the hardware specs? (i.e. CPU architecture, vendor, how much RAM, which BIOS etc.)
4. What features did you add or what changes did you make? Is this code you've added? Did you tweak the Makefile?
5. Is the bug deterministic? If not, under what conditions does it seem most likely to arise?

Creating an Issue

We've set up several labels to organize issues into logical groups. For example, if it is a bug that you're filing, label your issue with the bug label. Use every appropriate label from the list. We've created a few custom labels of interest:

• bug: Something is broken.
• beginner: This is an issue or feature that a newcomer would likely be able to work on.
• compilation: This is an issue specific to compilation and the build process.
• documentation: This is an issue that only relates to commenting or documentation.
• hardware only: This issue only affects bare hardware, not virtual environments.
• virtual only: This issue only relates to virtual environments, e.g. QEMU or KVM.
• hardware + virtual: The issue is relevant to both physical and virtual hardware.
• runtime: The issue relates to a particular runtime system, not Nautilus itself.
• enhancement: Used for feature requests (wish list items).
• question: General questions and inquiries.

Your First Code Contribution

Follow the process below, and submit a pull request.

Development Model

Getting Started

When developing with Nautilus, it's best to get a good dev environment set up first, e.g. using a personal Linux box or VM with QEMU installed, or a physical machine which you can control via serial port or BMC (e.g. with IPMI + PXE boot or with PXE boot and a serial cable).

We've provided a default Vagrantfile to use with Vagrant for rapid development. See more in the README.

Contributing Code

You'll first want to make sure you have a Github account. Then, head over to the Nautilus github page and use the fork feature. Now, on your dev machine, you can clone like so:

git clone git@github.com:USERNAME/nautilus.git

Keeping Your Fork Synchronized

Unless you're planning on submitting a quick fix, you'll want to make sure that your fork is up to date with our master branch. You can do this with upstream tracking:

# Add upstream to the list of remotes
git remote add upstream https://github.com/HExSA-Lab/nautilus.git

# Verify that the new remote is added
git remote -v

Now, whenever you want to make sure your fork is up to date with the latest upstream changes, you'll first fetch all the upstream branches:

# Fetch from upstream
git fetch upstream

# Look at all branches, including those from upstream
git branch -va

Now, checkout your own master branch and merge it with the upstream's master branch:

git checkout master
git merge upstream/master

If there are no unique commits on your local master branch, Git will just do a fast-forward here. If you have been making changes on master (you probably shouldn't be, see below), you might have to fix some merge conflicts when you do this. When doing so, be careful to respect the upstream changes.

At this point, you should be in sync with everything on the upstream.

Adding Your Own Features

When you start working on a new feature, enhancement, or bugfix, it is important that you start doing so in a new branch. This adheres to the standard Git workflow and makes it easy to logically separate different components/tasks you're working on and makes them easy to submit to the mainline codebase (using pull requests) when you finish and have them in an acceptable state. For example, if I'm implementing a new device driver for a newfangled Footrix NIC, I would do as follows from a fresh fork:

# We want our changes to be based on master
git checkout master

# Come up with a meaningful name for our new branch, related to the work we're doing
git branch footrix-nic

# Switch to the new branch
git checkout footrix-nic

Now have fun building!

Getting Ready for a Pull Request

To submit work to the main codebase, we use a pull request. The name is somewhat misleading, as it's not at all related to a git pull. This is essentially a request to have your code merged in. Before you do so, however, it's a good idea to do some cleanup that will make it easier for the maintainers to merge in your code. The most important thing is to integrate any changes that have been made into the upstream since you started your work. This will avoid annoying conflicts and make the merging process essentially the same as a fast-forward. To do this, we rebase on top of the mainline branch:

# Fetch from the upstream mainline branch, and merge with *your* repo's master
git fetch upstream
git checkout master
git merge upstream/master

# If we got new commits from the upstream, we'll now rebase on top of them
# by taking our commits whole sale and placing them back on top of our master branch.
# (again, assuming the same branches as above)
git checkout footrix-nic
git rebase -i master

If you've made a lot of very small commits, it might make sense to collapse them into one larger commit with everything logically related. In Git, we call this "squashing." We can do this during the rebase using the interactive (-i) mode:

# Rebase all commits made on our footrix-nic branch
git checkout
git rebase -i master

This will open up in your default text editor from which you can pick commits to squash together.

Submitting the Pull Request

Now that you've finished your work, you can submit a pull request. First, you should make sure to commit all your changes (as above) and then push them to your Github fork.

# push to my dev branch
git push origin footrix-nic

Now, go to the page for your Github fork, and select the appropriate branch. You can then click the "pull request" button. If you need to make changes after submitting the pull request, don't worry; the pull request will track new changes as you push them to the branch for which you created the request.

Testing

As you add components to Nautilus, it is a good idea to test them. We've now made this a bit easier with a testing framework. With this framework, you can add test code within whatever file you're editing and automatically have it invoked either from a qemu invocation or the automated build process when your code is merged in to the master branch.

Adding a Test

The first step is to add a function to whatever file you're working on which tests the relevant pieces of code. You can see an example called sample_test() near the bottom of src/test/test.c. All test functions must return an int and accept two arguments, int argc and char * argv[] (just like main() in a regular C program). Here is an example:

static int
my_test (int argc, char * argv[])
{
    int ret = do_something();
    if (ret != 0) {
        ERROR_PRINT("My unit test failed\n");
        return -1;
    }
    
    printk("My unit test succeeded\n");
    return 0;
}

This is a very simple test which calls the function do_something() and reports the return value. Note that any test which does not return 0 will be reported as a failure by the test framework. Each test is passed arguments in the traditional way, much like the C runtime. You can parse these manually or include the <nautilus/getopt.h> header and use the getopt() function to parse arguments for your test.

Once you have this function in place, you then need to register it with the testing framework (again, see src/test/test.c for examples). This is done as follows:

static struct nk_test_impl my_test_impl = {
    .name = "mytest",
    .handler = my_test,
    .default_args = "foo bar baz",
};
nk_register_test(my_test_impl);

This tells Nautilus that it should make this test available. name is the name of the test. We'll see how this is used in a bit. The important thing now is that you pass the test function you wrote before as the handler argument.

Invoking your test from the kernel command-line

The kernel command-line is passed to Nautilus from GRUB (the bootloader) using a configuration file. The default configuration file is found in configs/grub.cfg. This config file is used to generate the isoimage which you boot with QEMU. If you want to invoke your test manually, you can modify configs/grub.cfg.

Nautilus understands a special command-line flag (-test) used to invoke a test. The format is as follows:

-test <testname> "[arg1] [arg2] [arg...]"

Where <testname> corresponds to the name you gave in your test registration. You can even specify multiple invocations of the same test (e.g. with different arguments). For example, for the test we wrote above, I might invoke it like so:

-test mytest "foo bar baz" -test mytest "1 2 3"

This will cause our test to be invoked twice, first with the arguments foo bar baz and then with 1 2 3. To tie it together, we can invoke this test by modifying the default grub.cfg. For this example, we would want to fill it with, e.g. the following contents:

set timeout=0
set default=0
menuentry "Nautilus" {
    multiboot2 /boot/nautilus.bin -test mytest "foo bar baz" -test mytest "1 2 3"
    module2 /boot/nautilus.syms
    boot
}

This will tell GRUB to pass these command-line parameters along to the kernel.

Configuring the kernel for testing

By default, Nautilus will not run any tests at all. To get it to run tests on bootup, you must configure the kernel (with make menuconfig) with the option Configuration -> Run all tests from the testing framework at boot enabled. Once you've done this, you can now build the kernel (using make isoimage). In order to actually run your tests, you'll want to use QEMU. By default, however, the kernel will just hang after (and if) it passes all enabled tests. If you want it to shutdown after it has passed all tests (for example, if you're scripting your tests), you'll want to add the flag -device isa-debug-exit to your QEMU invocation. For example:

$> qemu-system-x86_64 -cdrom nautilus.iso -m 1G -serial stdio -monitor /dev/null -nographic -device isa-debug-exit

Special tests

As you might have guessed before, your test will only be run if you invoke it explicitly. If you'd like to run every test which has been registered, there is a special flag for that as well, called -test-all which will invoke all tests. Because there is no simple way to pass arguments to all tests which will be run in this fashion, you can specify a set of default arguments when writing your test which will be used when the -test-all flag is given. In the example above, I've provided .default_args = "foo bar baz", so that when I use the following grub configuration:

set timeout=0
set default=0
menuentry "Nautilus" {
    multiboot2 /boot/nautilus.bin -test-all
    module2 /boot/nautilus.syms
    boot
}

My test will be invoked with the arguments foo bar baz, and every other test will also be invoked using its default arguments.

Integration into the Build

The test framework is integrated with the build system, so that when new commits are made to the master branch or when pull requests are merged into that branch, a set of preselected tests will be run to make sure everything is still working. This integration is done with the Travis CI framework. To add a test that you've written to this set of tests, you must add it to the test matrix, which is specified in the file .test-matrix.yml. The Nautilus build system will parse this file, and for each test specification that it encounters it will generate a new GRUB configuration with the appropriate command-line arguments and launch a script to run the test (usually via a QEMU invocation of some sort).

Adding to the test matrix

The test matrix is specified using the YAML format. Here is an example of how we would add the test we created above to the matrix:

- mytest:
    configs:
        - default 
        - full-debug
    prep: echo "Hello World"
    run: qemu-system-x86_64 -cdrom nautilus.iso -m 1G -serial stdio -device isa-debug-exit -monitor /dev/null -nographic
    test_flags:
        - "-test mytest \"foo bar baz\""
        - "-test mytest \"1 2 3\""
        
- some-other-test:
    configs:
        - custom
        - custom2
    prep: dd if=/dev/zero of=hdd.img count=1024 bs=1024
    run: qemu-system-x86_64 -cdrom nautilus.iso -m 1G -hda hdd.img -device isa-debug-exit
    test_flags:
        - "-test some-other-test \"a b c\"

Each test is a list element, and comprises a one-element YAML dictionary, where the key is the test name and the value is another dictionary necessarily containing the following items:

• configs: a list of kernel configurations this test should be run with. There are two base configurations already provided, default and full-debug. These are different instances of .config files generated by Kconfig. default corresponds to the file configs/default.config. If you want another configuration, add it to the list in the test matrix and add the .config file for it (with a reasonable name) to the configs/ directory.
• prep: if you need to run any commands (external to Nautilus) before running your test, you can specify those here. The second test above uses this to generate a virtual disk image to use with QEMU.
• run: this is the command to run the test. It will almost always be an invocation of QEMU. Again, remember that you need the -device isa-debug-exit flag for QEMU to shutdown properly after the test is complete.
• test_flags: this is a list of specific tests which will be run on this test invocation. Note that the first example above corresponds directly to our manual configuration using grub.cfg before.

Make sure to commit your .config files and your additions to .test-matrix.yml. Then, if your PR gets merged in or your commits are pushed to the master branch, your tests will be run automatically. If any test fails, the automatic build will fail, indicating that something went wrong. This can be diagnosed using the Travis CI web interface.