denshooter/genode
1
0
Fork 0
mirror of https://github.com/mmueller41/genode.git synced 2026-01-21 12:32:56 +01:00
Code Issues Packages Projects Releases Wiki Activity

Imported Genode release 11.11

Browse Source
This commit is contained in:
Genode Labs
2011-12-22 16:19:25 +01:00
committed by Christian Helmuth
parent 6bcc9aef0e
commit da4e1feaa5
2462 changed files with 320115 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
doc/Makefile Normal file
View File

@@ -0,0 +1,38 @@
SHELL = bash
all: directories.txt
#
# The order of directories in the documentation can be expressed as dependencies.
#
../demo: ../base
directories.txt: $(shell find .. -name README) Makefile
@echo "creating $@"
@echo "Directory structure of the Genode source tree" > $@
@echo "#############################################" >> $@
@echo >> $@
@$(MAKE) --no-print-directory .. >> $@
DIRECTORIES = $(shell find .. -type d)
.PHONY: $(DIRECTORIES)
$(DIRECTORIES):
@if test -r $@/README; then \
dir=$@; dir="$${dir:3}"; \
if test -n "$$dir"; then \
dir="'$$dir'"; \
echo "$$dir"; \
echo "$${dir//?/=}"; \
echo; \
fi; \
cat $@/README; \
echo; \
echo; \
fi; \
recursion="$(shell find $@/* -mindepth 1 -name README -printf '%H\n')"; \
if test "$$recursion"; then $(MAKE) --no-print-directory -s $$recursion; fi
clean cleanall:
rm -f directories.txt

466
doc/build_system.txt Normal file
View File

@@ -0,0 +1,466 @@
=======================
The Genode build system
=======================
Norman Feske
Abstract
########
The Genode OS Framework comes with a custom build system that is designed for
the creation of highly modular and portable systems software. Understanding
its basic concepts is pivotal for using the full potential of the framework.
This document introduces those concepts and the best practises of putting them
to good use. Beside building software components from source code, common
and repetitive development tasks are the testing of individual components
and the integration of those components into complex system scenarios. To
streamline such tasks, the build system is accompanied with special tooling
support. This document introduces those tools.
Build directories and repositories
##################################
The build system is supposed to never touch the source tree. The procedure of
building components and integrating them into system scenarios is done at
a distinct build directory. One build directory targets a specific platform,
i.e., a kernel and hardware architecture. Because the source tree is decoupled
from the build directory, one source tree can have many different build
directories associated, each targeted at another platform.
The recommended way for creating a build directory is the use of the
'create_builddir' tool located at '<genode-dir>/tool/builddir/'. By starting
the tool without arguments, its usage information will be printed. For creating
a new build directory, one of the listed target platforms must be specified.
Furthermore, the location of the new build directory has to be specified via
the 'BUILD_DIR=' argument. For example:
! cd <genode-dir>
! ./tool/create_builddir linux_x86 BUILD_DIR=/tmp/build.linux_x86
This command will create a new build directory for the Linux/x86 platform
at '<genode-dir>/build.linux_x86/'.
Build-directory configuration via 'build.conf'
==============================================
The fresh build directory will contain a 'Makefile', which is a symlink to
'tool/builddir/build.mk'. This 'Makefile' is the front end of the build system
and not supposed to be edited. Beside the 'Makefile', there is a 'etc/'
subdirectory that contains the build-directory configuration. For most
platforms, there is only a single 'build.conf' file, which defines the parts of
the Genode source tree incorporated in the build process. Those parts are
called _repositories_.
The repository concept allows for keeping the source code well separated for
different concerns. For example, the platform-specific code for each target
platform is located in a dedicated 'base-<platform>' repository. Also, different
abstraction levels and features of the system are residing in different
repositories. The 'etc/build.conf' file defines the set of repositories to
consider in the build process. At build time, the build system overlays the
directory structures of all repositories specified via the 'REPOSITORIES'
declaration to form a single logical source tree. By changing the list of
'REPOSITORIES', the view of the build system on the source tree can be altered.
The 'etc/build.conf' as found in a fresh created build directory will list the
'base-<platform>' repository of the platform selected at the 'create_builddir'
command line as well as the 'base', 'os', and 'demo' repositories needed for
compiling Genode's default demonstration scenario. Furthermore, there are a
number of commented-out lines that can be uncommented for enabling additional
repositories.
Note that the order of the repositories listed in the 'REPOSITORIES' declaration
is important. Front-most repositories shadow subsequent repositories. This
makes the repository mechanism a powerful tool for tweaking existing repositories:
By adding a custom repository in front of another one, customized versions of
single files (e.g., header files or target description files) can be supplied to
the build system without changing the original repository.
Building targets
================
To build all targets contained in the list of 'REPOSITORIES' as defined in
'etc/build.conf', simply issue 'make'. This way, all components that are
compatible with the build directory's base platform will be built. In practice,
however, only some of those components may be of interest. Hence, the build
can be tailored to those components which are of actual interest by specifying
source-code subtrees. For example, using the following command
! make core server/nitpicker
the build system builds all targets found in the 'core' and 'server/nitpicker'
source directories. You may specify any number of subtrees to the build
system. As indicated by the build output, the build system revisits
each library that is used by each target found in the specified subtrees.
This is very handy for developing libraries because instead of re-building
your library and then your library-using program, you just build your program
and that's it. This concept even works recursively, which means that libraries
may depend on other libraries.
In practice, you won't ever need to build the _whole tree_ but only the
targets that you are interested in.
Cleaning the build directory
============================
To remove all but kernel-related generated files, use
! make clean
To remove all generated files, use
! make cleanall
Both 'clean' and 'cleanall' won't remove any files from the 'bin/'
subdirectory. This makes the 'bin/' a safe place for files that are
unrelated to the build process, yet required for the integration stage, e.g.,
binary data.
Controlling the verbosity of the build process
==============================================
To understand the inner workings of the build process in more detail, you can
tell the build system to display each directory change by specifying
! make VERBOSE_DIR=
If you are interested in the arguments that are passed to each invocation of
'make', you can make them visible via
! make VERBOSE_MK=
Furthermore, you can observe each single shell-command invocation by specifying
! make VERBOSE=
Of course, you can combine these verboseness toggles for maximizing the noise.
Enabling parallel builds
========================
To utilize multiple CPU codes during the build process, you may invoke 'make'
with the '-j' argument. If manually specifying this argument becomes an
inconvenience, you may add the following line to your 'etc/build.conf' file:
! MAKE += -j<N>
This way, the build system will always use '<N>' CPUs for building.
Caching inter-library dependencies
==================================
The build system allows to repeat the last build without performing any
library-dependency checks by using:
! make again
The use of this feature can significantly improve the work flow during
development because in contrast to source-codes, library dependencies rarely
change. So the time needed for re-creating inter-library dependencies at each
build can be saved.
Repository directory layout
###########################
Each Genode repository has the following layout:
Directory | Description
------------------------------------------------------------
'doc/' | Documentation, specific for the repository
------------------------------------------------------------
'etc/' | Default configuration of the build process
------------------------------------------------------------
'mk/' | The build system
------------------------------------------------------------
'include/' | Globally visible header files
------------------------------------------------------------
'src/' | Source codes and target build descriptions
------------------------------------------------------------
'lib/mk/' | Library build descriptions
For each custom source-code repository supplied to the build system, the
following subdirectories are mandatory:
! lib/mk/
! src/
! include/
Creating targets and libraries
##############################
Target descriptions
===================
A good starting point is to look at the init target. The source code of init is
located at 'os/src/init/'. In this directory, you will find a target description
file named 'target.mk'. This file contains the building instructions and it is
usually is very simple. The build process is controlled by defining the
following variables.
Build variables to be defined by you
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
:'TARGET': is the name of the binary to be created. This is the
only *mandatory variable* to be defined in a 'target.mk' file.
:'REQUIRES': expresses the requirements that must be satisfied in order to
build the target. You find more details about the underlying mechanism in
Section [Specializations].
:'LIBS': is the list of libraries that are used by the target.
:'SRC_CC': contains the list of '.cc' source files. The default search location
for source codes is the directory, where the 'target.mk' file resides.
:'SRC_C': contains the list of '.c' source files.
:'SRC_S': contains the list of assembly '.s' source files.
:'SRC_BIN': contains binary data files to be linked to the target.
:'INC_DIR': is the list of include search locations. Directories should
always be appended by using +=. Never use an assignment!
:'EXT_OBJECTS': is a list of Genode-external objects or libraries. This
variable is mostly used for interfacing Genode with legacy software
components.
Rarely used variables
---------------------
:'CC_OPT': contains additional compiler options to be used for '.c' as
well as for '.cc' files.
:'CC_CXX_OPT': contains additional compiler options to be used for the
C++ compiler only.
:'CC_C_OPT': contains additional compiler options to be used for the
C compiler only.
Specifying search locations
~~~~~~~~~~~~~~~~~~~~~~~~~~~
When specifying search locations for header files via the 'INC_DIR' variable or
for source files via 'vpath', relative pathnames are illegal to use. Instead,
you can use the following variables to reference locations within the
source-code repository, where your target lives:
:'REP_DIR': is the base directory of the current source-code repository.
Normally, specifying locations relative to the base of the repository is
never used by 'target.mk' files but needed by library descriptions.
:'PRG_DIR': is the directory, where your 'target.mk' file resides. This
variable is always to be used when specifying a relative path.
Library descriptions
====================
In contrast to target descriptions that are scattered across the whole source
tree, library descriptions are located at the central place 'lib/mk'. Each
library corresponds to a '<libname>.mk' file. The base of the description file
is the name of the library. Therefore, no 'TARGET' variable needs to be set.
The source-code locations are expressed as '$(REP_DIR)'-relative 'vpath'
commands.
Library-description files support the following additional declarations:
:'SHARED_LIB = yes': declares that the library should be built as a shared
object rather than a static library. The resulting object will be called
'<libname>.lib.so'.
Specializations
===============
Building components for different platforms likely implicates portions of code
that are tied to certain aspects of the target platform. For example, a target
platform may be characterized by
* A kernel API such as L4v2, Linux, L4.sec,
* A hardware architecture such as x86, ARM, Coldfire,
* A certain hardware facility such as a custom device, or
* Other properties such as software license requirements.
Each of these attributes express a specialization of the build process. The
build system provides a generic mechanism to handle such specializations.
The _programmer_ of a software component knows the properties on which his
software relies and thus, specifies these requirements in his build description
file.
The _user/customer/builder_ decides to build software for a specific platform
and defines the platform specifics via the 'SPECS' variable per build
directory in 'etc/specs.conf'. In addition to an (optional) 'etc/specs.conf'
file within the build directory, the build system incorporates the first
'etc/specs.conf' file found in the repositories as configured for the
build directory. For example, for a 'linux_x86' build directory, the
'base-linux/etc/specs.conf' file is used by default. The build directory's
'specs.conf' file can still be used to extend the 'SPECS' declarations, for
example to enable special features.
Each '<specname>' in the 'SPECS' variable instructs the build system to
* Include the 'make'-rules of a corresponding 'base/mk/spec-<specname>.mk'
file. This enables the customization of the build process for each platform.
* Search for '<libname>.mk' files in the 'lib/mk/<specname>/' subdirectory.
This way, we can provide alternative implementations of one and the same
library interface for different platforms.
Before a target or library gets built, the build system checks if the 'REQUIRES'
entries of the build description file are satisfied by entries of the 'SPECS'
variable. The compilation is executed only if each entry in the 'REQUIRES'
variable is present in the 'SPECS' variable as supplied by the build directory
configuration.
Automated integration and testing
#################################
Genode's cross-kernel portability is one of the prime features of the
framework. However, each kernel takes a different route when it comes to
configuring, integrating, and booting the system. Hence, for using a particular
kernel, profound knowledge about the boot concept and the kernel-specific tools
is required. To streamline the testing of Genode-based systems across the many
different supported kernels, the framework comes equipped with tools that
relieve you from these peculiarities.
Run scripts
===========
Using so-called run scripts, complete Genode systems can be described in a
concise and kernel-independent way. Once created, a run script can be used
to integrate and test-drive a system scenario directly from the build directory.
The best way to get acquainted with the concept is reviewing the run script
for the 'hello_tutorial' located at 'hello_tutorial/run/hello.run'.
Let's revisit each step expressed in the 'hello.run' script:
* Building the components needed for the system using the 'build' command.
This command instructs the build system to compile the targets listed in
the brace block. It has the same effect as manually invoking 'make' with
the specified argument from within the build directory.
* Creating a new boot directory using the 'create_boot_directory' command.
The integration of the scenario is performed in a dedicated directory at
'<build-dir>/var/run/<run-script-name>/'. When the run script is finished,
this directory will contain all components of the final system. In the
following, we will refer to this directory as run directory.
* Installing the Genode 'config' file into the run directory using the
'install_config' command. The argument to this command will be written
to a file called 'config' at the run directory picked up by
Genode's init process.
* Creating a bootable system image using the 'build_boot_image' command.
This command copies the specified list of files from the '<build-dir>/bin/'
directory to the run directory and executes the platform-specific steps
needed to transform the content of the run directory into a bootable
form. This form depends on the actual base platform and may be an ISO
image or a bootable ELF image.
* Executing the system image using the 'run_genode_until' command. Depending
on the base platform, the system image will be executed using an emulator.
For most platforms, Qemu is the tool of choice used by default. On Linux,
the scenario is executed by starting 'core' directly from the run
directory. The 'run_genode_until' command takes a regular expression
as argument. If the log output of the scenario matches the specified
pattern, the 'run_genode_until' command returns. If specifying 'forever'
as argument (as done in 'hello.run'), this command will never return.
If a regular expression is specified, an additional argument determines
a timeout in seconds. If the regular expression does not match until
the timeout is reached, the run script will abort.
Please note that the 'hello.run' script does not contain kernel-specific
information. Therefore it can be executed from the build directory of any base
platform by using:
! make run/hello
When invoking 'make' with an argument of the form 'run/*', the build system
will look in all repositories for a run script with the specified name. The run
script must be located in one of the repositories 'run/' subdirectories and
have the file extension '.run'.
For a more comprehensive run script, 'os/run/demo.run' serves as a good
example. This run script describes Genode's default demo scenario. As seen in
'demo.run', parts of init's configuration can be made dependent on the
platform's properties expressed as spec values. For example, the PCI driver
gets included in init's configuration only on platforms with a PCI bus. For
appending conditional snippets to the 'config' file, there exists the 'append_if'
command, which takes a condition as first and the snippet as second argument.
To test for a SPEC value, the command '[have_spec <spec-value>]' is used as
condition. Analogously to how 'append_if' appends strings, there exists
'lappend_if' to append list items. The latter command is used to conditionally
include binaries to the list of boot modules passed to the 'build_boot_image'
command.
The run mechanism explained
===========================
Under the hood, run scripts are executed by an expect interpreter. When the
user invokes a run script via 'make run/<run-script>', the build system invokes
the run tool at '<genode-dir>/tool/run' with the run script as argument. The
run tool is an expect script that has no other purpose than defining several
commands used by run scripts, including a platform-specific script snippet
called run environment ('env'), and finally including the actual run script.
Whereas 'tool/run' provides the implementations of generic and largely
platform-independent commands, the 'env' snippet included from the platform's
respective 'base-<platform>/run/env' file contains all platform-specific
commands. For reference, the most simplistic run environment is the one at
'base-linux/run/env', which implements the 'create_boot_directory',
'install_config', 'build_boot_image', and 'run_genode_until' commands for Linux
as base platform. For the other platforms, the run environments are far more
elaborative and document precisely how the integration and boot concept works
on each platform. Hence, the 'base-<platform>/run/env' files are not only
necessary parts of Genode's tooling support but serve as resource for
peculiarities of using each kernel.
Using run script to implement test cases
========================================
Because run scripts are actually expect scripts, the whole arsenal of
language features of the Tcl scripting language is available to them. This
turns run scripts into powerful tools for the automated execution of test
cases. A good example is the run script at 'libports/run/lwip.run', which tests
the lwIP stack by running a simple Genode-based HTTP server on Qemu. It fetches
and validates a HTML page from this server. The run script makes use of a
regular expression as argument to the 'run_genode_until' command to detect the
state when the web server becomes ready, subsequently executes the 'lynx' shell
command to fetch the web site, and employs Tcl's support for regular
expressions to validate the result. The run script works across base platforms
that use Qemu as execution environment.
To get the most out of the run mechanism, a basic understanding of the Tcl
scripting language is required. Furthermore the functions provided by
'tool/run' and 'base-<platform>/run/env' should be studied.
Automated testing across base platforms
=======================================
To execute one or multiple test cases on more than one base platform, there
exists a dedicated tool at 'tool/autopilot'. Its primary purpose is the
nightly execution of test cases. The tool takes a list of platforms and of
run scripts as arguments and executes each run script on each platform. The
build directory for each platform is created at
'/tmp/autopilot.<username>/<platform>' and the output of each run script is
written to a file called '<platform>.<run-script>.log'. On stderr, autopilot
prints the statistics about whether or not each run script executed
successfully on each platform. If at least one run script failed, autopilot
returns a non-zero exit code, which makes it straight forward to include
autopilot into an automated build-and-test environment.

267
doc/coding_style.txt Normal file
View File

@@ -0,0 +1,267 @@
Coding style guidelines for Genode
##################################
Things to avoid
===============
Please avoid using pre-processor macros. C++ provides language
features for almost any case, for which a C programmer uses
macros.
:Defining constants:
Use 'enum' instead of '#define'
! enum { MAX_COLORS = 3 };
! enum {
! COLOR_RED = 1,
! COLOR_BLUE = 2,
! COLOR_GREEN = 3
! };
:Meta-programming:
Use templates instead of pre-processor macros.
In contrast to macros, templates are type-safe
and fit well with the implementation syntax.
:Conditional-code inclusion:
Please avoid C-hacker style '#ifdef CONFIG_PLATFROM' - '#endif'
constructs but instead, factor-out the encapsulated code into a
separate file and introduce a proper function interface.
The build process should then be used to select the appropriate
platform-specific files at compile time. Keep platform dependent
code as small as possible. Never pollute existing generic code
with platform-specific code.
Header of each file
===================
! /*
! * \brief Short description of the file
! * \author Original author
! * \date Creation date
! *
! * Some more detailed description. This is optional.
! */
Identifiers
===========
* First character of class names uppercase, any other characters lowercase
* Function and variable names lower case
* 'Multi_word_identifiers' via underline
* 'CONSTANTS' upper case
* Private and protected members of a class begin with an '_'-character
* Accessor functions are named after their corresponding attributes:
! /**
! * Request private member variable
! */
! int value() { return _value; }
!
! /**
! * Set the private member variable
! */
! void value(int value) { _value = value; }
Indentation
===========
* Use one tab per indentation step. *Do not mix tabs and spaces!*
* Use no tabs except at the beginning of a line.
* Use spaces for alignment
See [http://web.archive.org/web/20050311153439/http://electroly.com/mt/archives/000002.html]
for a more detailed description.
This way, everyone can set his preferred tabsize in his editor
and the source code always looks good.
Switch statements
~~~~~~~~~~~~~~~~~
Switch-statement blocks should be indented as follows:
! switch (color) {
!
! case BLUE:
! <tab>break;
!
! case GREEN:
! <tab>{
! <tab><tab>int declaration_required;
! <tab><tab>...
! <tab>}
!
! default:
! }
Please note that the case labels have the same indentation
level as the switch statement. This avoids a two-level
indentation-change at the end of the switch block that
would occur otherwise.
Vertical whitespaces
====================
In header files:
* Leave two empty lines between classes.
* Leave one empty line between member functions.
In implementation files:
* Leave two empty lines between functions.
Braces
======
* Braces after class, struct and function names are placed at a new line:
! class Foo
! {
! public:
!
! void function(void)
! {
! ...
! }
! };
except for single-line functions.
* All other occurrences of open braces (for 'if', 'while', 'do', 'for',
'namespace', 'enum' etc.) are at the end of a line:
! if (flag) {
! ..
! } else {
! ..
! }
* Surprisingly, one-line functions should be written on one line.
Typically, this applies for accessor functions.
If slightly more space than one line is needed, indent as follows:
! int heavy_computation(int a, int lot, int of, int args) {
! return a + lot + of + args; }
Comments
========
Function header
~~~~~~~~~~~~~~~
Each public or protected (but no private) function in a header-file should be
prepended by a header as follows:
! /**
! * Short description
! *
! * \param a meaning of parameter a
! * \param b meaning of parameter b
! * \param c,d meaning of parameters c and d
! *
! * \return meaning of return value
! * \retval 0 meaning of the return value 0
! *
! * More detailed information about the function. This is optional.
! */
Descriptions of parameters and return values should be lower-case and brief.
More elaborative descriptions can be documented in the text area below.
In implementation files, only local and private functions should feature
function headers.
Single-line comments
~~~~~~~~~~~~~~~~~~~~
! /* use this syntax for single line comments */
A single-line comment should be prepended by an empty line.
Single-line comments should be short - no complete sentences. Use lower-case.
C++-style comments ('//') should only be used for temporarily commenting-out
code. Such commented-out garbage is easy to 'grep' and there are handy
'vim'-macros available for creating and removing such comments.
Variable descriptions
~~~~~~~~~~~~~~~~~~~~~
Use the same syntax as for single-line comments. Insert two or more
spaces before your comment starts.
! int size; /* in kilobytes */
Multi-line comments
~~~~~~~~~~~~~~~~~~~
Multi-line comments are more detailed descriptions in the form of
sentences.
A multi-line comment should be enclosed by empty lines.
! /*
! * This is some tricky
! * algorithm that works
! * as follows:
! * ...
! */
The first and last line of a multi-line comment contain no words.
Source-code blocks
~~~~~~~~~~~~~~~~~~
For structuring your source code, you can entitle the different
parts of a file like this:
! <- two empty lines
!
! /********************
! ** Event handlers **
! ********************/
! <- one empty line
Note the two stars at the left and right. There are two of them to
make the visible width of the border match its height (typically,
characters are ca. twice as high as wide).
A source-code block header represents a headline for the following
code. To couple this headline with the following code closer than
with previous code, leave two empty lines above and one empty line
below the source-code block header.
Order of public, protected, and private blocks
==============================================
For consistency reasons, use the following class layout:
! class Sandstein
! {
! private:
! ...
! protected:
! ...
! public:
! };
Typically, the private section contains member variables that are used
by public accessor functions below. In this common case, we only reference
symbols that are defined above as it is done when programming plain C.
Leave one empty line (or a line that contains only a brace) above and below
a 'private', 'protected', or 'public' label. This also applies when the
label is followed by a source-code block header.

357
doc/components.txt Normal file
View File

@@ -0,0 +1,357 @@
==========================
Genode components overview
==========================
Norman Feske
Abstract
########
Genode comes with a growing number of components apparently scattered across
various repositories. This document provides an overview of these components
and outlines the systematics behind them.
Categorization of components
############################
Genode components usually fall into one of four categories, namely device
drivers, resource multiplexers, protocol stacks, and applications. Each
of them is briefly characterized as follows:
:Device drivers: translate hardware resources into device-independent
session interfaces. Naturally, a device driver is specific to a
particular hardware platform. The hardware resources are accessed
via core's IO_MEM, IO_PORT, and IRQ services. The functionality of
the driver is made available to other system components by announcing
one of Genode's device-independent session interfaces, which are
'pci_session', 'framebuffer_session', 'input_session', 'block_session',
'audio_out_session', 'log_session', 'nic_session', and 'timer_session'
(see 'os/include/' for the interface definitions). Those interfaces are
uniform across hardware platforms and kernel base platforms. Usually,
each device driver can accommodate only one client at a time.
:Resource multiplexers: provide mechanisms to multiplex device resources
to multiple clients. A typical resource multiplexer requests one
of Genode's device-independent session interface (usually connected
to a device driver) and, in turn, announces a service of the same kind.
However, in contrast to a device driver, a resource multiplexer is able
to serve more than one client at the same time.
:Protocol stacks: translate low-level interfaces to higher-level
interfaces (or sometimes vice versa). Typically, a protocol stack comes
in the form of a library, which uses a device-independent session
interface as back end and provides a high-level library interface as
front end. However, protocol stacks also exist in the form of
distinct components that implement translations between different
session interfaces.
:Applications: implement functionality using APIs as provided by
protocol stacks.
:Runtime environments: enable existing 3rd-party software to be executed
as a Genode sub systems.
Device drivers
##############
Device drivers usually reside in the 'src/drivers' subdirectory of source-code
repositories. The most predominant repositories hosting device drivers are
'os', 'linux_drivers', 'dde_ipxe'.
Platform devices
================
:'os/src/drivers/pci':
Implements the PCI-session interface using the PCI controller as found on
x86 PC hardware. Using this interface, a client can probe for a particular
device and request information about physical device resources (using the
'pci_device' interface). These information are subsequently used to request
respective IO_MEM, IRQ, and IO_PORT sessions at core.
UART devices
============
The UART device drivers implement the terminal-session interface.
:'os/src/drivers/uart/pl011':
Driver for the PL011 UART as found on many ARM-based platforms.
:'os/src/drivers/uart/i8250':
Driver for the i8250 UART as found on PC hardware.
Framebuffer and input drivers
=============================
Framebuffer and input drivers implement the framebuffer-session interface and
input-session interfaces respectively.
:'os/src/drivers/input/dummy':
Pseudo input driver without accessing any hardware. This component is useful
to resolve a dependency from an input session for scenarios where no user
input is required.
:'os/src/drivers/input/fiasco_ux':
Driver for the virtual hardware provided by the user-mode version of the
Fiasco kernel.
:'os/src/drivers/input/ps2/x86':
Driver for the 'i8042' PS/2 controller as found in x86 PCs. It supports both
mouse (including ImPS/2, ExPS/2) and keyboard.
:'os/src/drivers/input/ps2/pl050':
Driver for the PL050 PS/2 controller as found on ARM platforms such as
VersatilePB. The physical base address used by the driver is obtained at
compile time from a header file called 'pl050_defs.h'. The version of the
VersatilePB platform can be found at 'os/include/platform/vpb926/' and
is made available to the driver via the SPECS machinery of the Genode build
system.
:'os/src/drivers/framebuffer/vesa':
Driver using VESA mode setting on x86 PCs. For more information, please refer
to the README file in the driver directory.
:'os/src/drivers/framebuffer/pl11x':
Driver for the PL110/PL111 LCD display.
:'os/src/drivers/framebuffer/sdl':
Serves as both framebuffer and input driver on Linux using libSDL. This
driver is only usable on the Linux base platform.
:'os/src/drivers/framebuffer/fiasco_ux':
Driver for the virtual framebuffer device provided by the user-mode Fiasco
kernel.
:'linux_drivers/src/drivers/usb':
USB driver that makes USB HID devices available as input sessions.
For an example of using this driver, refer to the run script at
'linux_drivers/run/usb_hid'.
Timer drivers
=============
All timer drivers implement the timer-session interface. Technically, a timer
driver is both a device driver (accessing a timer device) and a resource
multiplexer (supporting multiple timer-session clients at the same time). The
timer implementations differ in their use of different time sources and the
mode of internal operation of the timer sessions. Time sources are either
hardware timers, a time source provided by the kernel, or a pseudo time source
(busy).
The internal operation of the timer session depends on the kernel. On kernels
with support for out-of-order RPCs, all timer sessions are handled by a single
thread. Otherwise, each timer session uses a dedicated thread.
:'timer/nova': PIT as time source, multi-threaded
:'timer/codezero': busy time source, single-threaded
:'timer/okl4_arm': busy time source, single-threaded
:'timer/okl4_x86': PIT as time source, single-threaded
:'timer/foc': IPC timeout as time source, multi-threaded
:'timer/fiasco': IPC timeout as time source, single-threaded
:'timer/pistachio': IPC timeout as time source, single-threaded
:'timer/linux': nanosleep as time source, single-threaded
Audio output drivers
====================
All audio-output drivers implement the audio session interface defined at
'os/include/audio_out_session/'.
:'os/src/drivers/audio_out/linux':
Uses ALSA as back-end on the Linux base platform.
:'linux_drivers/src/drivers/audio_out':
Sound drivers for the most common PC sound hardware, ported from the Linux
kernel.
Block drivers
=============
All block drivers implement the block-session interface defined at
'os/include/block_session/'.
:'os/src/drivers/atapi':
Driver for ATAPI CD-ROM devices on x86 PCs.
:'os/src/drivers/sd_card':
Driver for SD-cards connected via the PL180 device as found on the PBX-A9
platform.
:'linux_drivers/src/drivers/usb':
USB driver that makes USB storage devices available as block sessions.
For an example of using this driver, refer to the run script at
'linux_drivers/run/usb_storage'.
:'os/src/drivers/ahci':
Driver for SATA disks on x86 PCs.
Network interface drivers
=========================
All network interface drivers implement the NIC session interface
defined at 'os/include/nic_session'.
:'os/src/drivers/nic/linux':
Driver that uses a Linux tap device as back end. It is only useful on the
Linux base platform.
:'os/src/drivers/nic/lan9118':
Native device driver for the LAN9118 network adaptor as featured on the
PBX-A9 platform.
:'dde_ipxe/src/drivers/nic':
Device drivers ported from the iPXE project. Supported devices are Intel
E1000 and pcnet32.
:'linux_drivers/src/drivers/madwifi':
The MadWifi wireless stack ported from the Linux kernel.
Resource multiplexers
#####################
By convention, resource multiplexers are located at the 'src/server'
subdirectory of a source repository.
:Framebuffer and input: The framebuffer and input session interfaces can be
multiplexed using the Nitpicker GUI server, which allows multiple clients to
create and manage rectangular areas on screen. Nitpicker uses one input
session and one framebuffer session as back end and, in turn, provides
so-called nitpicker sessions to one or multiple clients. Each nitpicker
session contains a virtual framebuffer and a virtual input session. Nitpicker
(including a README file) is located at 'os/src/server/nitpicker'.
:Audio output: The audio mixer located at 'os/src/server/mixer' enables
multiple clients to use the audio-out interface. The mixing is done by simply
adding and clamping the signals of all present clients.
:Networking: The NIC bridge located at 'os/src/server/nic_bridge' multiplexes
one NIC session to multiple virtual NIC sessions using a proxy-ARP
implementation. Each client has to obtain a dedicated IP address visible to
the physical network. DHCP requests originating from the virtual NIC sessions
are delegated to the physical network.
:Block: The block-device partition server at 'os/src/server/part_blk' reads
the partition table of a block session and exports each partition found as
separate block session. For using this server, please refer to the run
script at 'os/run/part_blk'.
Protocol stacks
###############
Protocol stacks come either in the form of separate components that translate
one session interface to another, or in the form of libraries.
Separate components:
:'os/src/server/nit_fb':
Translates a nitpicker session to a pair of framebuffer and input sessions.
Each 'nit_fb' instance is visible as a rectangular area on screen presenting
a virtual frame buffer. The area is statically positioned. For more
information, please refer to 'os/src/server/nit_fb/README'.
:'demo/src/server/liquid_framebuffer':
Implements the same translation as 'nit_fb' but by presenting an interactive
window rather than a statically positioned screen area.
:'os/src/server/tar_rom':
Provides each file contained in a tar file obtained via Genode's ROM session
as separate ROM session.
:'os/src/server/iso9660':
Provides each file of an ISO9660 file system accessed via a block session as
separate ROM session.
:'os/src/server/rom_loopdev':
Provides the content of a ROM file as a block session, similar to the
loop-mount mechanism on Linux
:'demo/src/server/nitlog':
Provides a LOG session, printing log output on screen via a nitpicker
session.
Libraries:
:'libports/lib/mk/libc_log':
Redirects the standard output of the libc to Genode's LOG session interface.
:'libports/lib/mk/libc_lwip_nic_dhcp':
Translates the BSD socket API to a NIC session using the lwIP stack.
:'libports/lib/mk/gallium' + 'linux_drivers/lib/mk/gpu_i915_drv':
Translates the OpenGL API to a framebuffer session using the MESA OpenGL
stack and the Intel GEM GPU driver.
:'libports/lib/mk/sdl':
Translates the libSDL API to framebuffer and input sessions.
:'qt4':
Qt4 framework, using nitpicker session and NIC session as back end.
Applications
############
Applications are Genode components that use other component's services but
usually do not provide services. They are typically located in the 'src/app/'
subdirectory of a repository. Most applications come with README files
located in their respective directory.
:'demo/src/app/backdrop':
Nitpicker client application that sets a PNG image as desktop background.
:'demo/src/app/launchpad':
Graphical application for interactively starting and killing subsystems.
:'demo/src/app/scout':
Graphical hypertext browser used for Genode's default demonstration scenario.
:'libports/src/app/eglgears':
Example program for using OpenGL via the Gallium3D graphics stack.
:'ports/src/app/arora':
Arora is a Qt4-based web browser using the Webkit engine.
:'ports/src/app/gdb_monitor':
Application that allows the debugging of a process via GDB over a remote
connection.
:'qt4/src/app/qt_launchpad':
Graphical application starter implemented using Qt4.
:'qt4/src/app/examples/':
Several example applications that come with Qt4.
:'os/src/app/xvfb':
Is a proxy application that enables the integration of a virtual X server
into a Nitpicker session on the Linux base platform.
Runtime environments
####################
:'ports/src/noux': Noux is an experimental implementation of a UNIX-like API
that enables the use of unmodified command-line based GNU software. For using
noux, refer to the run script 'ports/run/noux.run'.
:'ports-okl4/src/oklinux': OKLinux is a paravirtualized Linux kernel that
enables the use of Linux-based OSes as subsystems on the OKL4 kernel. For
using OKLinux, refer to the run script 'ports-okl4/run/lx_block.run'.
:'ports-foc/src/l4linux': L4Linux is a paravirtualized Linux kernel that
enables the use of Linux-based OSes as subsystems on the Fiasco.OC kernel.
For using L4Linux, refer to the run script 'ports-foc/run/l4linux.run'.

78
doc/conventions.txt Normal file
View File

@@ -0,0 +1,78 @@
Conventions for the Genode development
Norman Feske
Documentation
#############
We use the GOSH syntax
[http://os.inf.tu-dresden.de/~nf2/files/GOSH/current/gosh.txt]
for documentation and README files.
README files
############
Each directory should contain a file called 'README' that briefly explains
what the directory is about. In 'doc/Makefile' is a rule for
generating a directory overview from the 'README' files automatically.
You can structure your 'README' file by using the GOSH style for subsections:
! Subsection
! ~~~~~~~~~~
Do not use chapters or sections in your 'README' files.
Filenames
#########
All normal filenames are lowercase. Filenames should be chosen to be
expressive. Someone who explores your files for the first time might not
understand what 'mbi.cc' means but 'multiboot_info.cc' would ring a bell. If a
filename contains multiple words, use the '_' to separate them (instead of
'miscmath.h', use 'misc_math.h').
Coding style
############
A common coding style helps a lot to ease collaboration. The official coding
style of the Genode base components is described in 'doc/coding_style.txt'.
If you consider working closely together with the Genode main developers,
your adherence to this style is greatly appreciated.
Include files and RPC interfaces
################################
Never place include files directly into the '<repository>/include/' directory
but use a meaningful subdirectory that corresponds to the component that
provides the interfaces.
Each RPC interface is represented by a separate include subdirectory. For
an example, see 'base/include/ram_session/'. The headerfile that defines
the RPC function interface has the same base name as the directory. The RPC
stubs are called 'client.h' and 'server.h'. If your interface uses a custom
capability type, it is defined in 'capability.h'. Furthermore, if your
interface is a session interface of a service, it is good practice to
provide a connection class in a 'connection.h' file for managing session-
construction arguments and the creation and destruction of sessions.
Specialization-dependent include directories are placed in 'include/<specname>/'.
Service Names
#############
Service names as announced via the 'parent()->announce()' function follow
the following convention:
Core's services, which are the most fundamental base services are written
completely upper case. Each developer should be aware of the meaning of the
used acronyms such as RAM, RM, ROM. All other service names should be
descriptive names rather than acronyms. Service names should contain only
letters, numbers, and underline characters. The first character must
always be an uppercase letter, all other characters are lowercase.

68
doc/future_optimizations.txt Normal file
View File

@@ -0,0 +1,68 @@
Future optimizations of Genode
Norman Feske
Abstract
########
This document outlines possible optimizations for Genode.
In the first place, Genode was meant as a feasibility study.
Therefore, optimizing performance and memory-consumption was
not a primary goal of the experiment. However, there exist
several ideas to improve these properties. These ideas are
definitely not to be regarded as ToDo items. Each idea should
only be conducted if it fixes a _real_ hotspot in the system
that was found by quantitative analysis.
Memory consumption
##################
Currently, we use an AVL-tree-based best-fit memory allocator
('Allocator_avl') as Heap. Despite, this allocator was
intended to be used only for the root allocator for physical
memory inside Core, we use it as the basis for the current
heap implementation. This way, we can reuse the code,
reducing the overall code complexity.
The AVL-tree-based allocator uses a meta-data entry of 32
bytes per free or used memory block. This means that each
memory allocation from the heap introduces a meta-data
overhead of 32 up to 64 bytes. When allocating a lot of
small objects, this overhead is very high.
:Question:: Is this issue a real bottleneck?
Possible improvements are:
* Using slab allocators for known allocation sizes.
Slab entries have a much smaller footprint.
* Creating a list-based allocator implementation for
the heap. This comes at the cost of additional
code complexity.
RPC Performance
###############
We use C++ streams for RPC communication and therefore,
introduced run-time overhead for the dynamic marshaling.
Using an IDL compiler would improve the RPC performance.
Is the RPC performance a real performance problem? What
is the profile of RPC usage? (Number of RPCs per second,
Percentage of CPU time spent on RPCs, Secondary effects
for RPCs such as cache and TLB pollution).
Locking
#######
On L4v2-Genode, locking is implemented via yielding spin locks.
We may consider a L4env-like lock implementation.
Misc
####
Take a look at include/util/string.h and judge by yourself :-)

116
doc/getting_started.txt Normal file
View File

@@ -0,0 +1,116 @@
=============================
How to start exploring Genode
=============================
Norman Feske
Abstract
########
This guide is meant to provide you a painless start with using the Genode OS
Framework. It explains the steps needed to get a simple demo system running
on Linux first, followed by the instructions on how to run the same scenario
on a microkernel.
Quick start to build Genode for Linux
#####################################
The best starting point for exploring Genode is to run it on Linux. Make sure
that your system satisfies the following requirements:
* GNU 'Make' version 3.81 or newer
* 'libSDL-dev'
* 'tclsh' and 'expect'
* 'byacc' (only needed for the L4/Fiasco kernel)
* 'qemu' and 'genisoimage' (for testing non-Linux platforms via Qemu)
Furthermore, you will need to install the official Genode toolchain, which
you can download at [http://genode.org/download/tool-chain].
The Genode build system never touches the source tree but generates object
files, libraries, and programs in a dedicated build directory. We do not have a
build directory yet. For a quick start, let us create one for the Linux base
platform:
! cd <genode-dir>
! ./tool/create_builddir linux_x86 BUILD_DIR=build.lx
The new build directory is called 'build.lx' and configured for the 'linux_x86'
platform. To give Genode a try, build and execute a simple demo scenario via:
! cd build.lx
! make run/demo
By invoking 'make' with the 'run/demo' argument, all components needed by the
demo scenario are built and the demo is executed. If you are interested in
looking behind the scenes of the demo scenario, please refer to
'doc/build_system.txt' and the run script at 'os/run/demo.run'.
Using platforms other than Linux
================================
Running Genode on Linux is the most convenient way to get acquainted with the
framework. However, the point where Genode starts to shine is when used as the
user land executed on a microkernel. The framework supports a variety of
different kernels such as L4/Fiasco, L4ka::Pistachio, OKL4, and NOVA. Those
kernels largely differ in terms of feature sets, build systems, tools, and boot
concepts. To relieve you from dealing with those peculiarities, Genode provides
you with an unified way of using them. For each kernel platform, there exists
a dedicated directory called 'base-<platform>'. Within this directory, you will
find a 'Makefile', which automates the task of downloading the source codes of
the kernel and interfacing the kernel with Genode. Just change to the
respective 'base-<platform>' directory and issue:
! make prepare
Note that each 'base-<platform>' directory comes with a 'README' file, which
you should revisit first when exploring the base platform. Additionally, most
'base-<platform>' directories provide more in-depth information within their
respective 'doc/' subdirectories.
Now that the base platform is prepared, the 'create_builddir' tool can be used
to create a build directory for your platform of choice by giving the platform
as argument. To see the list of available platforms, execute 'create_builddir'
with no arguments.
For example, to give the demo scenario a spin on the OKL4 kernel, the following
steps are required:
# Download the kernel:
! cd <genode-dir>
! make -C base-okl4 prepare
# Create a build directory
! ./tool/create_builddir okl4_x86 BUILD_DIR=build.okl4
# Build and execute the demo using Qemu
! make -C build.okl4 run/demo
The procedure works analogously for the other base platforms.
How to proceed with exploring Genode
####################################
Now that you have taken the first steps into using Genode, you may seek to
get more in-depth knowledge and practical experience. The foundation for doing
so is a basic understanding of the build system. The documentation at
'build_system.txt' provides you with the information about the layout of the
source tree, how new components are integrated, and how complete system
scenarios can be expressed. Equipped with this knowledge, it is time to get
hands-on experience with creating custom Genode components. A good start is the
'hello_tutorial', which shows you how to implement a simple client-server
scenario. To compose complex scenarios out of many small components, the
documentation of the Genode's configuration concept at 'os/doc/init.txt' is an
essential reference.
Certainly, you will have further questions on your way with exploring Genode.
The best place to get these questions answered is the Genode mailing list.
Please feel welcome to ask your questions and to join the discussions:
:Genode Mailing Lists:
[http://genode.org/community/mailing-lists]

830
doc/release_notes-08-11.txt Normal file
View File

@@ -0,0 +1,830 @@
==============================================
Release notes for the Genode OS Framework 8.11
==============================================
Genode Labs
Summary
#######
This document presents the new features and major changes introduced
in version 8.11 of the Genode OS Framework. It is geared towards
people interested in closely following the progress of the Genode
project and to developers who want to adopt their software to our
mainline development. The document aggregates important fragments
of the updated documentation such that you won't need to scan existing
documents for the new bits. Furthermore, it attempts to provide our
rationale behind the taken design decisions.
The general theme for the release 8.11 is enabling the use of the
Genode OS framework for real-world applications. Because we regard
the presence of device drivers and a way to reuse existing library
code as fundamental prerequisites for achieving this goal, the major
new additions are an API for device drivers written in C, an API for
handling asynchronous notifications, and a C runtime. Other noteworthy
improvements are the typification of capabilities at the C++-language
level, a way for receiving and handling application faults, the
introduction of managed dataspaces, and a new API for scheduling
timed events.
Base framework
##############
This section documents the new features and changes affecting the
'base' repository, in particular the base API.
New features
============
Connection handling
~~~~~~~~~~~~~~~~~~~
The interaction of a client with a server involves the definition of
session-construction arguments, the request of the session creation via
its parent, the initialization of the matching RPC-client stub code
with the received session capability, the actual use of the session
interface, and the closure of the session. A typical procedure of
using a service looks like this:
!#include <rom_session/client.h>
!...
!
!/* construct session-argument string and create session */
!char *args = "filename=config, ram_quota=4K");
!Capability session_cap = env()->parent()->session("ROM", args);
!
!/* initialize RPC stub code */
!Rom_session_client rsc(session_cap);
!
!/* invoke remote procedures, 'dataspace' is a RPC function */
!Capability ds_csp = rsc.dataspace();
!...
!
!/* call parent to close the session */
!env()->parent()->close(session_cap);
Even though this procedure does not seem to be overly complicated,
is has raised the following questions and criticism:
* The quota-donation argument is specific for each server. Most services
use client-donated RAM quota only for holding little meta data and,
thus, are happy with a donation of 4KB. Other services maintain larger
client-specific state and require higher RAM-quota donations. The
developer of a client has to be aware about the quota requirements for
each service used by his application.
* There exists no formalism for documenting session arguments.
* Because session arguments are passed to the 'session'-call as a plain
string, there are no syntax checks for the assembled string performed
at compile time. For example, a missing comma would go undetected until
a runtime test is performed.
* There are multiple lines of client code needed to open a session to
a service and the session capability must be maintained manually for
closing the session later on.
The new 'Connection' template provides a way to greatly simplify the
handling of session arguments, session creation, and destruction on the
client side. By implementing a service-specific connection class
inherited from 'Connection', session arguments become plain constructor
arguments, session functions can be called directly on the 'Connection'
object, and the session gets properly closed when destructing the
'Connection'. By convention, the 'Connection' class corresponding to a
service resides in a file called 'connection.h' in the directory of the
service's RPC interface. For each service, a corresponding 'Connection'
class becomes the natural place where session arguments and quota
donations are documented. With this new mechanism in place, the example
above becomes as simple as:
!#include <rom_session/connection.h>
!...
!
!/* create connection to the ROM service */
!Rom_connection rom("config");
!
!/* invoke remote procedure */
!Capability ds_csp = rom.dataspace();
[http://genode.org/documentation/api/base_index#Connecting_to_services - See the API documentation for the connection template...]
Typed capabilities
~~~~~~~~~~~~~~~~~~
A plain 'Capability' is an untyped reference to a remote object of any
type. For example, a capability can reference a thread object or a
session to a service. It is loosely similar to a C void pointer, for which
the programmer maintains the knowledge about which data type is actually
referenced. To facilitate the type-safe use of RPC interfaces at the C++
language level, we introduced a template for creating specialized
capability types ('Typed_capability' in 'base/typed_capability.h') and
the convention that each RPC interface declares a dedicated capability
type. Note that type-safety is not maintained across RPC interfaces. As
illustrated in Figure [img/layered_ipc], typification is done at the
object-framework level on the server side and via in the 'Connection'
classes at the client side.
[image img/layered_ipc]
From the application-developer's perspective, working with capabilities
has now become type-safe, making the produced code more readable and robust.
[http://genode.org/documentation/api/base_index#Capability_representation - See the updated API documentation for the capability representation...]
Fifo data structure
~~~~~~~~~~~~~~~~~~~
Because the 'List' data type inserts new list elements at the list head,
it cannot be used for implementing wait queues requiring first-in
first-out semantics. For such use cases, we introduced a dedicated
'Fifo' template. The main motivation for introducing 'Fifo' into the
base API is the new semaphore described below.
[http://genode.org/documentation/api/base_index#Structured_data_types - See the new API documentation for the fifo template...]
Semaphore
~~~~~~~~~
Alongside lock-based mutual exclusion of entering critical sections,
organizing threads in a producer-consumer relationship via a semaphore
is a common design pattern for thread synchronization. Prior versions
of Genode provided a preliminary semaphore implementation as part of
the 'os' repository. This implementation, however, supported only one
consumer thread (caller of the semaphore's 'down' function). We have
now enhanced our implementation to support multiple consumer threads
and added the semaphore to Genode's official base API. We have made
the wake-up policy in the presence of multiple consumers configurable
via a template argument. The default policy is first-in-first-out.
[http://genode.org/documentation/api/base_index#Synchronization - See the new API documentation for the semaphore...]
Thanks to Christian Prochaska for his valuable contributions to the new
semaphore design.
Asynchronous notifications
~~~~~~~~~~~~~~~~~~~~~~~~~~
Inter-process communication via remote procedure calls requires both
communication partners to operate in a synchronous fashion. The caller
of an RPC blocks as long as the RPC is not answered by the called
server. In order to receive the call, the server has to explicitly
wait for incoming messages. There are a number of situations where
synchronous communication is not suited.
For example, a GUI server wants to deliver a notification to one of its
clients about new input events being available. It does not want to
block on a RPC to one specific client because it has work to do for
other clients. Instead, the GUI server wants to deliver this
_notification_ with _fire-and-forget_ semantics and continue with
its operation immediately, regardless of whether the client received
the notification or not. The client, in turn, does not want to poll
for new input events at the GUI server but it wants to be _waken_up_
when something interesting happens. Another example is a block-device
driver that accepts many requests for read/write operations at once.
The operations may be processed out of order and may take a long time.
When having only synchronous communication available, the client and
the block device driver would have to employ one distinct thread for
each request, which is complicated and a waste of resources. Instead,
the block device driver just wants to acknowledge the completeness of
an operation _asynchronously_.
Because there are many more use cases for asynchronous inter-process
communication, we introduced a new signalling framework that complements
the existing synchronous RPC mode of communication with an interface for
issuing and receiving asynchronous notifications. It defines interfaces
for signal transmitters and signal receivers. A signal receiver can
receive signals from multiple sources, whereas the sources of incoming
signals are clearly distinguishable. One or multiple threads can either
poll or block for incoming signals. Each signal receiver is addressable
via a capability. The signal transmitter provides fire-and-forget
semantics for submitting signals to exactly one signal receiver. Signals
are communicated in a reliable fashion, which means that the exact number
of signals submitted to a signal transmitter is communicated to the
corresponding signal receiver. If notifications are generated at a higher
rate than as they can be processed at the receiver, the transmitter
counts the notifications and delivers the total amount with the next
signal transmission. This way, the total number of notifications gets
properly communicated to the receiver even if the receiver is not highly
responsive. Notifications do not carry any payload because this payload
would have to be queued at the transmitter.
[image img/signals]
Image [img/signals] illustrates the roles of signaller thread,
transmitter, receiver, and signal-handler thread.
[http://genode.org/documentation/api/base_index#Asynchronous_notifications - See the new API documentation for asynchronous notifications...]
The current generic implementation of the signalling API employs one
thread at each transmitter and one thread at each receiver. Because
the used threads are pretty heavy weight with regard to resource usage,
ports of Genode should replace this implementation with platform-
specific variants, for example by using inter-process semaphores or
native kernel support for signals.
Region-manager faults
~~~~~~~~~~~~~~~~~~~~~
In Genode, region-manager (RM) sessions are used to manage the
address-space layout for processes. A RM session is an address-space
layout that can be populated by attaching (portions of) dataspaces to
(regions of) the RM session. Normally, the RM session of a process is
first configured by the parent when decoding the process' ELF binary.
During the lifetime of the process, the process itself may attach
further dataspaces to its RM session to access the dataspace's content.
Core as the provider of the RM service uses this information for
resolving page faults raised by the process. In prior versions of
Genode, core ignored unresolvable page faults, printed a debug message
and halted the page-faulted thread. However, this condition may be of
interest, in particular to the process' parent for reacting on the
condition of a crashed child process. Therefore, we enhanced the RM
interface by a fault-handling mechanism. For each RM session, a fault
handler can be installed by registering a signal receiver capability.
If an unresolvable page fault occurs, core delivers a signal to the
registered fault handler. The fault handler, in turn, can request the
actual state of the RM session (page-fault address) and react upon
the fault. One possible reaction is attaching a new dataspace at the
fault address and thereby implicitly resolving the fault. If core
detects that a fault is resolved this way, it resumes the operation
of the faulted thread.
This mechanism works analogously to how page faults are handled by
CPUs, but on a more abstract level. A (n-level) page table corresponds
to a RM session, a page-table entry corresponds to a dataspace-
attachment, the RM-fault handler corresponds to a page-fault
exception handler, and the resolution of page-faults (RM fault)
follows the same basic scheme:
# Application accesses memory address with no valid page-table-entry
(RM fault)
# CPU generates page-fault exception (core delivers signal to fault
handler)
# Kernel reads exception-stack frame or special register to determine
fault address (RM-fault handler reads RM state)
# Kernel adds a valid page-table entry and returns from exception
(RM-fault handler attaches dataspace to RM session, core resumes
faulted thread)
The RM-fault mechanism is not only useful for detecting crashing child
processes but it enables a straight-forward implementation of growing
stacks and heap transparently for a child process. An example for
using RM-faults is provided at 'base/src/test/rm_fault'.
Note that this mechanism is only available on platforms on which core
resolves page faults. This is the case for kernels of the L4 family.
On Linux however, the Linux kernel resolves page faults and suspends
processes performing unresolvable memory accesses (segmentation fault).
Managed dataspaces (experimental)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The RM-fault mechanism clears the way for an exciting new feature
of Genode 8.11 called managed dataspaces. In prior versions of Genode,
each dataspace referred to a contiguous area of physical memory (or
memory-mapped I/O) obtained by one of core's RAM, ROM, or IO_MEM
services, hence we call them physical dataspaces. We have now added
a second type of dataspaces called managed dataspaces. In contrast
to a physical dataspace, a managed dataspace is backed by the content
described by an RM session. In fact, each RM session can be used as
dataspace and can thereby be attached to other RM sessions.
Combined with the RM fault mechanism described above, managed
dataspaces enable a new realm of applications such as dataspaces
entirely managed by user-level services, copy-on-write dataspaces,
non-contiguous large memory dataspaces that are immune to physical
memory fragmentation, process-local RM fault handlers (e.g., managing
the own thread-stack area as a sub-RM-session), and sparsely populated
dataspaces.
Current limitations
-------------------
Currently, managed dataspaces still have two major limitations. First,
this mechanism allows for creating cycles of RM sessions. Core must
detect such cycles during page-fault resolution. Although, a design for
an appropriate algorithm exists, cycle-detection is not yet implemented.
The missing cycle detection would enable a malicious process to force
core into an infinite loop. Second, RM faults are implemented using the
new signalling framework. With the current generic implementation, RM
sessions are far more resource-demanding than they should be. Once the
signalling framework is optimized for L4, RM sessions and thereby
managed dataspaces will become cheap. Until then, we do not recommend
to put this mechanism to heavy use.
Because of these current limitations, managed dataspaces are marked as
an experimental feature. When building Genode, experimental features are
disabled by default. To enable them, add a file called 'specs.conf'
with the following content to the 'etc/' subdirectory of your build
directory:
! SPECS += experimental
For an example of how to use the new mechanism to manage a part of a
process' own address space by itself, you may take a look at
'base/src/test/rm_nested'.
Changes
=======
Besides the addition of the new features described above, the following
parts of the base framework underwent changes worth describing.
Consistent use of typed capabilities and connection classes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
We applied capability typification to all interfaces of Genode including
the base API and the interfaces defined in the 'os' repository. Figure
[img/base_cap_types] provides an overview about the capability types
provided by the base API.
[image img/base_cap_types]
Overview about the capability types provided by the base API
Furthermore, we have complemented all session interfaces with
appropriate 'Connection' classes taking service-specific session
arguments into account.
For session-interface classes, we introduced the convention to declare
the service name as part of the session-interface via a static member
function:
! static const char *service_name();
Allocator refinements
~~~~~~~~~~~~~~~~~~~~~
Throughout Genode, allocators are not only used for allocating memory
but also for managing address-space layouts and ranges of physical
resources such as I/O-port ranges or IRQ ranges. In these cases, the
address '0' may be a valid value. Consequently, this value cannot be
used to signal allocation errors as done in prior versions of Genode.
Furthermore, because managed dataspaces use the RM session interface to
define the dataspace layout, the address-'0' problem applies here as
well. We have now refined our allocator interfaces and the RM-session
interface to make them fit better for problems other than managing
virtual memory.
Misc changes
~~~~~~~~~~~~
We revised all interfaces to consistently use _exceptions_ to signal
error conditions rather than delivering error codes as return values.
This way, error codes become exception types that have a meaningful
name and, in contrast to global 'errno' definitions, an error exception
type can be defined local to the interface it applies to. Furthermore,
the use of exceptions allows for creating much cleaner looking interfaces.
Traditionally, we have provided our custom _printf_ implementation as C
symbol to make this function available from both C and C++ code. However,
we observed that we never called this function from C code and that the
'printf' symbol conflicts with the libc. Hence, we turned 'printf'
into a C++ symbol residing in the 'Genode' namespace.
Operating-system services and libraries
#######################################
This section documents the new features and changes affecting
the 'os' repository.
New Features
============
Device-driver framework for C device drivers
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Genode's base API features everything needed to create user-level device
drivers. For example, the 'os' repository's PS/2 input driver and the
PCI bus driver are using Genode's C++ base API directly. However, most of
today's device drivers are written in C. To ease the reuse of existing
drivers on Genode, we have introduced a C API for device drivers into
Genode's 'os' repository. The API is called DDE kit (DDE is an acronym
for device-driver environment) and it is located at 'os/include/dde_kit'.
The DDE kit API is the result of long-year experiences with porting device
drivers from Linux and FreeBSD to custom OS environments. The following
references are the most significant contributions to the development of
the API.
;
Christian Helmuth created the initial version of the Linux device-driver
environment for L4. He describes his effort of reusing unmodified sound
drivers on the L4 platform in his thesis
[http://os.inf.tu-dresden.de/papers_ps/helmuth-diplom.pdf - Generische Portierung von Linux-Gerätetreibern auf die DROPS-Architektur].
;
Gerd Griessbach approached the problem of re-using Linux USB drivers
by following the DDE approach in his diploma thesis
[http://os.inf.tu-dresden.de/papers_ps/griessbach-diplom.pdf - USB for DROPS].
;
Marek Menzer adapted Linux DDE to Linux 2.6 and explored the DDE
approach for block-device drivers in his student research project
[http://os.inf.tu-dresden.de/papers_ps/menzer-beleg.pdf - Portierung des DROPS Device Driver Environment (DDE) für Linux 2.6 am Beispiel des IDE-Treibers ]
and his diploma thesis
[http://os.inf.tu-dresden.de/papers_ps/menzer-diplom.pdf - Entwicklung eines Blockgeräte-Frameworks für DROPS].
;
Thomas Friebel generalized the DDE approach and introduced the DDE kit
API to enable the re-use of device driver from other platforms than
Linux. In particular, he experimented with the block-device drivers of
FreeBSD in his diploma thesis
[http://os.inf.tu-dresden.de/papers_ps/friebel-diplom.pdf - Übertragung des Device-Driver-Environment-Ansatzes auf Subsysteme des BSD-Betriebssystemkerns].
;
Dirk Vogt successfully re-approached the port of USB device drivers
from the Linux kernel to L4 in his student research project
[http://os.inf.tu-dresden.de/papers_ps/vogt-beleg.pdf - USB for the L4 Environment].
The current incarnation of the DDE kit API provides the following
features:
* General infrastructure such as init calls, assertions, debug output
* Interrupt handling (attach, detach, disable, enable)
* Locks, semaphores
* Memory management (slabs, malloc)
* PCI access (find device, access device config space)
* Virtual page tables (translation between physical and virtual
addresses)
* Memory-mapped I/O, port I/O
* Multi-threading (create, exit, thread-local storage, sleep)
* Timers, jiffies
For Genode, we have created a complete reimplementation of the DDE kit
API from scratch by fully utilizing the existing Genode infrastructure
such as the available structured data types, core's I/O services,
the synchronization primitives, and the thread API.
[image img/dde_kit]
Figure [img/signals] illustrates the role of DDE kit when re-using an
unmodified device driver taken from the Linux kernel. DDE kit translates
Genode's C++ base API to the DDE kit C API. The DDE kit API, in turn, is
used as back end by the Linux driver environment, which translates Linux
kernel interfaces to calls into DDE kit. With this translation in place,
an unmodified Linux device driver can be embedded into the Linux driver
environment. The device API is specific for a class of devices such as
NICs, block devices, or input devices. It can either be used directly as
a function interface by an application that is using the device driver
as a library, or it can be made accessible to external processes via an
RPC interface.
Limitations
-----------
The PCI sub system is not completely implemented, yet.
Alarm API providing a timed event scheduler
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The scheduling of timed events is a recurring pattern found in device
drivers, application frameworks such as Qt4 ('qeventdispatcher'), and
applications. Therefore, we have added a timed event scheduler to the
'os' repository. The new alarm API ('os/include/os/alarm.h') allows
for the scheduling of both one-shot alarms and periodic alarms.
Changes
=======
PS/2 input driver
~~~~~~~~~~~~~~~~~
The original PS/2 driver tried to switch the PS/2 keyboard to
scan-code set 2 and assumed that all modern keyboards support this
mode of operation. However, this assumption was wrong. We observed
that the legacy PS/2 support of some USB keyboards covers only the
emulated (xlate) scan-code set 1 mode. This is also case for the PS/2
emulation in VirtualBox. Therefore, we changed our PS/2 driver to
never touch the keyboard mode but to only detect the current mode
of operation. The driver has now to support both, scan-code set 1 and
scan-code set 2. This change comes along with a slightly more complex
state machine in the driver. Hence, we moved the state machine from
the IRQ handler to a distinct class and changed the control flow of
the driver to fetch only one value from the i8042 PS/2 controller
per received interrupt.
PCI bus driver
~~~~~~~~~~~~~~
Until now, Genode's PCI bus driver was only used for experimentation
purposes. With the forthcoming driver framework however, the PCI bus
driver will play a central role in the system. Therefore, we adapted
the interface of the PCI driver to these requirements. Specifically,
the scanning of the PCI bus can now be performed without constraining
the results by a specific vendor ID.
Nitpicker GUI server
~~~~~~~~~~~~~~~~~~~~
We improved the _output_latency_ of the Nitpicker GUI server by flushing
pixels eagerly and deferring the next periodically scheduled flush.
This change has a positive effect on the responsiveness of the GUI to
user input.
Misc changes
~~~~~~~~~~~~
Prior versions of the 'os' repository came with a custom 'os/include/base'
directory with interfaces extending the base API. To avoid confusion
between the 'base' repository and the 'os' repository, 'os'-local API
extensions are now located at 'os/include/os'. This way, the folder
prefix of include statements indicates well from which repository the
included header files comes from.
C runtime
#########
Most of existing libraries rely on the presence of a C library. For
making the reuse of this software on Genode possible, we have now
made a complete C library available for Genode. It comes as a separate
source-code repository called 'libc' and is based on the code of FreeBSD.
The original code is available at the official FreeBSD website.
:FreeBSD website:
[http://www.freebsd.org/developers/cvs.html]
Our libc port comprises the libraries 'gdtoa', 'gen', 'locale', 'stdio',
'stdlib', 'stdtime', 'string', and 'msun'. Currently, it supports the
x86 architecture. Support for other architectures is planned as future
addition. At the current stage, our back end is very basic and most of
its functions are dummy stubs. We used Christian Prochaska's forthcoming
Genode port of Qt4 as test case and successfully used the new libc as
foundation for building graphical Qt4 applications. We will further
extend the back end in correspondence to the growing feature set of the
Genode OS framework.
:Usage:
To use the libc in your application, just add 'libc' to the 'LIBS'
declaration in your build-description file. This declaration will make
the libc headers available for the include path of your target and link
the C library. When building, make sure that the 'libc' repository is
included in your build configuration ('etc/build.conf').
:Limitations:
The current version of the C library is not thread-safe. For most
string and math functions, this is not a problem (as these functions
do not modify global state) but be careful with using more complex
functions such as 'malloc' from multiple threads. Also, 'errno' may
become meaningless when calling libc functions from multiple threads.
We have left out the following files from the Genode port of the
FreeBSD libc: gdtoa 'strtodnrp.c' (gdtoa), 'getosreldate.c' (gen),
'strcoll.c', 'strxfrm.c', 'wcscoll.c', 'wcsxfrm.c' (string),
's_exp2l.c' ('msun').
The current back end is quite simplistic and it may help you to revisit
the current state of the implementation in the 'libc/src/lib/libc'
directory. If one of the functions in 'dummies.c' is called, you will
see the debug message:
! "<function-name> called, not yet implemented!"
However, some of the back-end function implemented in the other files
have dummy semantics but have to remain quiet because they are called
from low-level libc code.
Build infrastructure
####################
Build-directory creation tool
=============================
Because we think that each Genode developer benefits from knowing the
basics about the functioning of the build system, the manual creation of
build directories is described in Genode's getting-started document.
However, for regular developers, creating build directories becomes a
repetitive task. Hence, it should be automated. We have now added a
simple build-directory creation tool that creates pre-configured build
directories for some supported platforms. The tool is located at
'tool/builddir/create_builddir'. To print its usage information, just
execute the tool without arguments.
Improved linking of binary files
================================
For linking binary data, binary file have to be converted to object
files. Over the time, we have used different mechanisms for this
purpose. Originally, we used 'ld -r -b binary'. Unfortunately, these
linker options are not portable. Therefore, the mechanism was changed
to a 'hexdump' and 'sed' magic that generated a C array from binary data.
This solution however, is complicated and slow. Now, we have adopted
an idea of Ludwig Hähne to use the 'incbin' directive of the GNU
assembler, which is a very clean, flexible, and fast solution.
Lib-import mechanism
====================
Libraries often require specific include files to be available at the
default include search location. For example, users of a C library
expect 'stdio.h' to be available at the root of the include search
location. Placing the library's include files in the root of the
default search location would pollute the include name space for
all applications, regardless if they use the library or not. To
keep library-include files well separated from each other, we have
enhanced our build system by a new mechanism called lib-import.
For each library specified in the 'LIBS' declaration of a build
description file, the build system incorporates a corresponding
'import-<libname>.mk' file into the build process. Such as file
defines library-specific compiler options, in particular additional
include-search locations. The build system searches for lib-import
files in the 'lib/import/' subdirectories of all used repositories.
Using 'ar' for creating libraries
=================================
The previous versions of Genode relied on incremental linking ('ld -r')
for building libraries. This approach is convenient because the linker
resolves all cross-dependencies between libraries regardless of the
order of how libraries are specified at the linker's command line.
However, incremental linking prevents the linker from effectively
detecting dead code. In contrast, when linking '.a' files, the linker
detects unneeded object files. Traditionally, we have only linked our
own framework containing no dead code. This changed with the new 'libc'
support. When linking the 'libc', the presence of dead code becomes
the normal case rather than the exception. Consequently, our old
incremental-linking approach produced exceedingly large binaries
including all functions that come with the 'libc'. We have now adopted
the classic 'ar' mechanism for assembling libraries and use the linker's
'start-group' 'end-group' feature to resolve inter-library-dependencies.
This way, dead code gets eliminated at the granularity of object files.
In the future, we will possible look into the '-ffunction-sections' and
'-gc-sections' features of the GNU tool chain to further improve the
granularity to function level.
If your build-description files rely on custom rules referring to
'lib.o' files, these rules must be adapted to refer to 'lib.a' files
instead.
Misc changes
============
* Added sanity check for build-description files overriding 'INC_DIR'
instead of extending it.
* Restrict inclusion of dependency files to those that actually matter
when building libraries within 'var/libcache'. This change significantly
speeds up the build process in the presence of large libraries such as
Qt4 and libc.
* Added rule for building 'cpp' files analogously to the 'cc' rule.
Within Genode, we name all C++ implementation files with the 'cc'
suffix. However, Qt4 uses 'cpp' as file extension so we have to
support both.
* Build-description files do no longer need the declaration
'REQUIRES = genode'. Genode's include search locations are now
incorporated into the build process by default.
Applications
############
This section refers to the example applications contained in Genode's
'demo' repository.
We have enhanced the _Scout_widgets_ as used by the launchpad and the
Scout tutorial browser to perform all graphical output double-buffered,
which effectively eliminates drawing artifacts that could occur when
exposing intermediate drawing states via direct (unbuffered) output.
Furthermore, we have added a way to constrain the maximum size of
windows to perform pixel-buffer allocations on realistic window sizes.
Both launchpad and Scout can now start child applications. In Scout
this functionality is realized by special "execute" links. We have
generalized the underlying application logic for creating and
maintaining child processes between both applications and placed
the unification into a separate 'launchpad' library.
We have replaced the default document presented in Scout with an
_interactive_walk-through_guide_ explaining the basic features of Genode.
The document uses the new "execute" link facility to let the user start
a launchpad instance by clicking on a link.
Platform-specific changes
#########################
Genode used to define _fixed-width_integer_types_ in a file 'stdint.h'
placed in a directory corresponding to bit-width of the platform, for
example 'include/32bit/stdint.h'. When building for a 32bit platform,
the build system included the appropriate directory into the
include-search path and thereby made 'stdint.h' available at the root
of the include location. Unfortunately, this clashes with the 'stdint.h'
file that comes with the C library. To avoid conflict with libc header
files, we moved the definition of fixed-width integer types to
'32bit/base/fixed_stdint.h'.
For the L4/Fiasco version of Genode, there existed some x86-specific
header files that did not specifically depend on L4/Fiasco, for example
atomic operations. Because these files are not L4/Fiasco-specific and
may become handy for other platforms as well, we moved them to the
generic 'base' repository.
Linux 32bit
===========
:Dissolving Genode's dependency from the glibc:
The port of the C runtime to Genode posed an interesting challenge to
the Linux version of Genode. This version used to rely on certain
functions provided by the underlying glibc:
* For creating and destroying threads, we used to rely on POSIX threads
as provided by the 'pthread' library
* The lock implementation was based on the POSIX semaphore functions
'sem_init', 'sem_wait', and 'sem_post'
* Shared memory was realized by using files ('open', 'close',
'ftruncate') and the 'mmap' interface
* Starting and killing processes was implemented using 'fork' and 'kill'
* Inter-process communication used the glibc's socket functions
For our custom C runtime, we want to override the glibc functionality
with our own implementation. For example, we want to provide the 'mmap'
interface to a Genode application by implementing 'mmap' with
functions of our base API. On Linux, however, this base API, in turn,
used to rely on 'mmap'. This is just an example. The problem applies
also for the other categories mentioned above. We realized that we cannot
rely on the glibc on one hand but at the same time replace it by a custom
C runtime (in fact, we believe that such a thing is possible by using
awkward linker magic but we desire a clean solution). Consequently, we
have to remove the dependency of Genode from the glibc on Linux. Step
by step, we replaced the used glibc functions by custom Linux system-call
bindings. Each binding function has a prefix 'lx_' such that the symbol
won't collide with 'libc' symbols. The new bindings are located at the file
'base-linux/src/platform/linux_syscalls.h'. It consist of 20 functions,
most of them resembling the original interface ('socket', 'connect',
'bind', 'getsockname', 'recvfrom', 'write', 'close', 'open', 'fork',
'execve', 'mmap', 'ftruncate', 'unlink', 'tkill', 'nanosleep').
For other functions, we simplified the semantics for our use case
('sigaction', 'sigpending', 'sigsetmask', 'create_thread'). The most
noteworthy changes are the creation and destruction of threads by
directly using the 'clone' and 'tkill' system calls, and the lock
implementation. Because we cannot anymore rely on the convenience of
using futexes indirectly through the POSIX semaphore interface, we
have adopted the simple locking approach that we already use for the
L4/Fiasco version. This lock implementation is a simple sleeping
spinlock.
:Compromises:
The introduction of custom Linux system-call bindings for Genode has
several pros and cons. With this change, The Linux version of Genode is
not anymore easy to port to other POSIX platforms such as the Darwin
kernel. For each POSIX kernel used as Genode platform, a custom
implementation of our system-call bindings must be created. The
original POSIX variant could still be reanimated, but this version
would inherently lack support for Genode's C runtime, and thus would
have limited value. A positive side effect of this solution, however,
is that 'linux_syscalls.h' documents well the subset of the Linux'
kernel interface that we are actually using.
The replacement of POSIX semaphores with sleeping spinlocks decreases
locking performance quite significantly. In the contention case, the
wakeup from sleeping introduces a high latency of up to one millisecond.
Furthermore, fairness is not guaranteed and the spinning produces a bit
of system load. If this approach turns out to become a serious performance
bottleneck, we will consider creating custom bindings for Linux' futexes.
L4/Fiasco
=========
The concepts of _RM_faults_ and _managed_dataspaces_ as described in
Section [Base framework], had been implemented into the L4/Fiasco
version of core. Although the introduction of these concepts involved
only minimal changes at the API level, the required core-internal
changes had been quite invasive, affecting major parts of the pager
and RM-session implementations.
Prior versions of the L4/Fiasco version of core did not implement
the _cancel-blocking_mechanism_ as specified by the CPU-session API.
The missing implementation resulted in lock-ups when destructing a
thread that blocks for lock. With the new implementation based on
L4/Fiasco's inter-task ex-regs system call, such threads can now
be gracefully destructed.

460
doc/release_notes-09-02.txt Normal file
View File

@@ -0,0 +1,460 @@
==============================================
Release notes for the Genode OS Framework 9.02
==============================================
Genode Labs
Summary
#######
Whereas the focus of the previous release 8.11 was the refinement of
Genode's base API and the creation of the infrastructure needed to build
real-world applications, the release 9.02 is focused on functional
enhancements in two directions. The first direction is broadening the
number of possible base platforms for the framework. At present, most
microkernels bring along a custom user land, which is closely tied to the
particular kernel. Our vision is to establish Genode as a common ground for
developing applications, protocol stacks, and device drivers in such a way
that the software becomes easily portable among different kernels. This
release makes Genode available on the L4ka::Pistachio kernel. Hence,
software developed with the Genode API can now run unmodified on
Linux/x86, L4/Fiasco, and L4ka::Pistachio. In the second direction, we
are steadily advancing the functionality available on top of Genode. With
this release, we introduce a basic networking facility and support for
native Qt4 applications as major new features. Thanks to Genode's
portability, these features become automatically available on all
supported base platforms.
Our original plan for the release 9.02 also comprised the support of a
Linux-on-Genode (para-)virtualization solution. Initially, we intended to
make [http://os.inf.tu-dresden.de/L4/LinuxOnL4/ - L4Linux] available on
the L4/Fiasco version of Genode. However, we identified several downsides
with this approach. Apparently, the development of the officially available
version of L4/Fiasco has become slow and long-known issues remain unfixed.
L4Linux, however, is closely tied to L4/Fiasco and the L4 environment. For
us at Genode Labs, maintaining both a custom port of L4Linux for Genode
and L4/Fiasco by ourself in addition to developing Genode is unfeasible.
In contrast, the Pistachio kernel features more advanced options for
virtualization ([http://l4ka.org/projects/virtualization/afterburn/ - Afterburner]
and VT support) that we want to explore. Furthermore, there exists another
version of L4Linux called OKLinux for the OKL4 kernel developed at
[http://ok-labs.com - OK-Labs], which is very interesting as well.
Therefore, we decided against an ad-hoc solution and deferred this feature
to the next release. [http:/about/road-map - See our updated road map...]
Major new Features
##################
Genode on L4ka::Pistachio
=========================
From the very beginning, the base API of the Genode OS Framework was
designed for portability. We put a lot of effort into finding API
abstractions that are both implementable on a wide range of kernels and as
close to the hardware as possible to keep the abstraction overhead low. For
this reason, we developed the framework in parallel for the Linux kernel and
the L4/Fiasco kernel. To validate our claim that Genode is highly portable,
Julian Stecklina ported the framework to another member of the L4 family,
namely the [http://l4ka.org/projects/pistachio/ - L4ka::Pistachio kernel].
This high-performance kernel implements the latest official L4 API called
L4.x2 and has a number of advanced features such as multi-processor support
and virtualization support.
After Julian successfully created the first Pistachio version of Genode,
we successively refined his work and conducted further unifications among
the platform-dependent code for the different kernels. The result of this
effort is included in this release and comes in the form of the
'base-pistachio' source-code repository.
;Interesting technical notes:
* The IRQ handling on Pistachio is slightly different from L4/Fiasco.
On L4/Fiasco, an IRQ becomes unmasked only when the user-level IRQ
handler thread blocks for an IRQ by issuing an IPC call to the
kernel's corresponding IRQ thread. In contrast, Pistachio unmasks an
IRQ as soon as the user-level IRQ handler associates itself with an
IRQ. Thus, an IRQ message may occur not only when the user-level IRQ
handler blocks for any IRQ but anytime. In particular, IRQ messages
may interfere with the IRQ handler's IPC communication with other
threads. To ensure that IRQ messages do only occur when expecting
them, we lazily associate the IRQ handler thread to the IRQ the
first time we wait for an IRQ and issue an unmasking IPC call
subsequent times.
* Genode provides a mechanism for gracefully destructing threads that
are in a blocking state, for example waiting for an IPC message.
Such a thread may hold locks or other resources that would not
get properly freed when just killing the thread by force. Therefore,
Genode provides a way to issue the cancellation of a blocking
operation by another thread (e.g., the thread calling the destructor).
Once, a blocking operation got canceled, a C++ exception
('Blocking_canceled') is raised such the thread can fall back into
a defined state and then be destructed. On L4ka::Pistachio, we use
Pistachio's pager-exchange-registers feature in combination with
the user-defined UTCB handle for cancelling blocking operations and
detecting cancellations. The interesting code bits can be found in
'src/base/ipc/ipc.cc', 'src/base/lock/lock.cc',
'src/core/platform_thread.cc', and in the Pistachio-specific
timer-service implementation.
* During the refinement of the Pistachio version, we were able to further
generalize code that was previously specific for L4/Fiasco and
L4ka::Pistachio respectively. Now, the platform-specific code comprises
less than 3,000 lines of code (LOC) for L4/Pistachio, circa 2,000 LOC
for L4/Fiasco, and circa 1,000 LOC for Linux. Hence, we expect that
porting the framework to further kernels is possible at reasonable
engineering costs.
:Current limitations:
* The current version does not use superpages (4M mappings) because we
experienced problems with mapping 4K pages out of 4M pages. This is an
issue that we like to investigate further because using 4M mappings
would improve the boot time and reduce the kernel-memory usage.
* Currently, we use a simple sleeping spinlock for synchronization, which
is not optimal for several reasons. There are no fairness guarantees,
the spinning consumes CPU time, and threads that got blocked in the
contention case are woken up at the coarse granularity of the kernel's
timer tick, which is typically one millisecond.
* Nested RM sessions as introduced as an experimental feature in the
Genode release 8.11 are not yet supported.
:Further details:
You can find further technical details and usage instructions at our
dedicated [http://genode.org/community/wiki/GenodeOnL4kaPistachio - Wiki page].
Qt4 on Genode
=============
The minimalism of the Genode OS Framework with regard to its code
complexity raised the question of whether this framework is feasible
for hosting real-world applications and widely used runtime environments.
Christian Prochaska took the challenge to port Trolltech's Qt4 application
framework, which serves as the basis for the popular KDE desktop, to Genode.
Because Christian started his work more than a year ago at a time when no
C library was available on Genode, several intermediate steps were needed.
The first step was the integration of the Qt4 tools such as the meta-object
compiler (moc) and resource compiler properly into the our build systion.
With the tools in place, the Linux version of Genode came to an advantage.
In this environment, a Genode application is able to use glibc functionality.
So the problem of a missing C library could be deferred and Christian was
able to focus on interfacing Qt with the existing Genode services such as
the Nitpicker GUI sever. Next, the glibc dependencies were successively
replaced by custom implementations or simple dummy stubs. Thereby, all
needed functionalities such as timed semaphores and thread-local storage
had to be mapped to existing Genode API calls. Once, all glibc dependencies
had been dissolved, Qt could be compiled for the L4/Fiasco version.
Since a C library has become available in Genode 8.11, we were able to
replace Christian's intermediate stub codes with a real C library. We also
utilize recently added features of Genode such as its alarm framework to
simplify the Qt4 port. Furthermore, we were able to remove all
platform-specific bits such that the Qt4 port has now become completely
generic with regard to the underlying kernel. Qt4 can be executed on Linux,
L4/Fiasco, and L4ka::Pistachio without any changes. Figure [img/qt4_screenshot]
shows a screenshot of Qt's Tetrix example running side-by-side with native
Genode applications.
[image img/qt4_screenshot]
:Current state:
* The Qt4 port comes in the form of a source-code repository, which contains
all Qt source codes, and some example programs such as Tetrix. You can
download the Qt4 repository as a separate archive at the download page of
the Genode release 9.2. For the next release, we plan to separate the
Genode-specific parts from Qt original code and make the Genode-specific
parts a regular component of the Genode main line.
* The Qt4 port consists of Qt's Core library, GUI library, Script library,
XML library, and the UI tools library. Other libraries such as Webkit
are not ported yet.
* This first version of Qt4 on Genode is not to be considered as stable.
There are several known issues yet to be addressed. In particular,
the 'QEventDispatcher' is still work in progress and not fully stabilized.
* Because, we use to statically link programs, the binaries of Qt
applications are exceedingly large. For example the Tetrix binary is
100MB including debug information and 11MB in the stripped form. For
employing Qt on Genode at a larger scale, Genode should be enhanced with
[http://genode.org/community/wiki/SharedLibrarySupport - shared-library support].
Networking
==========
With Genode 8.11, we introduced the Device-Driver-Environment Kit (DDE Kit)
API, which is a C API specifically designed for implementing and porting
device drivers written in plain C. We have now complemented DDE Kit with an
environment for executing Linux device drivers natively on Genode. This
library is called 'dde_linux26' and contained in our new 'linux_drivers'
source-code repository. The environment consists of several parts, which
correspond to the different sub systems of the Linux kernel 2.6, such as
'arch', 'drivers', 'kernel'.
The first class of device-drivers supported by DDE Linux 2.6 is networking.
At the current stage, the DDE Linux network library comprises general
network-device infrastructure as well as an exemplary driver for the PCnet32
network device.
Based on this library, we have created a basic TCP/IP test utilizing the
uIP stack, which uses the DDE Linux network library as back end. The test
program implements a basic web server displaying uIP packet statistics.
When executed on Qemu, you can use your host's web browser to connect to
the web server running on Genode:
For booting Genode on L4/Fiasco with the web-server demo, use a GRUB
entry in your 'menu.lst' file as follows.
! title Genode: DDE Linux 2.6 NET on L4/Fiasco
! kernel /fiasco/bootstrap -maxmem=64 -modaddr=0x02000000
! module /fiasco/fiasco -nokd -serial -serial_esc
! module /fiasco/sigma0
! module /genode/core
! module /genode/init
! module /config
! module /genode/timer
! module /genode/pci_drv
! module /genode/test-dde_linux26_net
The first four lines are L4/Fiasco specific. When using L4ka::Pistachio,
the 'menu.lst' entry looks like this:
! title Genode: DDE Linux 2.6 NET on L4/Pistachio
! kernel /pistachio/kickstart
! module /pistachio/x86-kernel
! module /pistachio/sigma0
! module /genode/core
! module /genode/init
! module /config
! module /genode/timer
! module /genode/pci_drv
! module /genode/test-dde_linux26_net
The web-server test requires the PCI bus driver and the timer service.
Therefore, the 'config' file for Genode's init should have following
content:
! <config>
! <start>
! <filename>timer</filename>
! <ram_quota>512K</ram_quota>
! </start>
! <start>
! <filename>pci_drv</filename>
! <ram_quota>512K</ram_quota>
! </start>
! <start>
! <filename>test-dde_linux26_net</filename>
! <ram_quota>16M</ram_quota>
! </start>
! </config>
Now, its time to create an ISO image from all files specified in
the GRUB configuration. For this, the new utility 'tool/create_iso'
becomes handy. The ISO image can then be booted on Qemu using the
following arguments:
! qemu -m 64 -serial stdio -no-kqemu -cdrom <iso-image> \
! -net nic,model=pcnet -net user -redir tcp:5555::80
The '-redir' argument tells qemu to redirect TCP connections with
localhost:5555 to the guest OS at port 80. After having booted
up Genode on Qemu, you can use your host's web browser to access
the web server:
! firefox http://localhost:5555
:Notes about using the TAP version:
* Preparations
* You must be permitted to sudo and have installed the tunctl
utility. Under Debian/Ubuntu execute
! sudo apt-get install uml-utilities
* Create TAP device
! TAPDEV=$(sudo tunctl -b -u $USER)
! sudo /sbin/ifconfig $TAPDEV 10.0.0.1
* setup DHCP server on $TAPDEV and 10.0.0.0/8
* Run qemu
! qemu -m 64 -serial stdio -no-kqemu -cdrom dde.iso \
! -net nic,model=pcnet \
! -net tap,ifname=$TAPDEV,script=no,downscript=no
* Ping
* Cleanup
* Stop DHCP server
* Remove TAP device
! sudo tunctl -d $TAPDEV
Operating-system services and libraries
#######################################
C Runtime
=========
We have replaced the 'malloc' implementation of the original FreeBSD C
library with a custom implementation, which relies on Genode's 'Heap' as
allocator. The FreeBSD libc reserves a default memory pool of 1MB, which
is no problem on FreeBSD because virtual memory is backed lazily with
physical pages on demand. On Genode however, we immediately account the
allocated memory, which implicates high quota requirements even for
applications that use little memory. In contrast, Genode's heap allocates
and accounts its backing store in relatively small chunks of a few KB.
Therefore, the quota accounting for applications is much more in line with
the actual memory usage. Furthermore, our custom 'malloc' implementation
has the additional benefit of being thread safe.
* Added i386-specific parts of gen lib, in particular longjmp, setjmp.
Device-Driver-Environment Kit
=============================
* The DDE Kit uses our alarm framework (located in the 'os' repository) now
rather than its own event-scheduler implementation formerly called 'Tick'.
* We refined the DDE Kit API and reduced the number of custom types. For
example, we removed the custom 'dde_kit_lock_t' and using
'struct dde_kit_lock' instead, and replaced 'dde_kit_thread_t' with
'struct dde_kit_thread'.
Because of the apparent stabilization of the DDE Kit API, we have now added
this API to Genode's official API reference.
[http://genode.org/documentation/api/dde_kit_index - See the documentation of the DDE Kit API...]
PS/2 input driver
=================
We improved the PS/2 keyboard driver by adding missing scan-code translations
for the scan code set 1, in particular the cursor keys.
Applications
############
Launchpad configuration
=======================
Launchpad is a graphical application for interactively starting and killing
programs. It is used for the default demonstration of Genode. By default,
launchpad displays a preconfigured list of programs and their respective
default memory quotas. The user can tweak the memory quota for each entry
with mouse and then start a program by clicking on its name. As an
alternative to using the default list, you can now define the list manually
by supplying a configuration to Launchpad. The following example tells
launchpad to display a list of two launcher entries:
!<config>
! <launcher>
! <filename>sdl_pathfind</filename>
! <ram_quota>10M</ram_quota>
! </launcher>
! <launcher>
! <filename>liquid_fb</filename>
! <ram_quota>10M</ram_quota>
! </launcher>
!</config>
To use this configuration for a Launchpad started via init, you can simply
insert the launchpad configuration into the '<start>' node of the launchpad
entry in init's 'config' file.
Platform-specific changes
#########################
L4/Fiasco
=========
* Raise 'Blocking_canceled' exceptions on canceled IPC calls
32-bit Linux
============
* We continued dissolving the dependency of Genode from the glibc by using
a custom 'getenv' implementation used during process creation.
* By default, we compile now with '-nostdinc' and explicitly specify
'/usr/include' as include search directory only when needed. Previously,
a Genode application, which included a host include file by mistake, has
not raised any compilation error when compiled for the Linux version of
Genode. Now, all Genode platforms behave equally with regard to include
search directories.
* We enforce using the actual compiler's C++ support libraries rather than
the default libraries installed on the host.
Tools and build infrastructure
##############################
Official tool chain
===================
At the download section of our website, we used to provide a crosstool-based
tool chain as pre-compiled binaries. Since we got several requests about
how to build such a tool chain from scratch, we created custom utility for
downloading, building, and installing the official Genode tool chain. You
can find the utility at 'tool/tool_chain'. For usage instructions, just
start 'tool_chain' without arguments. Because this utility is a plain script,
you can follow and verify each step that we use for creating the Genode tool
chain. Currently, this official tool chain is based on binutils 2.18 and
gcc 4.2.4.
As an alternative to installing the tool chain from source, we also
provide pre-compiled binaries at the download section of our website.
[http://genode.org/download/tool-chain - Visit our tool-chain download website...]
For the Linux version of Genode, we still use the host's default gcc
as tool chain. This way, we spare the hassle of downloading and installing
a custom tool chain for somebody who wants to give Genode a quick try.
With this is mind, we have fixes several small issues with gcc 4.3.2:
* Fixed dependency generation for gcc-4.3.2. Older version of gcc used to
append a '.o' dependency at the target of '.d'-files. However, gcc-4.3.2
seems to handle the option '-MT' differently, resulting in a rule that
contains only the '.d' as target. Now, we explicitly specify both the
'.o' file and the '.d' file as target. Consequently, on older gcc
versions, the '.o' file appears twice but that is no problem.
* Fixed assembler issue with the 'fnstsw' instruction in the C library.
This instruction does not accept eax but only ax as argument.
Build-directory creation tool
=============================
We added a rule for creating a pre-configured build directory for the
Pistachio version to our build-directory creation tool
('tool/builddir/create_builddir'). Furthermore, we changed the default
build configuration such that the official Genode tool chain is used for
L4/Fiasco and L4ka::Pistachio.
Build system
============
* Improved clean rule - visit each target directory only once
* Stop the build process on the first error by default, for continuing
the build process depite of an error, you can use the '-i' argument
of make.
* Compiler flags can now be set specific for compiling C and C++ sources.
This is needed because both variants allow different sets of warning
options. The new variables are called 'CC_CXX_OPT' and 'CC_C_OPT'.
ISO image creation tool
=======================
We have created a convenient front end for 'genisoimage', which we
use for testing Genode on Qemu. You can find this ISO-image-creation
tool at 'tool/create_iso'. For usage instructions, simply start the
tool without arguments.

585
doc/release_notes-09-05.txt Normal file
View File

@@ -0,0 +1,585 @@
==============================================
Release notes for the Genode OS Framework 9.05
==============================================
Genode Labs
Shortly after including support for the L4ka::Pistachio kernel in the
previous release, the Genode version 9.05 gives a further evidence of
Genode's high portability by making the framework available on top of
the OKL4 kernel. This kernel is a commercial-grade microkernel
developed by [http://ok-labs.com - Open Kernel Labs]. In Section
[Supporting the OKL4 kernel as new base platform], we elaborate on the
new base platform and summarize the experiences made during our porting
work.
The previous Genode release was accompanied by a source-code archive containing
the initial version of Qt4 for Genode. Our approach is to make the Qt4
framework available for building Genode applications running natively on the
microkernel rather than within a virtualization environment. As advertised in
our [http://genode.org/about/road-map - road map], we have now seamlessly
integrated the Qt4 framework into our mainline source tree. Furthermore, we
have adapted our port to the Qt4 version 4.5.1. Section [Integration of Qt4
into the mainline repository] gives a rough overview of the changes and an
introduction on how to use the Qt4 framework with Genode.
The third major feature of the release is the addition of preliminary USB
support. We have been able to port major parts of Linux' USB infrastructure
to Genode using the DDE Kit introduced in autumn 2008. Section [USB support]
presents an overview about the pursued design and the current state of
implementation.
Section [OKLinux on Genode] outlines our ongoing efforts of running Linux
as a node in Genode's process tree.
Supporting the OKL4 kernel as new base platform
###############################################
The OKL4 kernel developed by Open Kernel Labs started as a fork of the
L4ka::Pistachio kernel. Whereas L4ka::Pistachio remained true to the L4
x.2 specification, OKL4 was subject of major API changes geared towards high
performance on the ARM architecture. OKL4 earned much fame for executing a
user-level variant of Linux (OKLinux) on top the microkernel, which turned out
to be faster than executing Linux natively on the ARM9 architecture. Even
though OKL4 is primary targeted at the ARM architecture, we wanted to go for
the x86 variant because of two reasons. First, there exists the just mentioned
user-level port of Linux for OKL4, which looks like an attractive way to execute
Linux on Genode once Genode runs on OKL4. Second, we think that distributing
Genode in the form of ISO images bootable on plain PC hardware is the best
way to reach the OS community. Therefore, we decided to use OKL4 version 2.1 as
the base for our work. In contrast to later releases, this version supports
both x86 and ARM. The following section reviews the unique features of the
OKL4 kernel from our perspective.
OKL4 viewed from the angle of a Genode developer
================================================
On the kernel-API level, OKL4 has several interesting properties that had been
both welcome and challenging. We want to highlight the following points:
In contrast to prior L4 kernels, OKL4 has *removed wall-clock timeouts* from
the kernel interface. On L4, timeouts were used as arguments for for blocking
IPC operations serving two purposes. First, specifying IPC timeouts allowed the
IPC caller for regaining control over the blocking thread after the specified
time elapsed. This is considered as important in the case that the called
thread misbehaves and never answers the call. However, the problem of choosing
an appropriate timeout was never properly resolved. When dimensioning the
timeout too small, the called thread may miss the timeout just because it had
no chance to be selected by the scheduler in time. Such timeouts rely on the
presumption that there is low load on the system. On the other hand, when
dimensioning the timeout too high, the system will become sluggish when the
called thread misbehaves. For example, a simple GUI server may want to send
input events to its clients with a timeout to be robust against misbehaving
clients that never wait for events. When choosing a timeout too small, chances
are high that an event will occur at a time when the receiver is handling a
previous event. The timeout would trigger and the event would get lost. When
choosing the timeout too large, say 1 second, any misbehaving client could make
the GUI server freeze for 1 second. Therefore, timeouts for regaining control
over a blocked thread seem to be a bad idea. So we welcome their absence in
OKL4. The second use of timeouts is their use as user-level time source. On L4,
sleep is typically implemented as a blocking IPC with a timeout set to the
sleep value. For this purpose, a system built on top of OKL4 has to employ a
user level device driver accessing a timer device. In Genode, we already have a
timer service for this purpose. So we won't miss timeouts at all.
Classical L4 kernels provide two variants of *synchronous IPC*. So called long
IPC could copy any amount of memory from the sending to the receiving address
space. This is complicated operation because either communication partner may
specify communication buffers that contain unmapped pages. Hence, page faults
may occur during long-IPC operations. On L4, page faults, in turn, are handled
by the user land. Not until a user-level pager thread resolves the page fault
by establishing a mapping at the faulting address, the kernel can proceed the
IPC operation. This sounds pretty complicated, and it is. The second IPC
variant is called short IPC. It constrains the transferable payload to CPU
registers. Hence, these IPC operations should only be used for messages with a
payload of a maximum of 2 machine words. Because short IPCs are never touching
user-level memory pages, no page faults can occur.
On OKL4, there is only one IPC operation, which copies payload from the
sender's user-level thread-control block (UTCB) to the receiver's UTCB. An
UTCB is an always-mapped memory region. Hence no page faults can occur during
IPC operations. On Genode, the UTCB size of 256 bytes may become a limitation
when RPC messages get large. For example, session requests may include large
session-argument strings specifying session-constructor arguments. Current
services rely only on a few arguments so the size limitation is not an
apparent problem. But that may change for future services. Furthermore, in
contrast to L4 x.2, OKL4 does not allow for transferring payload other than
plain data. In particular, OKL4 does not support the transfer of memory
mappings via IPC. Removing memory mappings from the IPC operation is a very
good idea. On Genode, only roottask (core) establishes mappings and shared
memory is implemented as a user-level protocol (data spaces). There is no need
to allow arbitrary processes to establish memory mapping via IPC.
The *boot procedure* of OKL4 largely differs from other L4 kernels. This is
attributed to Open Kernel Labs' focus on embedded systems, which mostly rely on
single-image boot loading. OKL4 employs a tool (elfweaver) for creating a
bootable image from a bunch of files and an XML configuration file. Among the
declarations about which processes to be loaded and which policies to enforce,
the configuration file contains platform parameters such as the amount of
physical memory of the machine. This static approach to configure a system is
certainly useful for embedded systems but PC hardware uses to vary a lot. In
this case, evaluating boot-time memory descriptors would be the preferred
solution.
OKL4 introduces kernel support for *user-level synchronization*. Prior L4
kernels facilitated user-level synchronization through a combination of
synchronous IPC operations with either priorities or delayed preemption.
OKL4's mutexes can make the life in the user land much easier. However, we have
not looked into OKL4 mutexes yet.
There does not exist a recursive *map operation* as the source operand of the
map operation is a physical memory descriptor rather than a virtual address in
the mapper's address space. Consequently, this design eliminates the need for
having a recursive unmap operation and thereby, the need to maintain a mapping
data base in the kernel. This is cool because Genode keeps track of the
mappings at the user level anyway (within core). From our perspective, there is
no need to maintain mapping relationships in the kernel. Removing the mapping
database effectively discards a lot of much-discussed problems about how to
manage the mapping database in a clever way.
There exists *no root memory manager* (sigma0). Because the map operation
takes a physical memory descriptor as argument instead of a virtual address
in the mapper's address space. The mapper does not need to have the mapped
page locally mapped within its own address space. In fact, core (as the only
mapper in a Genode system) does only have very little memory mapped locally.
This design accelerates the boot time because there is no need to map each
physical page in core at startup as performed when running core on the other
L4 kernels.
These differences of OKL4 compared with the microkernels already supported
by Genode posed a number of interesting challenges and opportunities. We have
thoroughly documented the process in
[http://genode.org/documentation/articles/genode-on-okl4 - Bringing Genode to OKL4].
Usage
=====
For using Genode with OKL4, please refer to the following dedicated Wiki page:
:[http://genode.org/community/wiki/GenodeOnOKL4 - Genode on OKL4]:
Wiki page about building and using Genode with the OKL4 kernel.
Limitations of the current implementation
=========================================
The current implementation is able to execute the complete Genode demonstration
scenario on OKL4. This means, we can build and destroy arbitrary trees of
processes, have all the needed infrastructure in place to execute user-level
device drivers such as VESA and PS/2, perform inter-process communication
via RPC and shared memory, and have all basic framework API functions available.
We regard the current state as the first functional version. However, there are
the following points that need improvement and are subject to our future work.
:Incomplete timer driver:
On x86, the timer driver should program the PIT to dispatch sleep requests.
However, the I/O ports of the PIT can only by made available to one party in
the system (which naturally would be the timer driver). Unfortunately, there
are some VESA BIOSes around, which try using the PIT directly. The current
version of our VESA driver does not virtualize these accesses. It rather
tries to gain direct access to the I/O ports from core. This would not work
if the timer already uses this device resource. Our plan is to supplement
our VESA driver with a virtual PIT that uses the timer service as back end.
Then we can safely use the PIT by the timer driver.
:Signalling framework not yet implemented:
We have not yet implemented Genode's API for asynchronous notifications
in the OKL4 version. In fact, the goal of the initial version of the
OKL4 support was running the default demonstration scenario, which does
not rely on signals. The second and more technical reason is that we
consider exploiting OKL4's event mechanism for implementing the signalling
API but have not finalized the design. The generic implementation as used
on the other platforms cannot be used on OKL4 because this implementation
utilizes one helper thread per signal transmitter. Within core, each RM
session is a potential signal transmitter, which means that we need to
create a helper thread per process. Unfortunately, by default, OKL4
limits the number of threads within roottask (core) to only 8 threads,
which would impose a severe limit on the number of processes we could
run on OKL4.
:OKL4's kernel mutexes yet to be used:
We have not yet explored the use of mutexes provided by the OKL4 kernel
for implementing Genode synchronization APIs but we rather rely on a
yielding spin lock for now. This has a number of drawbacks such as high
wake-up latencies in the contention case (depending on the scheduling
time slice), no support for priorities, and no fairness. Although it
is a simple and robust solution to start with, we plan to facilitate
the OKL4 kernel feature with our upcoming work.
:Overly simplistic UTCB allocation:
Right now, we allocate a fixed amount of 32 UTCBs per address space and
thereby limit the maximum number of threads per process. In the future,
this limit should be made configurable.
:Managed dataspaces not yet supported:
The support of managed dataspaces relies on the signal API, which is
not yet available for OKL4.
:Message buffers are limited to 256 bytes:
Because OKL4 performs message-based inter-process communication by
copying data between the UTCBs of the communicating threads, the
UTCB size constaints the maximum message size. Therefore, message
must not exceed 256 bytes. This is not a huge problem for the currently
available Genode programs but we can imagine session argument-lists
to become larger in the future.
:Advanced thread functions are incomplete:
Thread functions such as querying registers of remote threads are not yet
implemented.
Integration of Qt4 into the mainline repository
###############################################
Qt4 is a tool kit for developing platform-independent applications. It
comprises a complete platform-abstraction layer and a rich GUI tool kit
widely used for commercial and open-source applications. It is particularly
known as the technical foundation of the KDE project. The previous Genode
release was accompanied by a snapshot of our initial port of Qt4 to Genode. For
the current release, we have turned this proof-of-concept implementation into a
properly integrated part of the Genode mainline development. This enables Qt4
applications to be executed natively on the full range of kernels supported by
Genode.
Usage
=====
We complemented Genode's source tree with the new 'qt4' source-code repository,
which contains the Genode-specific parts of the Qt4 framework. The most
portions for the Qt4 framework are used unmodified and thereby have not been
made part of the Genode source tree. Instead, we fetch the original Qt4 source
code from Trolltech's FTP server. This way, our source tree remains tidy and
neat.
For using Qt4 for your Genode applications, you first need to download and
prepare the original Qt4 source codes and build a few Qt4 tools such as the
meta-object compiler and the resource compiler. The makefile found in the
top-level directory of the 'qt4' repository automates this task:
! make prepare
To include the 'qt4' repository into the Genode build process, just add the
'qt4' directory to the 'REPOSITORIES' declaration of the 'etc/build.conf' file
within your build directory. Make sure that the repositories 'demo' and 'libc'
are included as well. The 'qt4' repository comes with a couple of demo applications.
The 'qt_launchpad' is especially interesting because it makes use of both the
Qt4 framework and the Genode framework in one application.
Features and limitations
========================
The Qt4 port comprises Qt's Core library, GUI library, Script library, XML
library, and the UI tools library.
For using Qt4 on the Linux version of Genode, we recommend using the Genode
tool chain rather than your host's tool chain. Qt4 makes use of a lot of libc
functionality, supplied by Genode's 'libc' repository. However, on Linux we
still link against your host's libc. This becomes a problem if your host
compiler's C++ support code references libc functionality that is not part of
Genode's libc. Thereby the linker will silently create references to glibc
symbols, making both libraries collide. So if using Qt4, we recommend using the
Genode tool chain:
:[http://genode.org/download/tool-chain]:
Information about downloading and using the Genode tool chain
USB support
###########
This release introduces the first fragments of USB support to Genode, taking
the USB human-interface device (HID) class as starting point. With this work,
we follow our approach of reusing unmodified Linux device drivers executed
within a device-driver environment called DDE Linux. In the previous release,
we already utilized this approach for realizing basic networking on Genode.
With this release, we complement DDE Linux with support required by USB
drivers. We are grateful for being able to base our implementation on the
excellent foundation laid by Dirk Vogt. He described his work in
[http://os.inf.tu-dresden.de/papers_ps/vogt-beleg.pdf - USB for the L4 environment].
For USB HID support, we added the Linux USB and input subsystems to the DDE
Linux 2.6 framework. Besides the 'dde_linux26/net.h' API for network drivers
added in Genode 9.02, the current version also includes APIs for input
('dde_linux26/input.h') and USB ('dde_linux26/usb.h'). We intend these
interfaces to mature towards generic driver-library APIs in the future. For
example, BSD-based drivers shall transparently provide the same functionality
as the current Linux drivers, which permits the simple reuse of driver server
implementations.
[image img/usb_current]
Image [img/usb_current] illustrates the current implementation of the USB-based
human-interface device (HID) driver. In this monolithic setup, all parts of the
USB stack and the device API are executed within one address space. These parts
are
* Input server glue code
* HID driver and input subsystem
* Core functions for management of USB request buffers (URBs),
attached devices, and registered drivers
* Host controller drivers for UHCI, OHCI, and EHCI
[image img/usb_aspired]
We regard this as an intermediate step towards our goal to decompose the USB
stack. Image [img/usb_aspired] shows our aspired design. In this design, the
USB server and one or more USB gadget drivers run in dedicated address spaces.
The USB server provides two interfaces called USB session interface and USB
device interface. A USB session interface corresponds to a virtual root hub,
from which USB devices can be queried. The client of the USB session interface
is usually an USB gadget driver that uses the USB device interface. Because
this interface is used for transferring the actual payload at a potentially
high bandwidth, it is based on shared memory and signals. The USB server
consists of the following components:
* USB server glue code
* Virtual USB device driver managing all attached devices
* Core functions including hardware hub management
* Host controller drivers
The USB server presents a virtual USB hub to each USB gadget driver. Such
a driver consists of:
* Device interface, e.g., input server glue code
* Gadget driver, e.g., HID driver and input subsystem
* Core functions
* Virtual host controller
* USB client glue code
The HID driver uses the USB session API to monitor ports of its virtual root
hub and submit URBs to attached devices. The session interface facilitates the
signalling framework for event notification and a shared-memory dataspace for
URB transmission.
The 'os' repository already contains the USB session and USB device interfaces.
However, the decomposition is not yet in a functional state.
:Current limitations:
The current monolithic implementation of the USB HID service can already be
used as a replacement of the PS/2 driver. However, both drivers cannot be used
at the same time, yet. To enable the use of both USB HID and PS/2, we plan to
create a further component that merges multiple streams of input events and
passes the result to the GUI server.
OKLinux on Genode
#################
According to our road map, we pursued the goal to run Linux as a node in
Genode's process tree. We explored two approaches:
:Reanimating the Afterburner project conducted by the [http://l4ka.org - L4Ka group]:
This approach is the result of the L4Ka groups's long-year experience with
manually supporting L4Linux on top of the L4ka::Pistachio kernel. Because of
the high costs of maintaining the paravirtualized Linux kernel, a
semiautomatic paravirtualization technique was created. According to the
impressive results presented in
[http://l4ka.org/publications/paper.php?docid=2025 - Pre-Virtualization: Soft Layering for Virtual Machines],
this approach is able to drastically reduce maintenance costs while retaining
good performance. Furthermore, the approach was applied not only to Linux
running on the L4 kernel but also for using Xen or Linux as underlying
host operating systems.
:Porting the OKL4-specific version of L4Linux to Genode:
Open Kernel Labs maintain a custom version of L4Linux that runs on OKL4. This
version is mostly referred to as OKLinux aka Wombat. Since Genode can now use OKL4
as base platform, the reuse of OKLinux in combination with Genode has become
a feasible option.
Both approaches have pros and cons. Whereas Afterburner is a intriguing
approach, this project seems to be stalled. It relies on a rather old tool
chain, and recent Linux features such as thread-local storage support are not
considered, yet. To pick up this solution for Genode will require us to fully
understand the mechanisms and the code. So we consider this as a mid-term
solution. In short term, running OKLinux on Genode is more feasible. We were
already able to create a prototype version of OKLinux running on Genode. This
version starts up the kernel including all Linux kernel threads, mounts the
boot partition supplied as memory image, and starts the init process. The
engineering costs had been rather low. We replaced the Iguana user land
libraries as originally used by Wombat by a Genode-specific reimplementation to
keep our manual adaptions of the Linux kernel code as small as possible.
Our custom reimplementation of the needed Iguana APIs consists of less than
1,000 lines of code (SLOC). The diff for our changes to the OKLinux kernel code
comprises less than 1,000 lines. We plan to make a snapshot of this prototype
publicly available soon.
Operating-system services and libraries
#######################################
Nitpicker GUI server
====================
We optimized the performance of the 'refresh' call, which updates all views of
a session, which display a given buffer portion. The new implementation restricts
the redraw operations to the fragment of each view that displays the specified
buffer portion. The performance improvement becomes most visible when updating
only small parts of a buffer.
USB session interface
=====================
Genode's emerging USB support introduces two new interfaces to the 'os' repository,
which are called USB session and USB device.
An _USB_session_ is a virtual USB bus with just one root hub with 'MAX_PORTS'
downstream ports. The client of such as session submits USB request blocks
(URBs) and is, in turn, informed about port changes on the root hub as well as
request completion. Connected USB devices can be referenced by USB device
capabilities and are associated with one port at the virtual root hub on
server side.
An _USB_device_ references a hardware device connected to a virtual USB bus's
root hub. The device capability enables the client to send USB request
blocks to the hardware device.
Input interface
===============
We updated the key codes of the input interface in response to recent changes
of Linux' 'dev/event' definitions.
VESA driver
===========
Until now, there existed different processes that tried to access the PCI bus
via I/O ports, in particular the VESA framebuffer driver and the PCI bus
driver.
Since core enforces that each I/O port can only be assigned exclusively to one
component in the system, multiple processes that need access to the same I/O
ports cannot run at the same time. For our default demonstration scenario, we
had been able to allow the VESA driver to use the PCI I/O ports because nobody
else needed them. However, our growing base of device drivers relies on the
PCI bus driver. To be able to use the VESA driver together with other drivers,
we virtualized the access to the PCI bus from within the VESA driver.
Our current PCI virtualization code is pretty limited. The VESA driver sees a
virtual PCI bus with only the VGA card attached. For now, we only allow reading
the PCI configuration space of this device, but not writing to it. Apparently,
this simple approach is sufficient to run the VESA BIOS of Qemu. However, other
VESA BIOS implementations may need further access to the PCI device's
configuration space. For example, for querying the size of a PCI resource,
write access to base address registers is required. In such an event, the VESA
driver will print a message about the missing virtualization feature:
! writing data register not supported
If you see such a message, we are very interested to see your log output such
that we can enhance our PCI virtualization code as needed. Please contact us!
Base framework
##############
In the process of bringing Genode to the OKL4 kernel, we have generalized much
of former platform-specific code:
* The initialization of C++ exception handling has now become part of the
generic 'cxx' library in the 'base' repository. All platforms except
Linux are using this generic library now.
* The 'server' library used to contain a platform-specific part that
implemented the 'manage' function of a 'Server_entrypoint'. The
generalized version of this library is now being used on all platforms
other than Linux.
* We unified core-internal interfaces and their implementations such as
'Dataspace_component', 'Cap_session_component', 'Rm_session_component',
and 'Irq_session_component'. The result has become part of the 'base'
repository.
* On OKL4, threads need to execute small startup code for querying their
own thread IDs. Therefore, we have extended the 'Thread_base' interface
with a platform-specific hook function called '_thread_bootstrap'.
* The types defined in 'base/native_types.h' had been complemented by a
new 'Native_thread_id' type. This type is exclusively used by core and the
framework libraries. For using the Genode API, this type is meaningless.
* For the 64bit support, we slightly refined the interfaces of some utility
template functions in 'util/misc_math.h'. Furthermore, parts of the generic
marshalling code of the IPC framework needed refinement, but no API changes
were needed.
Linux-specific changes
######################
Adaptation to 64 bit
====================
Because most Genode developers tend to work with the Linux version of Genode,
supporting 64-bit Linux becomes increasingly important. With the current release,
we start to officially support 64-bit Linux as base platform. This comes
along with the following changes:
* We replaced the 'spec-x86.mk' file with new 'spec-x86_32.mk' and 'spec-x86_64.mk'
files. The default version of 'base-linux/etc/specs.conf' automatically
chooses the right spec file according to the output of 'uname -m'. Therefore,
output of the build processes matches your host architecture. This behaviour
can be changed by placing a customized 'spec.conf' file in your build directory's
'etc/' subdirectory.
* We added type definitions for 64-bit-specific fixed-size integers in the form
of a 64-bit-specific 'fixed_stdint.h' file.
* Because using 64 bit instead of 32 bit changes the payload size of RPC
messages, we had to adjust several message buffers such as 'Ram_session_client'
and 'Input::Session_client', and adapted the used stack sizes.
* Towards the goal of completely dissolving Genode's dependency on the Linux' glibc,
we implemented custom system-call bindings. Apparently, Linux' syscall interface
differs between 32 bit and 64 bit. For example, the 32-bit version handles
all socket-related calls via a compound 'socketcall' whereas the 64-bit
version uses distinct syscalls. Another difference is the handling of the
'mmap' syscall and different behaviour of 'tkill'. The latter problem was
resolved by replacing 'tkill' with 'tgkill' and setting the thread-group
argument of the corresponding PID. Therefore, a 'Native_thread_id' on Linux
now comprises both the TID and the PID.
* The 'Platform_env' on Linux contains a local implementation of the 'Rm_session'
interface, which uses 'mmap' to attach dataspaces to the process' address
space and maintains the region list in the form of a static array. This array
was dimensioned to 256 entries, which constrained the maximum amount of
usable memory when allocating a lot of small blocks via Genode's heap. Since
the heap allocates backing store at the granularity of only 16KB, the worst
case for reaching this limit was about 4MB. This was OK for our simple test
applications. But for using Qt4, in particular on 64 bit, this has become a
serious limitation. For now, we increased the region limit to 4096 and plan
to replace the static array with a dynamically growing data structure.
Furthermore, we made the heap granularity depend on the actual machine-word
size. Therefore, the heap allocates its backing store in 32KB blocks when
running on 64 bit.
Debugging hooks
===============
On Linux, we use gdb for debugging Genode. This is feasible as long as the
targeted process is running. However, during low-level debugging, we had the
recurring problem of a thread dying shortly after starting up. So we added a hook
for halting a thread at the startup in order to be able to attach gdb to the
thread before it dies. This simple hook lets the thread wait for a key press by
directly calling the 'read' syscall. We also added a simple debug facility for
printing debug messages bypassing Genode's LOG service by calling the 'write'
syscall directly. Both hooks are now part of the Linux version of the 'env'
library (see 'base-linux/src/base/env/debug.cc'). Note that these hooks are not
part of the Genode API. There exists no header file.

573
doc/release_notes-09-08.txt Normal file
View File

@@ -0,0 +1,573 @@
==============================================
Release notes for the Genode OS Framework 9.08
==============================================
Genode Labs
Whereas the previous releases were focused on adding features to the framework,
the overall theme for the current release 9.08 was refinement. We took the
chance to revisit several parts of the framework that we considered as
interim solutions, and replaced them with solid and hopefully long-lasting
implementations. Specifically, we introduce a new lock implementation, a new
timer service, a platform-independent signalling mechanism, a completely
reworked startup code for all platforms, and thread-local storage support.
Even though some of the changes touches fundamental mechanisms, we managed
to keep the actual Genode API almost unmodified.
With regard to features, the release introduces initial support for dynamic
linking, a core extension to enable a user-level variant of Linux to run on the
OKL4 version of Genode, and support for super pages and write-combined I/O
memory access on featured L4 platforms.
The most significant change for the Genode Linux version is the grand unification with
the other base platforms. Now, the Linux version shares the same linker script
and most of the startup code with the supported L4 platforms. Thanks to our
evolved system-call bindings, we were further able to completely dissolve
Genode's dependency from Linux's glibc. Thereby, the Linux version of Genode is
on the track to become one of the lowest-complexity (in terms of source-code
complexity) Linux-kernel-based OSes available.
Base framework
##############
New unified lock implementation
===============================
Since the first Genode release one year ago, the lock implementation had been
a known weak spot. To keep things simple, we employed a yielding spinlock
as basic synchronization primitive. All other thread-synchronization
mechanisms such as semaphores were based on this lock. In principle, the
yielding spinlock used to look like this:
! class Lock {
! private:
! enum Lock_variable { UNLOCKED, LOCKED };
! Lock_variable _lock_variable;
!
! public:
! void lock() {
! while (!cmpxchg(&_lock_variable, UNLOCKED, LOCKED))
! yield_cpu_time();
! }
!
! void Lock::unlock() { _lock_variable = UNLOCKED; }
! }
The compare-exchange is an atomic operation that compares the current value
of '_lock_variable' to the value 'UNLOCKED', and, if equal, replaces the
value by 'LOCKED'. If this operation succeeds, 'cmpxchg' returns true, which
means that the lock acquisition succeeded. Otherwise, we know that the lock
is already owned by someone else, so we yield the CPU time to another thread.
Besides the obvious simplicity of this solution, it does require minimal
CPU time in the non-contention case, which we considered to be the common case. In
the contention case however, this implementation has a number of drawbacks.
First, the lock is not fair, one thread may be able to grab and release the
lock a number of times before another thread has the chance to be
scheduled at the right time to proceed with the lock acquisition if the lock
is free. Second, the lock does not block the acquiring thread but lets it
actively spin. This behavior consumes CPU time and slows down other threads that
do real work. Furthermore, this lock is incompatible with the use of thread
priorities. If the lock is owned by a low-priority thread and a high-priority
thread tries to acquire a lock, the high-priority thread keeps being active
after calling 'yield_cpu_time()'. Therefore the lock owner starves and has no
chance to release the lock. This effect can be partially alleviated by replacing
'yield_cpu_time()' by a sleep function but this work-around implies higher
wake-up latencies.
Because we regarded this yielding spinlock as an intermediate solution since the
first release, we are happy to introduce a completely new implementation now.
The new implementation is based on a wait queue of lock applicants that are
trying to acquire the lock. If a thread detects that the lock is already
owned by another thread (lock holder), it adds itself into the wait queue
of the lock and calls a blocking system call. When the lock owner releases
the lock, it wakes up the next member of the lock's wait queue.
In the non-contention case, the lock remains as cheap as the yielding
spinlock. Because the new lock employs a fifo wait queue, the lock guarantees
fairness in the contention case. The implementation has two interesting points
worth noting. In order to make the wait-queue operations thread safe, we use a simple
spinlock within the lock for protecting the wait queue. In practice, we
measured that there is almost never contention for this spin lock as two
threads would need to acquire the lock at exactly the same time. Nevertheless,
the lock remains safe even for this case. Thanks to the use of the additional spinlock within
the lock, the lock implementation is extremely simple. The seconds interesting
aspect is the base mechanism for blocking and waking up threads such
that there is no race between detecting contention and blocking.
On Linux, we use 'sleep' for blocking and 'SIGUSR1' to cancel the sleep operation.
Because Linux delivers signals to threads at kernel entry,
the wake-up signal gets reliably delivered even if it occurs prior
thread blocking. On OKL4 and Pistachio, we use the exchange-registers
('exregs') system call for both blocking and waking up threads. Because 'exregs'
returns the previous thread state, the sender of the wake-up
signal can detect if the targeted thread is already in a blocking state.
If not, it helps the thread to enter the blocking state by a thread-switch
and then repeats the wake-up. Unfortunately, Fiasco does not support the
reporting of the previous thread state as exregs return value. On this kernel,
we have to stick with the yielding spinlock.
New Platform-independent signalling mechanism
=============================================
The release 8.11 introduced an API for asynchronous notifications. Until
recently, however, we have not used this API to a large extend because it
was not supported on all platforms (in particular OKL4) and its implementation
was pretty heavy-weight. Until now signalling required one additional thread for each signal
transmitter and each signal receiver. The current release introduces a
completely platform-independent light-weight (in terms of the use of
threads) signalling mechanism based on a new core service called SIGNAL.
A SIGNAL session can be used to allocate multiple signal receivers, each
represented by a unique signal-receiver capability. Via such a capability,
signals can be submitted to the receiver's session. The owner of a SIGNAL
session can receive signals submitted to the receivers of this session
by calling the blocking 'wait_for_signal' function. Based on this simple
mechanism, we have been able to reimplement Genode's signal API. Each
process creates one SIGNAL session at core and owns a dedicated thread
that blocks for signals submitted to any receiver allocated by the process.
Once, the signal thread receives a signal from core, it determines
the local signal-receiver context and dispatches the signal accordingly.
The new implementation of the signal API required a small refinement.
The original version allowed the specification of an opaque argument
at the creation time of a signal receiver, which had been delivered with
each signal submitted to the respective receiver. The new version replaces
this opaque argument with a C++ class called 'Signal_context'. This allows
for a more object-oriented use of the signal API.
Generic support for thread-local storage
========================================
Throughout Genode we avoid relying on thread-local storage (TLS) and, in fact,
we had not needed such a feature while creating software solely using the
framework. However, when porting existing code to Genode, in particular Linux
device drivers and Qt-based applications, the need for TLS arises. For such
cases, we have now extended Genode's 'Thread' class with generic TLS
support. The static function 'Thread_base::myself()' returns a pointer to the
'Thread_base' object of the calling thread, which may be casted to a inherited
thread type (holding TLS information) as needed.
The 'Thread_base' object is looked up by using the current stack pointer
as key into an AVL tree of registered stacks. Hence, the lookup traverses a
plain data structure and does not rely on platform-dependent CPU features
(such as 'gs' segment-register TLS lookups on Linux).
Even though, Genode does provide a mechanism for TLS, we strongly discourage
the use of this feature when creating new code with the Genode API. A clean
C++ program never has to rely on side effects bypassing the programming
language. Instead, all context information needed by a function to operate,
should be passed to the function as arguments.
Core extensions to run Linux on top of Genode on OKL4
#####################################################
As announced on our road map, we are working on bringing a user-level variant
of the Linux kernel to Genode. During this release cycle, we focused on
enabling OKLinux aka Wombat to run on top of Genode. To run Wombat on Genode we
had to implement glue code between the wombat kernel code and the Genode API,
and slightly extend the PD service of core.
The PD-service extension is a great show case for implementing inheritance
of RPC interfaces on Genode. The extended PD-session interface resides
in 'base-okl4/include/okl4_pd_session' and provides the following additional
functions:
! Okl4::L4SpaceId_t space_id();
! void space_pager(Thread_capability);
The 'space_id' function returns the L4 address-space ID corresponding to
the PD session. The 'space_pager' function can be used to set the
protection domain as pager and exception handler for the specified
thread. This function is used by the Linux kernel to register itself
as pager and exception handler for all Linux user processes.
In addition to the actual porting work, we elaborated on replacing the original
priority-based synchronization scheme with a different synchronization mechanism
based on OKL4's thread suspend/resume feature and Genode locks. This way, all
Linux threads and user processes run at the same priority as normal Genode
processes, which improves the overall (best-effort) performance and makes
Linux robust against starvation in the presence of a Genode process that is
active all the time.
At the current stage, we are able to successfully boot OKLinux on Genode and
start the X Window System. The graphics output and user input are realized
via custom stub drivers that use Genode's input and frame-buffer interfaces
as back ends.
We consider the current version as a proof of concept. It is not yet included
in the official release but we plan to make it a regular part of the official
Genode distribution with the next release.
Preliminary shared-library support
##################################
Our Qt4 port made the need for dynamically linked binaries more than evident.
Statically linked programs using the Qt4 library tend to grow far beyond 10MB
of stripped binary size. To promote the practical use of Qt4 on Genode, we
ported the dynamic linker from FreeBSD (part of 'libexec') to Genode.
The port consists of three parts
# Building the 'ldso' binary on Genode, using Genode's parent interface to
gain access to shared libraries and use Genode's address-space management
facilities to construct the address space of the dynamically loaded program.
# Adding support for the detection of dynamically linked binaries, the starting
of 'ldso' in the presence of a dynamically linked binary, and passing the
program's binary image to 'ldso'.
# Adding support for building shared libraries and dynamically linked
programs to the Genode build system.
At the current stage, we have completed the first two steps and are able to
successfully load and run dynamically linked Qt4 applications. Thanks to
dynamic linking, the binary size of Qt4 programs drops by an order of
magnitude. Apparently, the use of shared qt libraries already pays off when
using only two Qt4 applications.
You can find our port of 'ldso' in the separate 'ldso' repository. We will
finalize the build-system integration in the next weeks and plan to support
dynamic linking as regular feature as part of the 'os' repository with the next
release.
Operating-system services and libraries
#######################################
Improved handling of XML configuration data
===========================================
Genode allows for configuring a whole process tree via a single configuration
file. Core provides the file named 'config' as a ROM-session dataspace to the
init process. Init attaches the dataspace into its own address space and
reads the configuration data via a simple XML parser. The XML parser takes
a null-terminated string as input and provides functions for traversing the
XML tree. This procedure, however, is a bit flawed because init cannot
expect XML data provided as a dataspace to be null terminated. On most platforms,
this was no problem so far because boot modules, as provided by core's ROM
service, used to be padded with zeros. However, there are platforms, in particular
OKL4, that do not initialize the padding space between boot modules. In this
case, the actual XML data is followed by arbitrary bits but possibly no null
termination. Furthermore, there exists the corner case of using a config
file with a size of a multiple of 4096 bytes. In this case, the null termination
would be expected just at the beginning of the page beyond the dataspace.
There are two possible solutions for this problem: copying the content of
the config dataspace to a freshly allocated RAM dataspace and appending the
null termination, or passing a size-limit of the XML data to the XML parser.
We went for the latter solution to avoid the memory overhead of copying
configuration data just for appending the null termination. Making the XML
parser to respect a string-length boundary involved the following changes:
* The 'strncpy' function had to be made robust against source strings that are not
null-terminated. Strictly speaking, passing a source buffer without
null-termination violates the function interface because, by definition,
'src' is a string, which should always be null-terminated. The 'size'
argument usually refers to the bound of the 'dst' buffer. However, in our
use case, for the XML parser, the source string may not be properly terminated.
In this case, we want to ensure that the function does not read any characters
beyond 'src + size'.
* Enhanced 'ascii_to_ulong' function to accept an optional size-limitation
argument
* Added support for size-limited tokens in 'base/include/util/token.h'
* Added support for constructing an XML node from a size-limited string
* Adapted init to restrict the size of the config XML node to the file size
of the config file
Nitpicker GUI server
====================
* Avoid superfluous calls of 'framebuffer.refresh()' to improve the overall
performance
* Fixed stacking of views behind all others, but in front of the background.
This problem occurred when seamlessly running another window system as
Nitpicker client.
Misc
====
:Alarm framework:
Added 'next_deadline()' function to the alarm framework. This function is
used by the timer server to program the next one-shot timer interrupt,
depending on the scheduled timeouts.
:DDE Kit:
* Implemented 'dde_kit_thread_usleep()' and 'dde_kit_thread_nsleep()'
* Removed unused/useless 'dde_kit_init_threads()' function
:Qt4:
Added support for 'QProcess'. This class can be used to start Genode
applications from within Qt applications in a Qt4-compatible way.
Device drivers
##############
New single-threaded timer service
=================================
With the OKL4 support added with the previous release, the need for a new timer
service emerged. In contrast to the other supported kernels, OKL4 imposed two
restrictions, which made the old implementation unusable:
* The kernel interface of OKL4 does not provide a time source. The kernel
uses a APIC timer internally to implement preemptive scheduling but, in
contrast to other L4 kernels that support IPC timeouts, OKL4 does not
expose wall-clock time to the user land. Therefore, the user land has to
provide a timer driver that programs a hardware timer, handles timer
interrupts, and makes the time source available to multiple clients.
* OKL4 restricts the number of threads per address space according to a
global configuration value. By default, the current Genode version set
this value to 32. The old version of the timer service, however, employed
one thread for each timer client. So the number of timer clients was
severely limited.
Motivated by these observations, we created a completely new timer service that
dispatches all clients with a single thread and also supports different time
sources as back ends. For example, the back ends for Linux, L4/Fiasco, and
L4ka::Pistachio simulate periodic timer interrupts using Linux' 'nanosleep' system
call - respective IPC timeouts. The OKL4 back end contains a PIT driver
and operates this timer device in one-shot mode.
To implement the timer server in a single-threaded manner, we used an
experimental API extension to Genode's server framework. Please note that we
regard this extension as temporary and will possible remove it with the next
release. The timer will then service its clients using the Genode's signal API.
Even though the timer service is a complete reimplementation, its interface
remains unmodified. So this change remains completely transparent at the API level.
VESA graphics driver
====================
The previous release introduced a simple PCI-bus virtualization into the VESA
driver. At startup, the VESA driver uses the PCI bus driver to find a VGA card
and provides this single PCI device to the VESA BIOS via a virtual PCI bus. All
access to the virtualized PCI device are then handled locally by the VESA
driver. In addition to PCI access, some VESA BIOS implementations tend to use
the programmable interval timer (PIT) device at initialization time. Because we
do not want to permit the VESA BIOS to gain access to the physical timer
device, the VESA driver does now provide an extremely crippled virtual PIT.
Well, it is just enough to make all VESA BIOS implementations happy that we
tested.
On the feature side, we added support for VESA mode-list handling and a
default-mode fallback to the driver.
Misc
====
:SDL-based frame buffer and input driver:
For making the Linux version of Genode more usable, we complemented the
existing key-code translations from SDL codes to Genode key codes.
:PS/2 mouse and keyboard driver:
Improved robustness against ring-buffer overruns in cases where input events
are produced at a higher rate than they can be handled, in particular, if
there is no input client connected to the driver.
Platform-specific changes
#########################
Support for super pages
=======================
Previous Genode versions for the OKL4, L4ka::Pistachio, and L4/Fiasco kernels used
4K pages only. The most visible implication was a very noticeable delay during
system startup on L4ka::Pistachio and L4/Fiasco. This delay was caused by core
requesting the all physical memory from the root memory manager (sigma0) -
page by page. Another disadvantage of using 4K pages only, is the resulting TLB footprint
of large linear mappings such as the frame buffer. Updating a 10bit frame buffer
with a resolution of 1024x768 would touch 384 pages and thereby significantly
pollute the TLB.
This release introduces support for super pages for the L4ka::Pistachio and
L4/Fiasco versions of Genode. In contrast to normal 4K pages, a super page
describes a 4M region of virtual memory with a single entry in the page
directory. By supporting super pages in core, the overhead of the startup
protocol between core and sigma0 gets reduced by a factor of 1000.
Unfortunately, OKL4 does not support super pages such that this feature remains
unused on this platform. However, since OKL4 does not employ a root memory
manager, there is no startup delay anyway. Only the advantage of super pages
with regard to reduced TLB footprint is not available on this platform.
Support for write-combined access to I/O memory
===============================================
To improve graphics performance, we added principle support for write combined I/O access
to the 'IO_MEM' service of core. The creator of an 'IO_MEM' session can now specify the
session argument "write_combined=yes" at session-creation time. Depending on the
actual base platform, core then tries to establish the correct page-table
attribute configuration when mapping the corresponding I/O dataspace. Setting
caching attributes differs for each kernel:
* L4ka::Pistachio supports a 'MemoryControl' system call, which allows for specifying
caching attributes for a core-local virtual address range. The attributes are
propagated to other processes when core specifies such a memory range
as source operand during IPC map operations. However, with the current version,
we have not yet succeeded to establish the right attribute setting, so the performance
improvement is not noticeable.
* On L4/Fiasco, we fully implemented the use of the right attributes for marking
the frame buffer for write-combined access. This change significantly boosts
the graphics performance and, with regard to graphics performance, serves us
as the benchmark for the other kernels.
* OKL4 v2 does not support x86 page attribute tables. So write-combined access
to I/O memory cannot be enabled.
* On Linux, the 'IO_MEM' service is not yet used because we still rely on libSDL
as hardware abstraction on this platform.
Unification of linker scripts and startup codes
===============================================
During the last year, we consistently improved portability and the support for
different kernel platforms. By working on different platforms in parallel,
code duplications get detected pretty easily. The startup code was a steady
source for such duplications. We have now generalized and unified the startup
code for all platforms:
* On all base platforms (Linux-x86_32, Linux-x86_64, OKL4, L4ka::Pistachio, and
L4/Fiasco) Genode now uses the same linker script for statically linked
binaries. Therefore, the linker script has now become part of the 'base'
repository.
* We unified the assembly startup code ('crt0') for all three L4 platforms.
Linux has a custom crt0 code residing in 'base-linux/src/platform'. For
the other platforms, the 'crt0' codes resides in the 'base/src/platform/'
directory.
* We factored out the platform-depending bits of the C++ startup code
('_main.cc') into platform-specific '_main_helper.h' files. The '_main.cc'
file has become generic and moved to 'base/src/platform'.
Linux
=====
With the past two releases, we successively reduced the dependency of the
Linux version of core from the 'glibc'. Initially, this step had been
required to enable the use of our custom libc. For example, the 'mmap'
function of our libc uses Genode primitives to map dataspace to the
local address space. The back end of the used Genode functions, in turn,
relied on Linux' 'mmap' syscall. We cannot use syscall bindings provided
by the 'glibc' for issuing the 'mmap' syscall because the
binding would clash with our libc implementation of 'mmap'. Hence we
started to define our own syscall bindings.
With the current version, the base system of Genode has become completely
independent of the 'glibc'. Our custom syscall bindings for the x86_32 and
x86_64 architectures reside in 'base-linux/src/platform' and consist of
35 relatively simple functions using a custom variant of the 'syscall'
function. The only exception here is the clone system call, which requires
assembly resides in a separate file.
This last step on our way towards a glibc-free Genode on Linux pushes the
idea to only use the Linux kernel but no further Linux user infrastructure
to the max. However, it is still not entirely possible to build a Linux
based OS completely based on Genode. First, we have to set up the loopback
device to enable Genode's RPC communication over sockets. Second, we
still rely on libSDL as hardware abstraction and libSDL, in turn, relies
on the glibc.
:Implications:
Because the Linux version is now much in line with the other kernel platforms,
using custom startup code and direct system calls, we cannot support
host tool chains to compile this version of Genode anymore. Host tool chains, in
particular the C++ support library, rely on certain Linux features
such as thread-local storage via the 'gs' segment registers. These things are
normally handled by the glibc but Genode leaves them uninitialized.
To build the Linux version of Genode, you have to use the official
Genode tool chain.
OKL4
====
The build process for Genode on OKL4 used to be quite complicated. Before
being able to build Genode, one had to build the original Iguana user land
of OKL4 because the Genode build system looked at the Iguana build directory
for the L4 headers actually used. We now have simplified this process by
not relying on the presence of the Iguana build directory anymore. All
needed header files are now shadowed from the OKL4 source tree
to an include location within Genode's build directory. Furthermore, we
build Iguana's boot-info library directly from within the Genode build system,
instead of linking the binary archive as produced by Iguana's build process.
Of course, to run Genode on OKL4, you still need to build the OKL4 kernel
but the procedure of building the Genode user land is now much easier.
Misc changes:
* Fixed split of unmap address range into size2-aligned flexpages. The
'unmap' function did not handle dataspaces with a size of more than 4MB
properly.
* Fixed line break in the console driver by appending a line feed to
each carriage return. This is in line with L4/Fiasco and L4ka::Pistachio,
which do the same trick when text is printed via their kernel debugger.
L4ka::Pistachio
===============
The previous version of core on Pistachio assumed a memory split of 2GB/2GB
between userland and kernel. Now, core reads the virtual-memory layout from
the kernel information page and thereby can use up to 3GB of virtual memory.
*Important:* Because of the added support for super pages, the Pistachio
kernel must be built with the "new mapping database" feature enabled!
L4/Fiasco
=========
Removed superfluous zeroing-out of the memory we get from sigma0. This change
further improves the startup performance of Genode on L4/Fiasco.
Build infrastructure
####################
Tool chain
==========
* Bumped binutils version to 2.19.1
* Support both x86_32 and x86_64
* Made tool_chain's target directory customizable to enable building and
installing the tool chain with user privileges
Build system
============
* Do not include dependency rules when cleaning. This change brings not
only a major speedup but it also prevents dependency rules from messing
with generic rules, in particular those defined in 'spec-okl4.mk'.
* Enable the use of '-ffunction-sections' combined with '-gc-sections'
by default and thereby reduce binary sizes by an average of 10-15%.
* Because all base platforms, including Linux, now depend on the Genode tool
chain, the build system uses this tool chain as default. You can still
override the tool chain by creating a custom 'etc/tools.conf' file
in your build directory.

1017
doc/release_notes-09-11.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1224
doc/release_notes-10-02.txt Normal file
View File

File diff suppressed because it is too large Load Diff

1211
doc/release_notes-10-05.txt Normal file
View File

File diff suppressed because it is too large Load Diff

618
doc/release_notes-10-08.txt Normal file
View File

@@ -0,0 +1,618 @@
===============================================
Release notes for the Genode OS Framework 10.08
===============================================
Genode Labs
The Genode project is back with the feature-packed release 10.08, set out to
bring device support of the Genode OS Framework to the next level. Our road
map hinted at two particular spots of activity, introducing wireless networking
and enabling hardware-accelerated graphics. To pursue the first goal, we pushed
the boundaries of our Linux device driver environment by porting Madwifi to
Genode. When it comes to hardware-accelerated graphics, today the Gallium3D
protocol stack and the corresponding GEM GPU drivers are the state of the art
on Linux and other UNIX-like operating systems. With the current release, we
make this powerful graphics architecture available on Genode, including
support for hardware-accelerated 3D graphics on Intel GMA GPUs. But we haven't
stopped with our device-driver related activities here as we introduce a new
ATAPI driver accommodated with an ISO9660 file-system implementation, and we
largely revisited our existing driver base.
Apart from device-driver support, two major features of the release are the
upgrade of the Qt4 framework from version 4.5.2 to version 4.6.3 alongside with
many performance and stability improvements, and the added support for dynamic
linking on ARM platforms with both the OKL4 and Codezero kernels.
Gallium3D and Intel's Graphics Execution Manager
################################################
Gallium3D is the most modern and most actively developed open-source software
stack regarding hardware-accelerated graphics. It is mainly deployed on
Linux but it has been ported to other operating systems such as the BSD family,
OpenSolaris, AROS, and now Genode. In the following, we will first provide
a little background about Gallium3D followed by a rough overview on how its
components fit into Genode.
Gallium3D was designed to displace the long-evolved but inherently insecure
direct rendering infrastructure on Linux. With DRI, each application that uses
hardware-accelerated graphics has unlimited access to the GPU and its resources
such as the frame buffer. To allow multiple applications to use the GPU in a
time-shared manner, those applications have to behave cooperatively. There is a
central service, the DRM driver in the kernel, orchestrating the applications
in such a way that well-behaved applications use distinct GPU resources such as memory
and contexts, and so come along nicely. However, there are no effective measures
against misbehaving applications. A further consequence of this architecture
is the multitude of vendor-specific protocol implementations. Because each
application contains an instance of the GPU driver accessing the broad hardware
interface of the graphics device, each vendor happened to take a different route
for translating graphics APIs such as OpenGL to the actual device interface.
Consequently, the code basis for graphics protocol stacks has become extremely
complex and fragmented. In contrast, the designers of Gallium3D set out to
modularize the protocol stack such that generic code is easy to use by different
vendors and the vendor-specific portion of the protocol stack stays as small as
possible, thereby lowering the costs for new-device support in the future.
In contrast to DRI, a Gallium-based application does not operate directly on
the GPU device. Instead, it uses a higher-level abstraction, namely buffer
objects for holding data operated on by the GPU. Buffer objects can contain
pixels, geometry data, and GPU command streams. The latter type of buffer
object can be issued for execution to perform actual GPU operations. The
buffer-object interface is provided by a central service called graphics
execution manager (GEM) normally residing in the kernel. GEM arbitrates the
allocation of buffer objects, manages cache coherency between GPU and CPU, and
passes buffers objects containing GPU command streams scheduled for execution to the
graphics device.
The high-complexity Gallium3D protocol stack is instantiated for each
application and acts as a client of the (relatively) low-complexity GEM.
Provided that the GPU command stream assembled by a Gallium3D application
cannot subvert the operation of GEM, this architecture removes the complex
protocol stack from the trusted computing base. Only the low-complexity GEM
service must be trusted with regard to security, robustness, and the absence
of GPU-based inter-application crosstalk. In contrast to DRI, Gallium3D is
perfectly in line with the architecturally principles of Genode. On Linux, the
Gallium3D stack communicates with GEM via 'ioctl' operations on the '/dev/drm/'
device interface. In the context of Genode, GEM should be executed as a
user-level device driver and resource multiplexer providing the GEM operations
as a GPU session interface by the means of RPC and shared memory. Each
Gallium3D application connects to the GPU server and operates on a GPU session.
The puzzle pieces of Mesa/Gallium3D
===================================
Gallium3D is part of the Mesa OpenGL library. It comes as a set of modules
consisting of state trackers (API implementations such as OpenGL),
drivers (translating generic graphics commands into device-specific command
streams), winsys (the glue between a window system and the Gallium3D
application), and a large library of utilities. Most of the code is
independent from the actual GPU device as well as the operating-system.
On Genode, Mesa-7.8.1 has been incorporated into the 'libports' repository.
However, we only use the Gallium3D-related parts of Mesa on Genode. For
example, the port does not make use of the Mesa swrast facility for
software-based rendering. It rather relies on the Gallium3D softpipe driver.
When built, all generic Gallium3D-related parts of Mesa are linked into the
single 'gallium.lib.so' library.
The following figure gives an overview of how the components of the graphics
software stack relate to each other. The components are described in the
following.
[image img/gallium3d]
EGL driver
~~~~~~~~~~
EGL is an OS-agnostic API for managing rendering buffers. The implementation
of this API is OS-specific because, among other things, it cares about the
interaction of the application with the native windowing system such that the
result of GPU-based rendering can be displayed in the GUI. EGL also plays a special
role among the Gallium state trackers because it is used by other state trackers.
The window-system-specific code is called EGL driver. Mesa-7.8.1 comes with
EGL drivers for the X window system and the Linux kernel-mode-switching
interface. For Genode, we have added a new EGL driver that uses Genode's
'Framebuffer_session' interface as back end. It is located at
'libports/src/lib/egl'. Because the EGL driver is GPU-independent, it is
part of 'gallium.lib.so'. The following screenshot shows the EGL driver
in action.
[image img/gallium_softpipe_screen]
Gallium i915 GPU driver
~~~~~~~~~~~~~~~~~~~~~~~
We picked Intel's GMA as the first series of supported GPUs because they
are well documented and GEM has been implemented first for Intel GPUs.
There are two Gallium3D drivers for Intel GPUs called i915 and i965.
Currently, we have ported the i915 driver that supports the following GPU
models: i915 G/GM, i945 G/GM/GME, G33G, Q33G, and Q35G. On Genode, the Gallium3D
GPU driver comes in the form of a shared library called 'gallium-i915.lib.so'.
Is gets dynamically loaded by the EGL driver using 'dlopen'. If the EGL
driver fails to load the 'gallium-i915.lib.so' driver, it falls back
to the softpipe driver compiled into 'gallium.lib.so'.
Because there is no build dependency to this shared library, we created a
pseudo build target at 'libports/src/lib/gallium/i915' for the sole purpose of
building this library as a side effect.
On Linux, the code contained in 'gallium-i915.lib.so' communicates with the '/dev/drm/'
device interface. However, the interaction with the device is not performed
directly but via a library called 'libdrm'.
libdrm
~~~~~~
The DRM library is a convenient front end to the '/dev/drm/' device. At
first glance, replacing this library with a Genode-specific implementation
seems natural. However, because 'libdrm' is not only a mere wrapper around the 'ioctl'
interface of the device but also contains a substantial amount of program logic
and device heuristics, we decided to reuse this library unmodified and go for
the 'ioctl' interface as a hook to connect Gallium3D with GEM. Hence, 'libdrm'
has become part of the 'libports' repository alongside Mesa. Ultimately,
'libdrm' translates all requests coming from 'gallium-i915.lib.so' to
(GPU-specific) 'ioctl' and 'mmap' operations on the '/dev/drm/' device. On
Genode, there is no such device. In fact, there are no device nodes at all.
Instead, we use our libc plugin mechanism to redirect operations on this
specific device file to a dedicated libc plugin. When 'libdrm' opens '/dev/drm/', the
libc plugin located at 'src/lib/libdrm/' takes over the responsibility of the
returned file descriptor. Therefore all file operations on this file handle are
handed over to the plugin. This is the point where we can transparently
incorporate RPC communication to the GEM service. For now, however, we do not
have RPC stub code yet. Instead, we link the GEM code directly into
'gallium-i915.lib.so'. This allows us to implement the GEM-related 'ioctl'
operations by calling GEM code directly. For the interaction between
'libdrm' and GEM, we added a preliminary 'Gpu_driver' interface located at
'os/include/gpu/'.
Graphics execution manager
~~~~~~~~~~~~~~~~~~~~~~~~~~
GEM is normally part of the Linux kernel and dispatches (a subset of)
operations on '/dev/drm/'. We use the Intel-specific GEM code taken from
Linux-2.6.34. This code relies on the Intel-AGP subsystem to manage the GPU's
graphics translation table. Therefore, we had to port the Intel-AGP sub system
as well. Because GEM is a relatively new feature of the Linux kernel, we
decided to not use our Linux device driver environment (currently based on the
kernel version 2.6.20) for our porting work but rather went for the creation
of a driver-specific Linux emulation environment. The code is part of the
'linux_drivers' repository and located at 'src/drivers/gpu/'. The 'contrib/'
subdirectory contains unmodified Linux code, the 'i915' subdirectory contains
the implementation of the 'Gpu_driver' interface and code for emulating Linux
interfaces in a way that the 'contrib/' code behaves like in its natural
execution environment.
Building an OpenGL application
==============================
As an example on how to build an OpenGL application on Genode, the 'libports'
repository provides a slightly modified version of the famous Gears demo.
You can find the code under 'libports/app/eglgears/'.
Prior building the application, make sure that you have issued 'make
prepare' for the 'mesa' and 'libdrm' libraries. In the root of the 'libports'
repository, issue the following commands to download the upstream source
codes for these libraries and to integrate them with the Genode build system:
! make prepare PKG=mesa
! make prepare PKG=libdrm
After having added 'libc' and 'libports' to the 'REPOSITORIES' declaration
of your '<build-dir>/etc/build.conf', you can building 'eglgears' via
'make app/eglgears'. The build process will create the 'eglgears' executable
alongside with 'gallium.lib.so'. You can start 'eglgears' using a plain
framebuffer such as 'vesa_drv'. The EGL driver included in 'gallium.lib.so'
will try to load a shared library called 'gallium-i915.lib.so' and, if not
present, revert to the softpipe driver.
If you want to give the hardware-accelerated version a spin, you will
need to build 'gallium-i915.lib.so'. The driver is only built when
the build 'SPECS' variable contains the keyword 'i915'. Simply add the
following line to your '<build-dir>/etc/specs.conf' file:
! SPECS += i915
Currently, the GEM (contained in the 'linux_drivers' repository') is
linked to 'gallium-i915.lib.so'. Hence, the 'linux_drivers' repository
must be specified in your 'build.conf'. The Gallium driver is built
as a side effect of building a pseudo target via:
! make lib/gallium
If you add the resulting 'gallium-i915.lib.so' to core's ROM service,
the EGL driver will attempt to use the hardware driver.
Current limitations
===================
At the current stage, Gallium3D on Genode is able to run the Gears
demo using Intel GMA GPUs. However, the work done so far must be
regarded as the first of several steps towards a complete solution.
Let us highlight the most important limitations and construction
sites:
* Both GEM and the Gallium3D protocol stack are executed as part of
a single process, accessing the GPU exclusively. Until we have
separated GEM from Gallium3D, only a single GPU-using application
can run at a time.
* Even though a Gallium3D application is able to use the GPU for
3D rendering, the EGL driver relies on CPU-based blitting
in order to transfer the rendering result to the screen.
* Interrupt-based synchronization has not been implemented yet. The
GPU is expected to be faster than the CPU. For this reason,
the EGL driver waits for 5 ms after each rendered frame.
Proper vblank handling is desired.
* On some platforms, we observed pixel artifacts, which
we attribute to cache coherency issues.
* Resource deallocation within the GEM driver has not been implemented
yet. Therefore, we expect that the current version is not suited for
highly dynamic applications.
* The 'eglgears' demo runs fine with resolutions up to 800x600
but we observed unstable behaviour with higher resolutions.
Despite of these limitations, the first version of Gallium3D on
Genode showcases that a subsystem as comprehensive as Gallium3D
plus GEM can be natively executed on Genode.
Operating-system services and libraries
#######################################
Init configuration concept
==========================
The previous release 10.05 introduced a new configuration concept that enables
the specification of mandatory access-control rules among flexible ways to
route service requests throughout the system. With the current release, this
concept is used by default. We have adapted all example configurations to the
new format, polished the new init implementation, and moved it from
'os/src/init/experimental/' to 'os/src/init/'. The old init variant is still
available at 'os/src/init/traditional/' but it is scheduled to be removed with
the next release 10.11.
The description of the configuration concept and the XML format is provided by
the document "Configuring the init process of Genode" located at
'os/doc/init.txt'.
Block session interface
=======================
The block session interface extends Genode's range of device-class interfaces by
a general-purpose interface to access block devices. It is based on the
packet-stream framework, using a single TX-stream to transmit block requests
to the server-side. Clients are free to request read/write operations on
several sequent blocks at once. Services implementing the server-side of the
block-session interface can choose an appropriate block size and the kind of
block-operation they support (e.g., read-only device). The new block session
interface is located at 'os/include/block_session/'.
ROM-loop block device
=====================
Based on the new block session interface, we implemented a service, which
provides a rom-file as read-only block-device. Linux users might know this
kind of service under the term "loop device". The following configuration
snippet shows how the file provided by a rom-session can be used as block
device:
! <config>
! ...
! <start name="rom_loopdev">
! <resource name="RAM" quantum="1M"/>
! <provides><service name="Block"/></provides>
! <config>
! <filename>livecd.iso</filename>
! </config>
! </config>
C runtime enhancements
======================
Both 'libc' and 'libm' are now built as *shared objects*, reducing the memory
footprint for scenarios with multiple libc-using applications. When starting
programs that use the libc, make sure to have 'libc.lib.so' and 'libm.lib.so'
available as files at the ROM service.
Motivated by our work on Gallium3D and libdrm, we extended the libc *plugin*
*interface* with the support of 'ioctl' and 'mmap'. This change enables us to
install custom handlers of those libc calls for a specific file. In the
particular case, libdrm performs 'ioctl' and 'mmap' calls referring to the
'/dev/drm' device interface. Now, we can supply a libc plugin specific for a
single file.
The update of Qt4 from version 4.5.2 to version 4.6.3 required refinements
of 'clock_gettime()', 'sysctl()', and 'getpagesize()'. Those functions
are still dummy stubs but with a meaningful behaviour from Qt4's perspective.
DDE Kit
=======
During our various device-driver activities, we improved the DDE Kit support
library for device-driver developments. The revised handling of I/O memory
resources now allows multiple requests of the same resource to support, e.g.,
multiple Linux 'ioremap()' calls. The I/O memory-mapping type is configurable
as uncached or write-combined, and DDE Kit automatically keeps track of
virtual-to-physical address mappings. Also, DDE Kit now provides 64-bit integer
types and a proper 'size_t'.
Dynamic linker
==============
In order to support shared libraries on ARM platforms, we added EABI support to
the dynamic linker and Genode's build-system environment. Thus shared libraries
are now supported on Codezero and OKL4 GTA01 targets. This also includes C++
exception handling.
Additionally, we implemented libc's dynamic linking interface ('dlfcn.h') and are
now able to support dynamic loading of libraries by applications via 'dlopen'
and friends.
Device drivers
##############
New ATAPI driver
================
With version 10.08, Genode provides a port of the low level ATA/ATPI driver
available from [http://ata-atapi.com]. Currently, the driver supports ATAPI
devices only and is implemented as a block-interface server (see Section
[Block device interface]). By default, the driver tries to take advantage of
the device's DMA engine but it can also operate in PIO mode as a fall-back
solution.
New wireless networking driver
==============================
According to our roadmap, we introduce initial support for wireless networking.
Based on DDE Linux 2.6, a port of the madwifi driver (version 0.9.4) is now
available for Genode. This driver supports widely used wifi-cards containing an
Atheros chipset (namely: 5210, 5211, 5212).
Due to the fact that part of the madwifi-project's contribution is binary code
in uuencoded form containing mandatory copyright headers, you need a 'uudecode'
binary installed on your system if you like to compile the madwifi driver for
Genode. If you're using Ubuntu/Debian or one of its derivates as your
development environment, you might install the necessary application via:
! sudo aptitude install sharutils
! emerge sharutils (on Gentoo)
This first wireless networking driver is in experimental stage and doesn't
support any form of encryption and authentication on the ieee80211 layer. So
you can only use it in conjunction with an unprotected access-point.
To set an appropriate ESSID you can tweak the driver's configuration, like in
the following example:
! <config>
! ...
! <start name="madwifi_drv">
! <resource name="RAM" quantum="2M"/>
! <provides><service name="Nic"/></provides>
! <config>
! <essid>My_access_point</essid>
! </config>
! </config>
When started, the 'madwifi_drv' announces a "Nic" session usable by the
lwIP stack.
PCI driver
==========
We enhanced the PCI bus scanning facility of our PCI driver with regard to
multi-function devices and added an accessor function for the physical
bus-device-function (BDF) ID.
VESA driver
===========
To support a wider range of graphics cards, we revised the VESA driver and
support more peculiarities of VBE implementations. Some of these are: unaligned
I/O port accesses, dependency on the physical BDF of PCI devices, support for
all flavours of PCI configuration space accesses.
PS/2 input driver
=================
Even after long years of intensive use, the PS/2 driver is sometimes good for a
surprise, which prompted us to improve the keyboard scan code and mouse button
handling. The driver fully supports scan code set 1 and 2 keyboards and copes
with oddities like "fake shift" events and "yet another scan code for Pause".
Timer
=====
The current release corrects a shortcoming of our timer driver on Pistachio and
Fiasco. Timing on these platforms is now more accurate.
Paravirtualized Linux
#####################
Based on the new block-session interface, we implemented a new stub driver
for OKLinux that enables the usage of a block-session device within Linux.
Thereby, the old stub driver that provided a ROM file as block device
is no longer needed and will be removed with the next release. A ROM file
can now be supplied to Linux via the new ROM loop service.
Protocol stacks and libraries
#############################
lwIP
====
Tweaking the configuration of the lightweight IP stack to better fit Genode's
application needs lead to a considerable improvement with respect to network
performance.
ISO9660 file system
===================
ISO9660 is the standard file system used on data CD/DVD medias. With the ATAPI
driver ready, we implemented ISO9660 support on top of the this driver. The
'iso9660' server implements the ROM-session interface and can be used by any
ROM connection. In order to take advantage of this new feature, we exploit
Genode's new configuration concept and route the client's ROM service
request to the ISO9660 server.
Configuration file snippet:
! <start name="atapi_drv">
! <resource name="RAM" quantum="1M" />
! <provides><service name="Block" /></provides>
! </start>
! <start name="iso9660">
! <resource name="RAM" quantum="10M" />
! <provides><service name="ROM" /></provides>
! </start>
! <start name="iso-client">
! <resource name="RAM" quantum="1M" />
! <route>
! <service name="ROM"><child name="iso9660"/></service>
! <any-service><parent /><any-child/></any-service>
! </route>
! </start>
:Limitations:
The memory necessary to read a file from the ATAPI driver into memory is
currently accounted on behalf of the ISO9660 server, not for the client side.
Because of this limitation, it becomes necessary to equip the ISO server with
a sufficient memory quota.
The 'Ecma-119' standard requires support for 8.3 upper-case file names only.
Because of this limitation, a number of unapproved ISO 9660 extensions have
evolved (eg. Joliet, Rock Ridge). Since we don't see using 8.3 file names within
Genode as an option, we added Rock Ridge extension support to the ISO 9660
server. Please make sure that your ISO-creation tool supports the Rock Ridge
extension and enable it during ISO creation.
Qt4.6.3
=======
We updated our port of the Qt4 framework from version 4.5.2 to version 4.6.3.
Thereby, we changed the way of how the source code is organized. Previously, we
maintained copies of modified files within the 'qt4' repository. Now, we
keep those changes in the form of patches, which get applied to the 'contrib'
code when 'make prepare' is issued within the 'qt4' repository. This change
significantly reduces the size of the 'qt4' repository. We applied the same
approach to the port of the Arora browser. Furthermore, the performance and
stability of Qt4 and Webkit in particular have received a lot of attention,
resulting in a much improved Arora browsing experience.
Platform-specific changes
#########################
OKL4
====
With the current release, we have started to maintain a few patches of the
official version of the OKL4v2 kernel. The patches are located at
'base-okl4/patches' and have the following purpose:
:'syscall_pic.patch':
The original distribution of the OKL4 kernel comes with x86 syscall bindings
that use absolute addressing modes. Therefore, code using L4 syscalls
cannot be compiled as position-independent code (gcc option '-fPIC').
Unfortunately, shared libraries must be compiled as position independent
because the location of such a library's text segment is not known at
compile time. Consequently, OKL4 syscalls cannot be issued by shared
libraries, which is a severe limitation. The patch fixes the problem
by changing all OKL4 syscall bindings and removing PIC-incompatible
addressing modes. It does not affect the functionality of the kernel.
:'eabi_build.patch':
The build system of the orignal OKL4 distribution is not prepared to
compile ARM EABI binaries as generated by modern tool chains such as the
Codesourcery GCC. The patch applies the needed changes to the OKL4 build
infrastructure.
Pistachio
=========
Similar to the situation with the OKL4 kernel, we need to patch the Pistachio
system-call bindings to enable syscalls from shared libraries. The
corresponding patch is located at 'base-pistachio/patches' and is known to work
with Pistachio revision 'r782:57124b75c67c'. Without applying this patch, the
linker generates text relocation infos, which result in a run-time error of the
'ldso' on the attempt to modify the read-only text segment of a shared library.
Codezero
========
Because we enhanced our dynamic linker to support ARM EABI, shared libraries
are now fully usable with the Codezero kernel.
Tools and build system
######################
Unified tool chain for building ARM targets
===========================================
With the previous versions of Genode,
we took the approach to use the tool chains of the respective kernel
to build Genode targets. For example, we used to rely on NICTA's ARM cross compiler
(based on gcc-3.4) for building Genode for the OKL4/gta01 platform.
Genode on Codezero, however, used the Codesourcery tool chain. We identified this approach as
a dead end because we would need to support modern tool chains alongside
ancient tool chains that are no longer used in practice. For accommodating
the latter, we had to introduce special workarounds and make compromises.
Therefore, we changed Genode to officially support one modern reference
tool chain to build all ARM-specific targets both on OKL4 and Codezero.
Currently, we use the Codesourcery tool chain version 2009q3-67, which
is available here:
:Codesourcery ARM EABI tool chain:
[http://www.codesourcery.com/sgpp/lite/arm/portal/release1039]
Because the original OKL4v2 distribution does not support modern ARM EABI tool
chains, it cannot be used out of the box anymore. But you can find a patch
to enable ARM EABI for OKL4v2 at 'base-okl4/patches/'.
Build system
============
* We changed the build system to link all shared libraries with
the '--whole-archive' option.
* All libraries are now built as position-independent code (compiler
option '-fPIC') by default. It is possible to explicitly disable
'-fPIC' by adding 'CC_OPT_PIC=' to the library description file.
* To ease the integration of third-party code into the Genode build
system, we have added a mechanism for setting source-file-specific
compiler options. Compiler arguments for a single file such as
'main.cc' can be assigned by setting the build variable 'CC_OPT_main'.

871
doc/release_notes-10-11.txt Normal file
View File

@@ -0,0 +1,871 @@
===============================================
Release notes for the Genode OS Framework 10.11
===============================================
Genode Labs
During the past three months, the Genode project was primarily driven by our
desire to create a bigger picture out of the rich set of components that we
introduced over time, in particular over the last year. Looking back at the
progress made since mid 2009, there were many functional additions to the
framework, waiting to get combined. To name a few, we added support for
networking, audio output, real-time priorities, mandatory access control,
USB, ATAPI block devices, Python, hardware-accelerated 3D graphics, Qt4,
the WebKit-based Arora browser, and the paravirtualized OKLinux kernel.
So many wonderful toys waiting to get played with. This is how the idea of
creating [http://genode.org/download/live-cds - the new Genode Live CD] was
born. In the past, Genode was mostly used in settings with a relatively static
configuration consisting of several components orchestrated to fulfill
a few special-purpose functions. Now, the time has come for the next step,
creating one dynamic setup that allows for the selection of different subsystems
at runtime rather than at boot time.
This step is challenging in several ways. First, the processes that form
the base system have to run during the entire time of all demo setups. If
any of those processes contained stability problems or leaked memory, it would
subvert the complete system. Second, the components of all subsystems combined
are far too complex to be loaded into memory at boot time. This would not
only take too long but would consume a lot of RAM. Instead, those components
and their data had to be fetched from disk (CDROM) on demand. Third, because
multiple demo subsystems can be active at a time, low-level resources such as
networking and audio output must be multiplexed to prevent different
subsystems from interfering with each other. Finally, we had to create a
single boot and configuration concept that is able to align the needs of all
demos, yet staying manageable.
Alongside these challenges, we came up with a lot of ideas about how Genode's
components could be composed in new creative ways. Some of these ideas such
as the browser-plugin concept and the http-based block server made it onto
the Live CD. So for producing the Live CD, we not only faced the said
technical challenges but also invested substantial development effort in new
components, which contributed to our overall goal. Two weeks ago, we released
the Live CD. This release-notes document is the story about how we got there.
To keep ourself focused on the mission described above, we deferred the
original roadmap goal for this release, which was the creation of a Unix-like
runtime environment to enable compiling Genode on Genode. This will be the
primary goal for the next release.
Execution environment for gPXE drivers
######################################
Up to now, DDE Linux provided Genode with drivers for hardware devices
ranging from USB HID to WLAN. In preparation of the live CD, we
noticed the demand for support of a broader selection of ethernet
devices. Intel's e1000 PCI and PCIe cards seemed to mark the bottom
line of what we had to support. The major advantage of NIC drivers
from Linux is their optimization for maximum performance. This emerges
a major downside if DDE Linux comes into play: We have to provide all
the nifty interfaces used by the driver in our emulation framework. To
achieve our short-term goal of a great live CD experience, we had to
walk a different path.
[http://gpxe.org/ - gPXE] is a lovely network boot loader / open-source
PXE ROM project and the successor of the famous Etherboot
implementation. Besides support for DNS, HTTP, iSCSI and AoE, gPXE
includes dozens of NIC drivers and applies a plain driver framework.
As we were also itching to evaluate DDE kit and the DDE approach at
large with this special _donator OS_, we went for implementing the
device-driver environment for gPXE (DDE gPXE).
The current version provides drivers for e1000, e1000e, and pcnet
devices. The emulation framework comprises just about 600 lines of
code compared to more than 22,000 LOC reused unmodified from gPXE.
Benchmarks with the PCNet32 driver showed that DDE gPXE's performance
is comparable to DDE Linux.
The gPXE driver environment comes in the form of the new 'dde_gpxe'
repository. For building DDE gPXE, you first need to download and patch
the original sources. The top-level makefile of this repository automates
this task. Just issue:
! make prepare
Now, you need to include the DDE gPXE repository into your Genode
build process. Just add the path to this directory to the
'REPOSITORIES' declaration of the 'etc/build.conf' file within your
build directory, for example
! REPOSITORIES += $(GENODE_DIR)/dde_gpxe
After successful build the DDE gPXE based ethernet driver is located
at 'bin/gpxe_nic_drv'.
On-demand paging
################
In the [http://genode.org/documentation/release-notes/8.11#section-8 - release 8.11],
we laid the foundation for implementing user-level dataspace managers.
But so far, the facility remained largely unused except for managing thread
contexts. This changed with this release.
So what is a user-level dataspace manager and who needs it? In short,
Genode's memory management is based on dataspaces. A dataspace is a
container for memory. Normally, it is created via core's RAM or ROM
services. The RAM service hands out dataspaces containing contiguous
physical memory. After allocating such a RAM dataspace, the creator can
attach the dataspace to its own address space to access the dataspace
content. In addition, it can pass a dataspace reference (called dataspace
capability) to other processes, which, in turn, can than attach the same
dataspace to their local address space, thereby establishing shared memory.
Similarly, core's ROM service hands out boot-time binary data as dataspaces.
For the most use cases of Genode so far, these two core services were the
only dataspace providers needed. However, there are use cases that require
more sophisticated memory management. For example, to implement swapping,
the content of a dataspace must be transferred to disk in a way that
is transparent to the users of the dataspace. In monolithic kernels, such
functionality is implemented in the kernel. But on a multi-server OS
such as Genode, this is no option. Implementing such a feature into
core would increase the trusted computing base of all applications
including those who do not need swapping. Core would need a hard-disk
driver, effectively subverting the Genode concept. Other examples for
advanced memory-management facilities are copy-on-write memory and
non-contiguous memory - complexity we wish to avoid at the root of the
process tree. Instead of implementing such memory management facilities
by itself, core provides a mechanism to let any process manage dataspaces.
This technique is also called user-level page-fault handling.
For the Live CD, we decided to give Genode's user-level page-fault handling
facility a go. The incentive was accessing files stored on CDROM in an
elegant way. We wanted to make the CDROM access completely transparent to
the applications. An application should be able to use a ROM session as
if the file was stored at core's ROM service. But instead of being
provided by core, the session request would be delegated to an
alternative ROM service implementation that reads the data from disk
as needed. Some of the files stored in the CDROM are large. For example,
the disk image that we use for the Linux demo is 160MB. So reading
this file at once and keeping it in memory is not an option. Instead, only
those parts of the file should be read from disk, which are actually
needed. To uphold the illusion of dealing with plain ROM files for
the client, we need to employ on-demand-paging in the CDROM server.
Here is how it works.
# The dataspace manager creates an empty managed dataspace. Core
already provides a tool for managing address spaces called
region manager (RM service). A RM session is an address space,
to which dataspaces can be attached. This is exactly what is
needed for a managed dataspace. So a dataspace manager uses the
same core service to define the layout of a managed dataspace
as is used to manage the address space of a process. In fact,
any RM session can be converted into a managed dataspace.
! enum { MANAGED_DS_SIZE = 64*1024*1024 };
! Rm_connection rm(0, MANAGED_DS_SIZE);
This code creates a RM session with the size of 64MB. This is an empty
address space. A dataspace capability that corresponds to this address
space can then be requested via
! Dataspace_capability ds = rm.dataspace();
# The dataspace capability can be passed to a client, which may
attach the dataspace to its local address space. Because the
managed dataspace is not populated by any backing store, however,
an access would trigger a page fault, halting the execution of
the client. Here, the page-fault protocol comes into play.
# The dataspace manager registers itself for receiving a signal each time
a fault occurs:
! Signal_receiver rec;
! Signal_context client;
! Signal_context_capability sig_cap = rec.manage(client);
! rm.fault_handler(sig_cap);
When an empty part of the managed dataspace is accessed by any
process, a signal is delivered. The dataspace manager can then
retrieve the fault information (access type, fault address) and
dispatch the page fault by attaching a real dataspace at the
fault address of the managed dataspace. In a simple case, the code
looks as follows:
! while (true) {
! Signal signal = rec.wait_for_signal();
! for (int i = 0; i < signal.num(); i++) {
! Rm_session::State state = rm.state();
! ds = alloc_backing_store_dataspace(PAGE_SIZE);
! rm.attach_at(ds, state.addr & PAGE_MASK);
! }
! }
This simple page-fault handler would lazily allocate a page of
backing store memory each time a fault occurs. When the backing
store is attached to the managed dataspace, core will automatically
wake up the faulted client.
# The example above has the problem that the dataspace manager has
to pay for the backing store that is indirectly used by the client.
To prevent the client from exhausting the dataspace manager's memory,
the dataspace manager may choose to use a limited pool of backing
store only. If this pool is exceeded, the dataspace manager can reuse
an already used backing-store block by first revoking it from its
current managed dataspace:
! rm.detach(addr);
This will flush all mappings referring to the specified address
from all users of the managed dataspace. The next time, this
address region is accessed, a new signal will be delivered.
This page-fault protocol has the following unique properties. First,
because core is used as a broker between client and dataspace manager, the
dataspace manager remains completely unaware of the identity of its client.
It does not even need to possess the communication right to the client. In
contrast, all other user-level page-fault protocols that we are aware of
require direct communication between client and dataspace manager. Second,
because dataspaces are used as first-level objects to resolve page faults,
page faults can be handed at an arbitrary granularity (of course, a multiple
of the physical page size). For example, a dataspace manager may decide to
attach backing-store dataspaces of 64K to the managed dataspace. So the
overhead produced by user-level page-fault handler can be traded for the
page-fault granularity. But most importantly, the API is the same across
all kernels that support user-level page fault handling. Thus the low-level
page-fault handling code becomes inherently portable.
Having said that, we have completed the implementation of the described
core mechanisms, in particular the 'detach' facility, for OKL4. The ISO9660
driver as featured on the Live CD implements the 'ROM' interface and
reads the contents of those files from CDROM on demand. It uses a
fixed pool of backing store, operates at a page-fault granularity of
64KB, and implements a simple fifo replacement strategy.
Base framework
##############
There had been only a few changes to the base framework described as
follows.
We unified the core-specific console implementation among all
base platforms and added synchronization of 'vprintf' calls.
The kernel-specific code resides now in the respective
'base-<platform>/src/base/console/core_console.h' files.
We removed the argument-less constructor from 'Allocator_avl_tpl'.
This constructor created an allocator that uses itself for
meta-data allocation, which is the usual case when creating
local memory allocators. However, on Genode, this code is typically
used to build non-memory allocators such as address-space regions.
For these use cases, the default policy is dangerous. Hence, we
decided to remove the default policy.
The 'printf' helper macros have been unified and simplified. The
available macros are 'PINF' for status information, 'PWRN' for warnings,
'PLOG' for log messages, and 'PERR' for errors. By default, the message
types are colored differently to make them easily distinguishable.
In addition to normal messages, there is the 'PDBG' for debugging
purposes. It remains to be the only macro that prints the function name
as message prefix and is meant for temporary messages, to be removed
before finalizing the code.
Genode's on-demand-paging mechanism relies on the signalling framework.
Each managed dataspace is assigned to a distinct signal context.
Hence, signal contexts need to be created and disposed alongside
with managed dataspaces. We complemented the signalling framework
with a 'dissolve' function to enable the destruction of signal
contexts.
Operating-system services and libraries
#######################################
Finished transition to new init concept
=======================================
With the release 10.05, we introduced the
[http://genode.org/documentation/release-notes/10.05#section-0 - current configuration concept of init].
This concept supports mandatory access control and provides flexible
ways for defining client-server relationships. Until now, we maintained
the old init concept. With the current release, the transition to the
new concept is finished and we removed the traditional init.
We retained the support for loading configurations for individual subsystems
from different files but adopted the syntax to the use of attributes.
Instead of
! <configfile>subsystem.config</configfile>
the new syntax is
! <configfile name="subsystem.config"/>
Virtual network bridge (Proxy ARP)
==================================
Since we originally added networking support to Genode, only one program could
use the networking facilities at a time. In the simplest form, such a program
included the network driver, protocol stack, and the actual application. For
example, the uIP stack featured with release 9.02 followed this approach.
In release 9.11 we added the 'Nic_session' interface to decouple the network
driver from the TCP/IP protocol stack. But the 1-to-1 relation between
application and network interface remained. With the current release, we
introduce the 'nic_bridge' server, which is able to multiplex the 'Nic_session'
interface.
The implementation is roughly based on the proxy ARP RFC 1027. At startup, the
'nic_bridge' creates a 'Nic_session' to the real network driver and, in turn,
announces a 'Nic' service at its parent. But in contrast to a network driver
implementing this interface, 'nic_bridge' supports an arbitrary number of
'Nic_sessions' to be opened. From the client's perspective, such a session
looks like a real network adaptor.
This way, it has become possible to run multiple TCP/IP stacks in
parallel, each obtaining a distinct IP address via DHCP. For example,
is has become possible to run multiple paravirtualized Linux kernels
alongside an lwIP-based web browser, each accessing the network via a
distinct IP address.
As a side effect for developing the 'nic_bridge', we created a set
of utilities for implementing network protocols. The utilities are
located at 'os/include/net' and comprise protocol definitions for
ethernet, IPv4, UDP, ARP, and DHCP.
Nitpicker GUI server
====================
Our work on the Live CD motivated several improvements of the Nitpicker
GUI server.
Alpha blending
~~~~~~~~~~~~~~
In addition to nitpicker's plain pixel buffer interface that is compatible
with a normal framebuffer session, each nitpicker session can now have
an optional alpha channel as well as an corresponding input-mask channel
associated. Both the alpha channel and the input mask are contained in the
same dataspace as the pixel buffer. The pixel buffer is followed by
the 8-bit alpha values, which are then followed by the input-mask values.
This way, the presence of an alpha channel does not interfere with the
actual pixel format. Each 8-bit input mask value specifies the user-input
policy for the respective pixel. If the value is zero, user input
referring to the pixel is not handled by the client but "falls through"
the view that is visible in the background of the pixel. This is typically
the case for drop shadows. If the input-mask value is '1', the input
is handled by the client.
With the input-mask mechanism in place, we no longer have a definitive
assignment of each pixel to a single client anymore. In principle, an
invisible client is able to track mouse movements by creating a full-screen
view with all alpha values set to '0' and all input-mask values set to '1'.
Once, the user clicks on this invisible view, the user input gets routed
to the invisible client instead of the actually visible view. This
security risk can be addressed at two levels:
* In X-Ray mode, nitpicker completely disables alpha blending
and the input-mask mechanism such that the user can identify the
client that is responsible for each pixel on screen.
* The use of the alpha channel is a session argument, which is specified
by nitpicker clients at session-creation time. Consequently, this
session argument is subjected to the policy of all processes involved
with routing the session request to nitpicker. Such a policy may permit
the use of an alpha channel only for trusted applications.
_Caution:_ The use of alpha channels implies read operations from
the frame buffer. On typical PC graphics hardware, such operations are
extremely slow. For this reason, the VESA driver should operate in
buffered mode when using alpha blending in Nitpicker.
Tinted views in X-Ray mode
~~~~~~~~~~~~~~~~~~~~~~~~~~
We added support for tinting individual clients or groups of clients
with different colors based on their label as reported at session-creation
time. By using session colors, nitpicker assists the user to tell apart
different security domains without reading textual information. In
addition to the tinting effect, the title bar presents the session
color of the currently focused session.
The following nitpicker configuration tints all views of the launchpad
subsystem in blue except for those views that belong to the testnit
child of launchpad. Those are tinted red.
! <config>
! <policy label="launchpad" color="#0000ff"/>
! <policy label="launchpad -> testnit" color="#ff0000"/>
! </config>
Misc Nitpicker changes
~~~~~~~~~~~~~~~~~~~~~~
We introduced a so-called 'stay-top' session argument, which declares
that views created via this session should stay on top of other views.
This function is useful for menus that should always remain accessible
or banner images as used for Live CD.
Nitpicker's reserved region at the top of the screen used to cover up
the screen area as seen by the clients. We have now excluded this area
from the coordinate system of the clients.
We implemented the 'kill' mode that can be activated by the 'kill' key.
(typically this is the 'Print Screen' key) This feature allows the user
to select a client to be removed from the GUI. The client is not
actually killed but only locked out. The 'kill' mode is meant as an
emergency brake if an application behaves in ways not wanted by the
user.
ISO9660 server
==============
As outlined in Section [On-demand paging], we revisited the ISO9660 server
to implement on-demand-paged dataspaces. It is the first real-world
use case for Genode's user-level page-fault protocol. The memory pool
to be used as backing store for managed dataspaces is dimensioned according
to the RAM assigned to the iso9660 server. The server divides this backing
store into blocks of 64KB and assigns those blocks to the managed dataspaces
in a fifo fashion. We found that using a granularity of 64KB improved the
performance over smaller block sizes because this way, we profit from reading
data ahead for each block request. This is particularly beneficial for
CDROM drives because of their extremely long seek times.
Audio mixer
===========
We added a new *channel synchronization* facility to the 'Audio_out_session'
interface. An 'Audio_out_session' refers to a single channel. For stereo
playback, two sessions must be created. At session-creation time, the
client can provide a hint about the channel type such as "front-left" as
session-construction argument. This design principally allows for supporting
setups with an arbitrary amount of channels. However, those channels must
be synchronized. For this reason, we introduced the 'sync_session' function
to the 'Audio_out_session' interface. It takes the session capability of
another 'Audio_out_session' as argument. The specified session is then
used as synchronization reference.
To reduce the latency when stopping audio replay, we introduced a new *flush*
function to the 'Audio_out_session' interface. By calling this function,
a client can express that it is willing to discard all audio data already
submitted to the mixer.
Furthermore, we improved the audio mixer to support both long-running
streams of audio and sporadic sounds. For the latter use case, low latency
is particularly critical. In this regard, the current implementation is a
vast improvement over the initial version. However, orchestrating the
mixer with audio drivers as well as with different clients (in particular
ALSA programs running on a paravirtualized Linux) is not trivial. In the
process, we learned a lot, which will eventually prompt us to further
optimize the current solution.
Nitpicker-based virtual Framebuffer
===================================
To support the browser-plugin demo, we introduced 'nit_fb', which is a
framebuffer service that uses the nitpicker GUI server as back end. It
is similar to the liquid framebuffer as featured in the 'demo' repository
but in contrast to liquid framebuffer, 'nit_fb' is non-interactive.
It has a fixed screen position and size. Furthermore, it does not
virtualize the framebuffer but passes through the framebuffer portion of
the nitpicker session, yielding better performance and lower latency.
If instantiated multiple times, 'nit_fb' can be used to statically arrange
multiple virtual frame buffers on one physical screen. The size and screen
position of each 'nit_fb' instance can be defined via Genode's configuration
mechanism using the following attributes of the 'nit_fb' config node:
! <config xpos="100" ypos="150"
! width="300" height="200"
! refresh_rate="25"/>
If 'refresh_rate' isn't set, the server will not trigger any refresh
operations by itself.
On the Live CD, each browser plugin instantiates a separate instance of
'nit_fb' to present the plugin's content on screen. In this case, the
view position is not fixed because the view is further virtualized by the
loader, which imposes its policy onto 'nit_fb' - Genode's nested
policies at work!
TAR ROM service
===============
For large setups, listing individual files as boot modules in single-image
creation tools (e.g., elfweaver) or multiboot boot loaders can be
cumbersome, especially when many data files or shared libraries are
involved. To facilitate the grouping of files, 'tar_rom' is an
implementation of the 'ROM' interface that operates on a 'tar' file.
The name of the TAR archive must be specified via the 'name' attribute of
an 'archive' tag, for example:
! <config>
! <archive name="archive.tar"/>
! </config>
The backing store for the dataspaces exported via ROM sessions is accounted
on the 'rom_tar' service (not on its clients) to make the use of 'rom_tar'
transparent to the regular users of core's ROM service. Hence, this service
must not be used by multiple clients that do not trust each other.
Typically, 'tar_rom' is instantiated per client.
The Live CD uses the 'tar_rom' service for the browser demo. Each plugin
is fetched from the web as a tar file containing the config file of the
plugin subsystem as well as supplemental binary files that are provided
to the plugin subsystem as ROM files. This way, a plugin can carry along
multiple components and data that form a complete Genode subsystem.
DDE Kit
=======
The DDE kit underwent slight modifications since the previous release.
It now provides 64-bit integer types and a revised virtual PCI bus
implementation.
Device drivers
##############
PCI bus
=======
Genode was tested on several hardware platforms in preparation of the
current release. This revealed some deficiencies with the PCI bus
driver implementation. The revised driver now efficiently supports
platforms with many PCI busses (as PCIe demands) and correctly handles
multi-function devices.
VESA framebuffer
================
We updated the configuration syntax of the VESA driver to better match
the style of new init syntax, preferring the use of attributes rather than
XML sub nodes. Please refer to the updated documentation at
'os/src/drivers/framebuffer/vesa/README'.
:Buffered output:
To accommodate framebuffer clients that need to read from the frame buffer,
in particular the nitpicker GUI server operating with alpha channels, we
introduced a buffered mode to the VESA driver. If enabled, the VESA driver
will hand out a plain memory dataspace to the client rather than the
physical framebuffer. Each time, the client issues as 'refresh' operation
on the framebuffer-session interface, the VESA driver copies the corresponding
screen region from the client-side virtual framebuffer to the physical
framebuffer. Note that the VESA driver will require additional RAM quota
to allocate the client buffer. If the quota is insufficient, the driver will
fall back to non-buffered output.
:Preinitialized video modes:
As an alternative to letting the VESA driver set up a screen mode, the
driver has become able to reuse an already initialized mode, which is useful
if the VESA mode is already initialized by the boot loader. If the screen
is initialized that way, the 'preinit' attribute of the 'config' node can
be set to '"yes"' to prevent the driver from changing the mode. This way,
the driver will just query the current mode and make the already
initialized framebuffer available to its client.
Audio
=====
We observed certain hardware platforms (in particular VirtualBox) to
behave strangely after ALSA buffer-underrun conditions. It seems that the
VirtualBox audio driver plays actually more frames than requested by
ALSA's 'writei' function, resulting in recurring replay of data that
was in the buffer at underrun time. As a work-around for this problem,
we zero-out the sound-hardware buffer in the condition of an ALSA buffer
underrun. This way, the recurring replay is still there, but it is
replaying silence.
To improve the support for sporadic audio output, we added a check for the PCM
state for buffer underruns prior issuing the actual playback. In the event of
an underrun, we re-prepare the sound card before starting the playback.
Furthermore, we implemented the new flush and channel-synchronization
abilities of the 'Audio_out_session' interface for the DDE Linux driver.
Paravirtualized Linux
#####################
To support the demo scenarios that showcase the paravirtualized Linux kernel,
we enhanced our custom stub drivers of the OKLinux kernel. Thereby, we have
reached a high level of integration of OKLinux with native Genode services,
including audio output, block devices, framebuffer output, seamless integration
with the Nitpicker GUI, and networking. All stub drivers are compiled in by
default and are ready to use by specifying a device configuration in the config
node for the Linux kernel. This way, one Linux kernel image can be easily used
in different scenarios.
:Integration with the Nitpicker GUI:
We enhanced our fbdev stub driver with a mechanism to merge view reposition
events. If a X11 window is moved, a lot of subsequent events of this type are
generated. Using the new optimization, only the most recent state gets
reported to Nitpicker, making the X11 GUI more responsive.
:UnionFS:
As we noticed that unionfs is required by all our Linux scenarios, we decided
to include and enable the patch by default.
:Network support:
With the introduction of the 'nic_bridge', multiple networking stacks can run
on Genode at the same time, which paves the way for new use cases. We have now
added a stub driver using Genode's 'Nic_session' interface to make the new
facility available to Linux.
:Audio output:
We adapted the ALSA stub driver to the changes of the 'Audio_out_session'
interface, using the new channel synchronization and flush functions.
Thereby, we optimized the stub driver to keep latency and seek times of
Linux userland applications reasonably low.
:Removed ROM file driver:
With the addition of the 'Block_session' stub driver, the original ROM file
driver is no longer required. So we removed the stub. For using ROM files as
disk images for Linux, there is the 'rom_loopdev' server, which provides a
block session that operates on a ROM file.
:Asynchronous block interface:
To improve performance, we changed the block stub driver to facilitate the
asynchronous mode of operation as provided by the 'Block_session' interface.
This way, multiple block requests can be issued at once, thereby shadowing
the round trip times for individual requests.
Protocol stacks and libraries
#############################
Gallium3D / Intel GEM
=====================
We improved the cache handling of our DRM emulation code (implementing
'drm_clflush_pages') and our EGL driver, thereby fixing caching
artifacts on i945 GPUs. Furthermore, we added a temporary work-around
for the currently dysfunctional sequence-number tracking with i945 GPUs.
On this chipset, issuing the 'MI_STORE_DWORD_INDEX' GPU command used
for tracking sequence numbers apparently halts the processing the command
stream. This condition is normally handled by an interrupt. However,
we have not enabled interrupts yet.
To prepare the future support for more Gallium drivers than i915, we
implemented a driver-selection facility in the EGL driver. The code
scans the PCI bus for a supported GPU and returns the name of the
corresponding driver library. If no driver library could be found,
the EGL driver falls back to softpipe rendering.
lwIP
====
We revised our port of the lwIP TCP/IP stack, and thereby improved its
stability and performance.
* The lwIP library is now built as shared object, following the convention
for libraries contained in the 'libports' repository.
* By default (when using the 'libc_lwip_nic_dhcp' library), lwIP will
issue a DHCP request at startup. If this request times out, the loopback
device is set as default.
* If there is no 'Nic' service available, the lwIP stack will fall back to
the loopback device.
* We increased the default number of PCBs in lwIP to 64.
* We removed a corner case of the timed semaphore that could occur
when a timeout was triggered at the same time ,'up' was called.
In this case, the semaphore was unblocked but the timeout condition
was not reflected at the caller of 'down'. However, the lwIP code
relies on detecting those timeouts.
Qt4
====
We implemented a custom *nitpicker plugin widget*, which allows for the
seamless integration of arbitrary nitpicker clients into a Qt4 application.
The primary use case is the browser plugin mechanism presented at
the Live CD. In principle, the 'QNitpickerViewWidget' allows for creating
mash-up allocations consisting of multiple native Genode programs. As shown
by the browser plugin demo, a Qt4 application can even integrate other
programs that run isolated from the Qt4 application, and thereby depend on
on a significantly less complex trusted computing base than the Qt4
application itself.
[image img/nitpicker_plugin]
The image above illustrates the use of the 'QNitpickerViewWidget' in the
scenario presented on the Live CD. The browser obtains the Nitpicker view to be
embedded into the website from the loader service, which virtualizes the
Nitpicker session interface for the loaded plugin subsystem. The browser then
tells the loader about where to present the plugin view on screen. But it has
neither control over the plugin's execution nor can it observe any user
interaction with the plugin.
New Gems repository with HTTP-based block server
################################################
To give the web-browser demo of our Live CD a special twist, and to show off
the possibilities of a real multi-server OS, we decided to implement the
somewhat crazy idea of letting a Linux OS run on a disk image fetched at
runtime from a web server. This way, the Linux OS would start right away and
disk blocks would be streamed over the network as needed. Implementing this
idea was especially attractive because such a feature would be extremely hard
to implement on a classical OS but is a breeze to realize on Genode where all
device drivers and protocol stacks are running as distinct user-level
components. The following figure illustrates the idea:
[image img/http_block]
The block stub driver of the Linux kernel gets connected to a special block
driver called 'http_block', which does not access a real block device but
rather uses TCP/IP and HTTP to fetch disk blocks from a web server.
Because the 'http_block' server is both user of high-level functionality (the
lwIP stack) and provider of a low-level interface ('Block_session'), the
program does not fit well into one of the existing source-code repositories.
The 'os' repository, which is normally hosting servers for low-level interfaces
is the wrong place for 'http_block' because this program would make the 'os'
repository depend on the higher-level 'libports' repository where the 'lwip'
stack is located. On the other hand, placing 'http_block' into the 'libports'
repository is also wrong because the program is not a ported library. It merely
uses libraries provided by 'libports'. In the future, we expect that native
Genode components that use both low-level and high-level repositories will
become rather the norm than an exception. Therefore, we introduced a new
repository called 'gems' for hosting such programs.
Tools
#####
Automated coding-style checker
==============================
As Genode's code base grows and new developers start to get involved,
we noticed recurring questions regarding coding style. There is a
[http://genode.org/documentation/developer-resources/coding_style - document]
describing our coding style but for people just starting to get involved,
adhering all the rules can become tedious. However, we stress the importance
of a consistent coding style for the project. Not only does a consistent style
make the framework more approachable for users, but it also eases the work
of all regular developers, who can feel right at home at any part of
the code.
To avoid wasting precious developer time with coding-style fixes, we
have created a tool for the automated checking and (if possible) fixing
the adherence of source code to Genode's coding style. The tool is
located at 'tool/beautify'. It takes a source file as argument and
reports coding-style violations. The checks are fairly elaborative:
* Placement of braces and parenthesis
* Indentation and alignment, trailing spaces
* Vertical spacing (e.g., between member functions, above comments)
* Naming of member variables and functions (e.g., private members start with '_')
* Use of upper and lower case
* Presence of a file header with the mandatory fields
* Policy for function-header comments (comment at declaration, not
at implementation)
* Style of single-line comments, function-header comments, multi-line comments
The user of 'beautify' may opt to let the tool fix most of the violations
automatically by specifying the command line arguments '-fix' and '-write'.
With only the '-fix' argument, the tool will output the fixed version of
the code via stdout. By specifying the '-write' argument, the changes will
be written back to the original file. In any case, we strongly recommend
to manually inspect all changes made by the tool.
Under the hood, the tool consists of two parts. A custom C++ parser called
'parse_cxx' reads the source code and converts it to a syntax tree. In the
syntax tree, all formating information such as whitespaces are preserved.
The C++ parser is a separate command-line tool, which we also use for
other purposes (e.g., generating the API documentation at the website).
The actual 'beautify' tool calls 'parse_cxx', and applies its checks and
fixes to the output of 'parse_cxx'. For this reason, both tools have to
reside in the same directory.
Platform-specific changes
#########################
OKL4
====
:Added support for shared interrupts:
The Genode Live CD operates on a large number of devices that trigger
interrupts (USB, keyboard, mouse, ATAPI, timer, network). On most
platforms, the chances are extremely high that some of them use
the same IRQ line. Therefore, we enhanced core's IRQ service to
allow multiple clients to request the same IRQ. If the interrupt occurs,
all clients referring to this interrupt are notified. The interrupt
gets cleared after all of those clients responded. Even though, we regard
PIC interrupts as a legacy, the support of shared interrupts enables
us to use OKL4 with such complex usage scenarios.
:Revised page-fault handling:
If a page fault occurs, the OKL4 kernel delivers a message to the page-fault
handler. The message contains the page-fault address and type as well as the
space ID where the fault happened. However, the identity of the faulting
thread is not delivered. Instead, the sender ID of the page fault message
contains the KTCB index of the faulting thread, which is only meaningful
within the kernel. This KTCB index is used as a reply token for answering the
page fault message. We wondered about why OKL4 choose to deliver the KTCB
index rather then the global thread ID as done for plain IPC messages. The
only reasonable answer is that by using the KTCB index directly in OKL4's
page-fault protocol, one lookup from the userland-defined thread ID to the
KTCB index can be avoided. However, this comes at the cost of losing the
identity of the faulting thread. We used to take the space ID as a key for
the fault context within core. However, with Genode's user-level page-fault
mechanism, this simplification does not suffice anymore. We have to know the
faulting thread as a page fault may not be answered immediately but at a
later time. During that time, the page-fault state has to be stored at core's
representation of the faulting thread. Our solution is reverting OKL4's
page-fault protocol to operate with global thread IDs only and to never make
kernel-internal KTCB indices visible at the user land. You can find the patch
for the OKL4 kernel at 'base-okl4/patches/reply_tid.patch'.
:Reboot via kernel debugger:
We fixed the reboot code of OKL4's kernel debugger to improve our work
flow. The patch can be found at 'base-okl4/patches/kdb_reboot.patch'.
:Relieved conflict with libc 'limits.h':
For some reason, the OKL4 kernel bindings provide definitions
normally found in libc headers. This circumstance ultimately leads
to trouble when combining OKL4 with a real C runtime. We have
relieved the problem with the patch 'base-okl4/patches/char_bit.patch'.
:Exception handling:
We added a diagnostic message to core that reports about exceptions
such as division by zero.
Pistachio
=========
Our revised syscall bindings for supporting position-independent code
on L4ka::Pistachio have been integrated into the mainline development
of the kernel. Therefore, the patch is not needed anymore when using
a kernel revision newer than 'r791:0d25c1f65a3a'.
Linux
=====
On Linux, we let the kernel manage all virtual address spaces for us,
except for the thread-context area. Because the kernel does not know
about the special meaning of the thread-context area, it may choose to
use this part of the virtual address space as target for 'mmap'. This
may lead to memory corruption. Fortunately, there is a way to tell the
kernel about virtual address regions that should be reserved. The
trick is to pre-populate the said region with anonymous memory using
the 'mmap' arguments 'MAP_PRIVATE', 'MAP_FIXED', 'MAP_ANONYMOUS', and
'PROT_NONE'. The kernel will still accept a fixed-address mapping
within such a reserved region (overmap) but won't consider using the
region by itself. The reservation must be done at the startup of each
process and each time when detaching a dataspace from the thread
context area. For the process startup, we use the hook function
'main_thread_bootstrap' in 'src/platform/_main_helper.h'. For reverting
detached dataspaces to a reserved region within the context area, we
added as special case to 'src/base/env/rm_session_mmap.cc'.
For hybrid programs (Genode processes that link against native
shared libraries of the Linux system), which are loaded by the dynamic
linker of Linux, we must further prevent the dynamic linker from
populating the thread-context area. This is achieved by adding a
special program segment at the linking stage of all elf binaries.

876
doc/release_notes-11-02.txt Normal file
View File

@@ -0,0 +1,876 @@
===============================================
Release notes for the Genode OS Framework 11.02
===============================================
Genode Labs
One year ago, the release 10.02 was our break-through with regard to the support
of multiple kernels as base platform for Genode. With the added support for
the NOVA hypervisor and the Codezero kernel, Genode applications could be executed
on 6 different kernels. With the current release, we take our commitment to
kernel platform support even further. With the added support for the Fiasco.OC
kernel, we make Genode available on one of the most feature-rich modern microkernels.
Additionally, we entered the realms of kernel design with our new platform support
for the Xilinx MicroBlaze architecture. This platform support comes in the shape
of a custom kernel specifically targeted to the MicroBlaze CPU architecture.
Furthermore, we updated our support for the NOVA Hypervisor to the bleeding-edge
version 0.3, which has been released earlier this month.
With the current support for 8 different kernel platforms (L4/Fiasco, Linux,
L4ka::Pistachio, OKL4, NOVA, Codezero, Fiasco.OC, and native MicroBlaze), testing
and integrating application scenarios across all platforms becomes increasingly
challenging. Therefore, we introduce a new framework for automating such tasks.
Thanks to the tight integration of the automation tool with Genode's build system,
going back and forth between different kernels becomes an almost seamless
experience.
Functionality-wise, the release carries on our vision to create a highly secure
yet easy to use general-purpose operating system. Because the Genode framework
is developed on Linux using the wonderful GNU tools, we consider the
availability of the GNU user land on Genode as crucial for using the system by
ourself. This motivation drives the creation of a custom execution environment
for GNU software on top of Genode. With the current release, we are proud to
present the first pieces of this execution environment. Even though not fully
functional yet, it clearly shows the direction of where we are heading.
Support for Fiasco.OC
#####################
The OC in the name of the Fiasco.OC kernel stands for "object capability", hinting
at the most significant feature that sets current-generation microkernels such as
NOVA, seL4, and Fiasco.OC apart from their predecessors. Whereas previous L4 kernels
succeeded in protecting subsystems from each other, the new generation of kernels
is geared towards strict security policies. Traditionally, two protection domains
were able to communicate with each other if they both agreed. Communication partners
were typically globally known via their respective thread/task IDs. Obviously, this
policy is not able to guarantee the separation of subsystems. If two subsystems
conspire, they could always share information. Object-capability-based kernels
are taking the separation much further by prohibiting any communication between
protection domains by default. Two protection domains can communicate only if
a common acquaintance of both agrees. This default-deny policy facilitates the
creation of least-privilege security policies. From the ground up, Genode has
been designed as a capability-based system which is naturally capable of leveraging
kernel-based object-capability support if present. After NOVA, Fiasc.OC is the
second of Genode's base platforms that provides this feature.
Apart from being a capability-based kernel, Fiasco.OC has a number of compelling
features such as thorough support for ARM platforms and the x86 32/64 bit
architectures. It supports SMP, hardware virtualization, and provides special
optimizations for running paravirtualized operating systems.
Technically, Fiasco.OC is the successor of the L4/Fiasco kernel developed by
the OS group of the TU-Dresden. However, the kernel interface of Fiasco.OC has
not much in common with L4/Fiasco. Some heritages are still there (e.g., IPC
timeouts) but the kernel API has evolved to a fully object-oriented model.
:Thanks:
We are indebted to the main developer of Fiasco.OC Alexander Warg for being
very reponsive to our inquiries while doing the porting work. Thanks to his
support, the adaptation of Genode to this kernel has been an almost smooth
ride.
Prerequisites
=============
You need GNU C & C++ Compilers, GNU Binutils, GNU Make, and Perl to use the
Fiasco.OC build system. On Debian/Ubuntu systems, you have to install the
following packages:
! apt-get install make gawk g++ binutils pkg-config subversion
Moreover, you need to download and install the tool-chain used by Genode. Have
a look at this page:
:[http://genode.org/download/tool-chain]:
Genode tool-chain
Downloading and building Fiasco.OC
==================================
Checkout the Fiasco.OC sources and tool-chain to an appropriated directory:
! export REPOMGR_SVN_REV=27
! svn cat http://svn.tudos.org/repos/oc/tudos/trunk/repomgr |\
! perl - init http://svn.tudos.org/repos/oc/tudos fiasco l4re
Building the kernel
~~~~~~~~~~~~~~~~~~~
Create the build directory for the kernel:
! cd <path_to_fiasco_src_dir>/src/kernel/fiasco
! make BUILDDIR=<path_to_kernel_build_dir>
Go to the build directory, configure the kernel:
! cd mybuild
! make config
This will launch the configuration menu. Here you can configure your kernel.
The default config is just fine to test the Genode port. It will build a
uniprocessor IA32 kernel with debugging features enabled. You can exit the menu and
save the configuration by simply typing 'x'.
Now, build Fiasco.OC by invoking:
! make
Building necessary tools
~~~~~~~~~~~~~~~~~~~~~~~~
To practically use Fiasco.OC, you need in addition to the kernel a tool to
bootstrap it, and the initial pager of the system, namely 'sigma0'. Both tools
can be found in the L4 runtime environment's base directory. Outgoing from
the directory where you checked out the sources, you have to change to the
following directory:
! cd <path_to_fiasco_src_dir>/src/l4
Create another build directory:
! make B=<path_to_l4re_build_dir>
Again, you might want to tweak the configuration:
! make O=<path_to_l4re_build_dir> config
Finally, build the tools:
! make O=<path_to_l4re_build_dir>
Building the Fiasco.OC version of Genode
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The Fiasco.OC version of Genode is available at the Genode public subversion repository:
:http://genode.org/download/subversion-repository:
Information about accessing the Genode public subversion repository
Go to a directory where you want the Genode/Fiasco.OC build directory to remain. Use
the helper script in the 'tool/builddir' directory of the Genode source tree to
create the initial build environment. You need to state the absolute path to the
build directory of the L4 runtime environment as 'L4_DIR', as it contains the kernel
bindings needed by the Genode port.
! <path_to_genode_src_dir>/tool/builddir/create_builddir foc_x86_32 \
! L4_DIR=<path_to_l4re_build_dir> \
! GENODE_DIR=<path_to_genode_src_dir> \
! BUILD_DIR=<path_to_genode_build_dir>
Now, go to the newly created build directory and type make.
! cd <path_to_genode_build_dir>
! make
Booting Genode on top of Fiasco.OC
==================================
Example GRUB configuration entry:
! timeout 0
! default 0
!
! title Genode on Fiasco.OC
! kernel /bootstrap -modaddr=0x01100000
! module /fiasco -serial_esc
! module /sigma0
! module /core
! module /init
! module /config
! module /pci_drv
! module /vesa_drv
! module /ps2_drv
! module /timer
! module /nitpicker
! module /launchpad
! module /liquid_fb
! module /scout
! module /testnit
! module /nitlog
For an example of a matching Genode 'config' file, please take a look
at 'os/config/demo'.
The Genode binaries are located in '<path_to_genode_build_dir>/bin',
the 'fiasco' kernel in '<path_to_kernel_build_dir>'. Assuming you compiled
for x86/586 (the default), you can find the 'bootstrap' binary in
'bin/x86_586' and 'sigma0' in 'bin/x86_586/l4f' within the
'<path_to_l4re_build_dir>' directory.
Current state
=============
The adaptation of Genode to Fiasco.OC covers most parts of the Genode API
including advanced semantics such as cancelable locks and support for
real-time priorities. So far, it has been tested on the x86 architecture.
Because 'base-foc' does not contain x86-specific code, we expect no major
roadblocks for running Genode on Fiasco.OC on ARM. However, we have not
exercised tests in this regard.
As of today, there exist the following limitations of the Fiasco.OC support:
* The dynamic linker is not yet adapted to Fiasco.OC. Special care must
be taken for handling the parent capability for dynamically loaded
programs. We have already covered this issue for the NOVA version but
the adaptation to Fiasco.OC remains yet to be done.
* The destruction of sub systems is not yet fully stable. Because Genode
forms a more dynamic workload than the original userland accompanied with
the kernel, the usage pattern of the kernel API triggers different
effects. We are working with the Fiasco.OC developers to remedy this
issue.
* The signalling framework is not yet supported. A design exist but it is
not implemented yet.
We believe however that none of these limitations are a significant hurdle for
starting to use Genode with this kernel. Please expect this issues to be
resolved with the upcoming Genode release.
Technical details about 'base-foc'
==================================
The following technical bits are worth noting when exploring the use of
Genode with the 'base-foc' platform.
* The timer implementation uses a one thread-per-client mode of operation.
We use IPC timeouts as time source. Hence, the timer driver is hardware
independent and should work out of the box on all hardware platforms
supported by Fiasco.OC.
* Each 'Server_object' of Genode corresponds to a so-called IPC gate,
which is the Fiasco.OC kernel object used for capability invocation.
Therefore, protection and object integrity is provided at the fine
granularity of single 'Server_objects'. This is in line with our
support for NOVA's implementation of capability-based security.
* In contrast to the lock implementation that we used with the original
L4/Fiasco kernel, the 'base-foc' lock is a fully-featured Genode lock
with support for lock cancellation and blocking. For blocking and
waking up lock applicants, we use Fiasco.OC's IRQ objects.
* The allocator used for managing process-local capability selectors
does not yet support the reuse of capability selectors.
Further Information
===================
:genode/tool/builddir/README:
Reference manual for the 'create_builddir' script
:[http://os.inf.tu-dresden.de/fiasco]:
Official website for the Fiasco.OC microkernel.
Noux - an execution environment for the GNU userland
####################################################
Even though Genode is currently mainly geared to the classical special-purpose
application domains for microkernel-based systems, the main property that sets
Genode apart from traditional systems is the thorough support for dynamic
workloads and the powerful mechanisms for handling hardware resources and
security policies in highly dynamic setting. We are convinced that Genode's
architecture scales far beyond static special-purpose domains and believe in
the feasibility of Genode as a solid foundation for a fully-fledged general
purpose operating system. Internally at Genode Labs, we set up the ultimate
goal to switch from Linux to Genode for our day-to-day work. We identified
several functionalities that we could not live without and systematically try
to bring those features to Genode. Of course, the most fundamental programs
are the tools needed to develop and build Genode. Currently we are developing
on Linux and enjoy using the GNU userland.
Consequently, we require a solution for using this rich tool set on Genode.
The straight-forward way for making these tools available on Genode would be
running them within a virtualized Linux instance (e.g., using OKLinux on OKL4).
However, this approach would defeat our actual goal to create a highly secure
yet easy to use working environment because adding Linux to the picture would
involve administering the virtualized Linux system. We would prefer a native
solution that makes the overall system less, not more, complicated. This way
the idea for a native execution environment for the GNU userland on Genode
was born. The implementation is called Noux and the first bits of code are
featured in the 'ports' repository. Noux consists of two parts, a build
environment for compiling GNU programs such that they can be run as Genode
processes and an execution environment that provides the classical UNIX
functionality to these programs.
Noux build environment
======================
From our experience, porting existing UNIX applications to a non-UNIX system
tends to be a task of manual and time-consuming labour. One has to loosely
understand the build system and the relationship of the involved source codes,
implement dummy functions for unresolved references, and develop custom glue
code that interfaces the ported application to the actual system. Taking the
shortcut of changing the original code has to be avoided at any cost because
this produces recurring costs in the future when updating the application. In
short, this long-winding process does not scale. For porting a tool set such as
the GNU userland consisting of far more than a three-digit number of individual
programs, this manual approach becomes unfeasible. Therefore, we have created
a build environment that facilitates the use of the original procedure of
invoking './configure && make'. The challenge is to supply configure with
the right arguments and environment variables ('CFLAGS' and the like) such that
the package is configured against the Genode environment. The following
considerations must be taken:
* Configure must not detect any global headers (e.g., '/usr/include/')
or libraries (e.g., '/usr/lib/'). This can be achieved by the '-nostdinc' and
'-nostdlib' options
* Configure has to use the same include-search paths as used for compiling
normal libc-using Genode programs
* Configure must use the Genode tool chain
* The final linking stage must use the Genode linker script, the Genode
base libraries, and other Genode-specific linker arguments.
Thanks to the power of the GNU build system, all this can be achieved by
supplying arguments to './configure' and indirectly to the 'make' process via
environment variables. The new Noux build environment takes care of these
precautions. It comes in the form of the 'ports/mk/noux.mk' file which enables
the seamless integration of GNU packages with the Genode build system. To
compile a GNU package, the manual steps needed are reduced to the creation of a
'target.mk' file representing the package. This 'target.mk' defines the name
of the package (by default, the basename of the 'target.mk' enclosing directory
is assumed) and the location of the source package. With this approach, we
managed to build 'coreutils' (over 100 small UNIX utilities such as 'ls', 'cp',
'sort'), 'binutils' (GNU linker, assembler, object-file tools), 'findutils'
('find', 'xargs'), 'bash', 'dash', GNU make, and finally the GNU compiler
collection including 'g++'. The resulting binaries are ready to be executed as
native Genode processes. However, without the right environment that presents
the program the needed UNIX functionality, those programs won't do much.
This leads us to the Noux execution environment.
Noux execution environment
==========================
The Noux execution environment plays the role of a UNIX kernel for programs
built via the Noux build environment. In contrast to a real kernel, the Noux
environment is a plain Genode user-level process that plays the role of being
the parent of one or multiple Noux processes. In addition of providing the
'Genode::Parent' interface, Noux also provides a locally implemented service called
'Noux::Session' that offers UNIX-like system-calls via an RPC interface. Each
hosted program is linked against a special Noux libc plugin that catches all
libc calls that would normally result in a system call. It then transparently
forwards this function call to the 'Noux::Session' interface.
Currently the Noux execution environment implements the following
system calls: 'getcwd', 'write', 'stat', 'fstat', 'fcntl', 'open',
'close', 'dirent', 'fchdir', 'read', and 'execve'.
The execution environment submits arguments (argc, argv, environment) to the
hosted program, manages its current working directory and receives its exit
code. File operations are targeted to a custom VFS infrastructure, which
principally allows a flexible configuration of the virtual file system visible
to the hosted programs. At the current stage, Noux supports mounting plain tar
archives obtained from core's ROM service as read-only file system. On startup,
the Noux environment starts one process (the init process) and connects the
file descriptor 1 (stdout) to Genode's LOG service.
State of the implementation
~~~~~~~~~~~~~~~~~~~~~~~~~~~
The infrastructure implemented so far already allows the execution of many simple
UNIX tools such as 'ls -lRa', 'echo', 'seq', 'find'. The 'execve' system call
is implemented such that a new process is started that inherits the file
descriptors and the PID of the calling process. This allows using the exec
functionality of the 'bash' shell. However, because 'fork' is not implemented
yet, there is currently no way to start multiple programs hosted in a single
Noux execution environment.
As of today, the Noux environment is not considered to be usable for practical
purposes. However, it clearly shows the feasibility of the path we are walking.
With the foundation laid, we are looking forward to expanding Noux to a capable
solution for running our beloved GNU userland tools on Genode.
Vision
~~~~~~
The most significant intermediate result of pursuing the development of Noux is
the realization that such an environment is not exceedingly complex. Because of
the combination with Genode, we only need to provide a comfortable runtime as
expected by user processes but we can leave much of intricate parts of UNIX out
of the picture. For example, because we handle device drivers on Genode, we do
not need to consider device-user interaction in Noux. As another example,
because the problem of bootstrapping the OS is already solved by Genode, there
is no need to run an 'init' process within Noux. Our vision foresees that Noux
runtimes are to be created on demand for individual tasks such as editing a
file (starting a custom Noux instance containing only the file to edit and the
text editor), compiling source code (starting a custom Noux instance with only
the source code and the build tools). Because Noux is so simple, we expect the
runtime overhead of starting a Noux instance to be not more than the time
needed to spawn a shell in a normal UNIX-like system.
Test drive
~~~~~~~~~~
To give Noux a spin, we recommend using Linux as base platform as this is
the platform we use for developing it. First, you will need to download the
source code of the GNU packages. From within the 'ports' repository,
use the following command:
! make prepare PKG=coreutils
This command will download the source code of the GNU coreutils. You may
also like to give the other packages a try. To see what is available,
just call 'make' without any argument.
Create a build directory (e.g., using tool/builddir/create_builddir).
Change to the build directory and issue the command
! make run/noux
This command will execute the run script provided at 'ports/run/noux.run'.
First it builds core, init, and coreutils. Then it creates a tar archive
containing the installed coreutils. Finally, it starts the Noux environment on
Genode. Noux then mounts the TAR archive as file system and executes 'ls -laR',
showing the directory tree.
Approaching platform support for Xilinx MicroBlaze
##################################################
With the release 11.02, we are excited to include the first version of our
custom platform support for the Xilinx MicroBlaze CPU architecture. MicroBlaze
is a so-called softcore CPU, which is commonly used as part of FPGA-based
System-on-Chip designs. At Genode Labs, we are regularly using this IP core,
in particular for our Genode FPGA Graphics Project, which is a GUI software stack
and a set of IP cores for implementing fully-fledged windowed GUIs on FPGAs:
:Website of the Genode FPGA Graphics Project:
[http://genode-labs.com/products/fpga-graphics]
Ever since we first released the Genode FPGA project, we envisioned to combine
it with the Genode OS Framework. In Spring 2010, Martin Stein joined our team
at Genode Labs and accepted the challenge to bring the Genode OS Framework to
the realms of FPGA-based SoCs. Technically, this implies porting the framework
to the MicroBlaze CPU architecture. In contrast to most softcore CPUs such as
the popular Lattice Mico32, the MicroBlaze features a MMU, which is a fundamental
requirement for implementing a microkernel-based system. Architecturally-wise
MicroBlaze is a RISC CPU similar to MIPS. Many system parameters of the CPU
(caches, certain arithmetic and shift instructions) can be parametrized at
synthesizing time of the SoC. We found that the relatively simple architecture
of this CPU provides a perfect playground for pursuing some of our ideas about
kernel design that go beyond the scope of current microkernels. So instead of
adding MicroBlaze support into one of the existing microkernels already
supported by Genode, we went for a new kernel design. Deviating from the typical
microkernel, which is a self-sufficient program running in kernel mode that
executes user-level processes on top, our design regards the kernel as a part of
Genode's core. It is not a separate program but a library that implements the
glue between user-level core and the raw CPU. Specifically, it provides the
entrypoint for hardware exceptions, a thread scheduler, an IPC mechanism, and
functions to manipulate virtual address spaces (loading and flushing entries
from the CPU's software-loaded TLB). It does not manage any physical memory
resources or the relationship between processes. This is the job of core.
From the kernel-developer's point of view, the kernel part can be summarized as
follows:
* The kernel provides user-level threads that are scheduled in a round-robin
fashion.
* Threads can communicate via synchronous IPC.
* There is a mechanism for blocking and waking up threads. This mechanism
can be used by Genode to implement locking as well as asynchronous
inter-process communication.
* There is a single kernel thread, which never blocks in the kernel code paths.
So the kernel acts as a state machine. Naturally, there is no concurrency in the
execution paths traversed in kernel mode, vastly simplifying these code parts.
However, all code paths are extremely short and bounded with regard to
execution time. Hence, we expect the interference with interrupt latencies
to be low.
* The IPC operation transfers payload between UTCBs only. Each thread has a
so-called user-level thread control block which is mapped transparently by
the kernel. Because of this mapping, user-level page faults cannot occur
during IPC transfers.
* There is no mapping database. Virtual address spaces are manipulated by
loading and flushing physical TLB entries. There is no caching of mappings
done in the kernel. All higher-level information about the interrelationship
of memory and processes is managed by the user-level core.
* Core runs in user mode, mapped 1-to-1 from the physical address space
except for its virtual thread-context area.
* The kernel paths are executed in physical address space (MicroBlaze).
Because both kernel code and user-level core code are observing the same
address-space layout, both worlds appear to run within a single address
space.
* User processes can use the entire virtual address space (4G) except for a
helper page for invoking syscalls and a page containing atomic operations.
There is no reservation used for the kernel.
* The MicroBlaze architecture lacks an atomic compare-and-swap instruction. On
user-level, this functionality is emulated via delayed preemption. A kernel-
provided page holds the sequence of operations to be executed atomically and
prevents (actually delays) the preemption of a thread that is currently
executing instructions at that page.
* The MicroBlaze MMU supports several different page sizes (1K up to 16MB).
Genode fully supports this feature for page sizes >= 4K. This way, the TLB
footprint can be minimized by choosing sensible alignments of memory
objects.
Current state
=============
The MicroBlaze platform support resides in the 'base-mb' repository. At the
current stage, core is able to successfully start multiple nested instances of
the init process. Most of the critical kernel functionality is working. This
includes inter-process communication, address-space creation, multi-threading,
thread synchronization, page-fault handling, and TLB eviction.
This simple scenario already illustrates the vast advantage of
using different page sizes supported by the MicroBlaze CPU. If using
4KB pages only, a scenario with three nested init processes produces more than
300.000 page faults. There is an extremely high pressure on the TLB, which
only contains 64 entries. Those entries are constantly evicted so that
threshing effects are likely to occur. By making use of flexible page
sizes (4K, 16K, 64K, 256K, 1M, 4M, 16M), the number of page faults gets
slashed to only 1.800, speeding up the boot time by factor 10.
Currently, there is no restriction of IPC communication rights. Threads are
addressed using their global thread IDs (in fact, using their respective
indices in the KTCB array). For the future, we are planning to add
capabilty-based delegation of communication rights.
Building and using Genode on MicroBlaze
=======================================
For building Genode for the MicroBlaze platform, you need the MicroBlaze
tool chain as it comes with the Xilinx EDK. The tool chain is typically
prefixed with 'mb-'. Please make sure that the tool chain's 'bin/' directory
is included in your 'PATH' environment variable.
For building and starting Genode on MicroBlaze, you first need to create
a build directory using the build-directory creation tool:
! tool/builddir/create_builddir microblaze \
! BUILD_DIR=</path/to/build/dir> \
! GENODE_DIR=</path/to/genode/dir>
The 'base-mb' repository comes with support for Genode's run tool. In order to
use it, you will first need to declare the location of your qemu binary using
the 'QEMU=/path/to/qemu' variable in the '<build-dir>/etc/microblaze.conf'
file. Then you will be able to start an example scenario by issuing the
following command from within your build directory:
! make run/nested_init
Thereby, the 'run' tool will attempt to start core using the microblaze version
of qemu.
You can also find a simple hello-world example at 'base-mb/src/test/hello'.
The corresponding run script is located at 'base-mb/run/hello.run'. You can
execute it via 'make run/hello' from the build directory.
Note that currently, all boot modules are linked against the core binary.
To change the boot modules, the file 'base-mb/src/core/boot_modules.s' must
be modified.
For reference, we are using the following tools:
* mb-g++ (GCC) 4.1.1 20060524 (Xilinx 11.2 Build EDK_LS2.2
20 Apr 2009 Xilinx 11.2 Build EDK_LS2.2 23 Apr 2009)
* GNU ld version 2.16 Xilinx 11.2 Build EDK_LS2.2 23 Apr 2009
* GNU assembler 2.16 Xilinx 11.2 Build EDK_LS2.2 23 Apr 2009
* QEMU emulator version 0.14.50, Copyright (c) 2003-2008 Fabrice Bellard
Petalogix linux reference design targeting Xilinx Spartan 3ADSP-1800 boards.
Supporting the NOVA hypervisor version 0.3
##########################################
NOVA is a so called microhypervisor - a modern capability-based microkernel
with special support for hardware-based virtualization and IOMMUs. Since we
incorporated the initial support for the NOVA hypervisor in Genode one year
ago, this kernel underwent multiple revisions. The latest version was released
earlier this month. To our delight, much of the features that we missed from
the initial release had been implemented during the course of the last year. We
are especially happy about the fully functional 'revoke' system call and the
support for remote kernel-object creation.
With the Genode release 11.02, we officially support the latest NOVA version.
The update of Genode to the new version required two steps. First, because many
details of the kernel interface were changed between version 0.1 and version
0.3, we had to revisit our syscall bindings and adapting our code to changed
kernel semantics. Second, we filled our 'base-nova' code related to object
destruction and unmapping with life to benefit from NOVA's 'revoke' system
call. Consequently, we are now able to run the complete Genode software stack
including the dynamic linker on NOVA.
Note that for using Genode on NOVA, you will need to apply a small patch to the
NOVA source code. This patch enables the re-use of user-level thread control
blocks in the kernel. The patch can be found at 'base-nova/patches/utcb.patch'.
When executing NOVA on qemu, please specify the '-cpu coreduo' argument to the
qemu command line. When using Genode 'run' tool, you may assign this argument
to the 'QEMU_OPT' variable in '<build-dir>/etc/build.conf'.
:Thanks:
We are grateful for the ongoing very pleasant collaboration with Udo Steinberg
who is the driving force behind NOVA. Thanks for the ultra-fast responses to our
questions and for considering our suggestions regarding the feature set of
NOVA's kernel interface!
Base framework
##############
Upgrading existing sessions
===========================
Genode enables a client of a service to lend parts of its own resources to
the service when opening a session. This way, servers do not need to allocate
own resources on behalf of their clients and become inherently robust against
resource-exhaustion-based denial-of-service attacks.
However, there are cases when the client can not decide about the amount of
resources to lend at session-creation time. In such cases, we used to devise an
overly generous client policy. Now, we have added a new 'upgrade' function to
the 'Parent' and 'Root' interfaces that enables a client to upgrade the
resources of an existing session.
For the 'env()->rm_session()' and 'env()->ram_session()' of processes using
the Genode 'env' library, we implemented a transparent quota upgrade that kicks in
in the event of an exceeded metadata backing store.
Comprehensive accounting of core resources
==========================================
We changed all services of core to limit their respective resource usage
specifically for each individual session. For example, the number of dataspaces
that can be handled by a particular region-manager (RM) session depends on the
resource donation attached to the session. To implement this accounting scheme
throughout core, we added a generic 'Allocator_guard' utility to
'base/include/'. We recommend using this utility when implementing resource
multiplexers, in particular multi-level services. Thanks to this change in
core, the need for a slack memory reservation in core has vanished.
Various changes
===============
The remaining parts of the base API underwent no fundamental revision. The
changes are summarized as follows.
:C++ Support:
We removed 'libgcc' from our C++ support library ('cxx') and link
it to each individual final target and shared library instead. This change alleviates
the need to abuse the 'KEEP_SYMBOLS' mechanism that we used in 'cxx' to
keep libc-dependencies of GCC's support libraries local to the 'cxx'
library. Besides the benefit of reducing heuristics, this change improves
the compatibility with recent cross-compiling tool chains.
Furthermore, we added 'realloc' to the local libc support of the 'cxx'
library because recent ARM tool chains tend to use this function.
:Argument handling for 'main()':
We added a hook to the startup code to enable the implementation of
custom facilities for passing arguments to the main function. The
hook uses the global variables 'genode_argc' and 'genode_argv'.
:Child-exit policy hook:
We enhanced the 'Child_policy' with a new policy interface that allows
a simplified implementation of policies related to program termination.
:Changed API of 'Range_allocator':
We changed the return value of 'alloc_addr' to distinguish different error
conditions. Note that the boolean meaning of the return value is inverted.
Please check your uses of 'alloc_addr'!
Operating-system services and libraries
#######################################
C Runtime
=========
In conjunction with our work on Noux, we improved Genode's C runtime at many
places. First, we added libstdtime and some previously missing bits of libgdtoa
to the libc. These additions largely alleviate the need for dummy stubs, in
particular time-related functions. Second, we added the following functions to
our libc plugin interface: 'dup2', 'fchdir', 'fcntl', 'fstat', 'stat', and
'write'. This enables the creation of advanced libc plugins simulating a whole
file system as done with Noux. Still, there are a number of dummy stubs found
at 'libc/src/lib/libc/dummy.cc'. However, those stubs are now all defined as
weak symbols such that they can be overridden by libc plugins. Finally, we have
replaced the original 'exit' implementation that comes with the libc with a
Genode-specific version. The new version reports the exit code of the
application to the parent process via an 'Parent::exit()' RPC call.
Until now, Genode's libc magically handled output to stdout and stderr by
printing messages via Genode's LOG interface. We have now replaced this
hard-wired interface by an optional libc plugin called 'libc_log'. If present, write
operations to stdout are caught at the libc plugin interface and delegated to
the plugin, which implements the output to the LOG interface. If you have an
application using Genode's libc, you might consider adding the 'libc_log'
library to your 'target.mk' file.
Support for big numbers by the means of libgmp and libmpfr
==========================================================
We have now include both the GNU Multiple Precision Arithmetic Library and
(GMP) and MPFR to the 'ports' repository. This work was specifically motivated
by our port of GCC to Genode as GCC version 4.4.5 requires both libraries.
Because we intend to use those libraries primarily on x86_32, the current port
covers only this architecture. However, expanding the port to
further CPU architectures should be straight-forward if needed.
Furthermore, you can now also find GCC's 'longlong.h' header at
'libports/include/gcc'.
Qt4 updated to version 4.7.1
############################
The current release bumps the supported Qt4 version from 4.6.2 to 4.7.1 and the
Arora web browser (located at the ports repository) from version 0.10.2 to
version 0.11. Of course, we updated our custom additions such as our custom
Nitpicker plugin widget that enables the seamless integration of native
Nitpicker GUI clients into Qt4 applications to work with the new Qt4 version.
Tools
#####
Tool chain update to GCC 4.4.5 and Binutils 2.21
================================================
We upgraded the official Genode tool chain from gcc 4.2.4 to gcc 4.4.5. Please
update your tool chain by downloading the new binary archive (available for x86_32)
or building the tool chain from source using our 'tool/tool_chain' utility.
New support for automated integration and testing
=================================================
With the growing number of supported base platforms, the integration and testing
of Genode application scenarios across all kernels becomes
increasingly challenging. Each kernel has a different boot mechanism and
specific requirements such as the module order of multiboot modules (Fiasco's
bootstrap, Pistachio's sigma0 and kickstart), kernel parameters, or the
invocation of a single-image creation tool (OKL4's elfweaver). To make our
life supporting all those platforms easier, we have created a tool called
'run', which is tightly integrated within Genode's build system. In short 'run'
gathers the intrinsics in the form of a 'run/env' file specific for the
platform used by the current build directory from the respective
'base-<platform>' repository. It then executes a so-called run script, which
contains all steps needed to configure, build, and integrate an application
scenario. For example, a typical run script for building and running a test
case resides in a file called '<any-repository>/run/<run-script-name>.run' and
looks as follows:
! build "core init test/exception"
! create_boot_directory
! install_config {
! <config>
! <parent-provides>
! <!--<service name="ROM"/>-->
! <service name="LOG"/>
! </parent-provides>
! <default-route>
! <any-service> <parent/> </any-service>
! </default-route>
! <start name="test-exception">
! <resource name="RAM" quantum="1M"/>
! </start>
! </config>
! }
! build_boot_image "core init test-exception"
! append qemu_args "-nographic -m 64"
! run_genode_until {.*Exception \(label 0xffb0\) occured.*} 10
First, the build system is instructed to create the targets specified as argument
for the 'build' function. Next, for the integration part, a new boot directory is
created. On most kernel platform, the respective location of the boot directory
is '<build-dir>/var/run/<run-script-name>'. Initially, this directory is empty.
It gets populated with a 'config' file specified as argument of the 'install_config'
command, and by the boot modules specified at the 'build_boot_image' command.
Now that the integration is complete, the scenario is executed via the
'run_genode_until' command. This command takes a regular expression as
argument, which determines the successful termination of the test case. The
second argument is a timeout (is seconds). In the example, the test case will
fail if its output does not match the regular expression within the execution
time of 10 seconds.
The command 'append qemu_args' specifies run-script-specific qemu arguments in
the case that qemu is used to execute the scenario. This is the case for most
kernel platforms (except for Linux where core gets executed directly on the host).
Additional build-directory-specific qemu arguments can be specified in the
'etc/build.conf' file by defining the 'QEMU_OPT' variable. For example, to
prevent KVM being used on Ubuntu Linux, specify:
! QEMU_OPT = -no-kvm
To execute the run script from with build directory, you need to have Expect
installed. Typically, the Linux package is called 'expect'. Simply issue
the following command from within your build directory:
! make run/<run-script>
Note that you will need to have a GRUB 'stage2_eltorito' binary available
at '<genode-dir>/tool/grub' on base platforms that use an ISO image as boot
stategy.
Because the whole chain of actions, building, integrating, executing, and
validating an application scenario is now at the fingertips of issuing a
single command with no kernel-specific considerations needed, it has never
been easier to run the same scenario on a wide range of different kernels.
Please find further instructive examples at 'os/run/'. The 'ldso' run
script executes the test of the dynamic linker. It is completely generic.
The 'demo' run script starts Genode's default demo scenario and shows how
platform-specific considerations (e.g., which device drivers to use) can be
taken into account.
We found that the 'run' tool significantly boosted our productivity not
only for testing purposes but also for accelerating the development-test
cycle during our day-to-day work.
:Technical notes:
The 'run' tool uses Expect as automation tool. Expect is a Tcl interpreter,
which is accompanied by special functionality for automating interactive
command-line applications. Technically, a run script is an Expect script
which gets included by the 'tool/run' script. For the reference of
run-specific functions, please revise the documentation in the 'tool/run'
script. Because each run script is actual Expect source code, it is possible
to use all Tcl and Expect scripting features in a run script.
In particular, a run script may issue shell commands using Tcl's 'exec'
function. This way, even complex integration tasks can be accomplished.
For example, the integration of the Genode Live CD was done via a single
run script.
Build system
============
To facilitate the integration of 3rd-party build systems into the Genode build
process, we added support for pseudo targets that do not require any 'SRC'
declaration. Such 'target.mk' may contain custom rules that will be executed
when the target is revisited by the build system. The bindings are as follows:
! build_3rd_party:
! ...custom commands...
!
! $(TARGET): build_3rd_party
!
! clean_3rd_party:
! ...custom commands...
!
! clean_prg_objects: clean_3rd_party:

1289
doc/release_notes-11-05.txt Normal file
View File

File diff suppressed because it is too large Load Diff

703
doc/release_notes-11-08.txt Normal file
View File

@@ -0,0 +1,703 @@
===============================================
Release notes for the Genode OS Framework 11.08
===============================================
Genode Labs
One of Genode's most distinctive properties is its support for various
different kernels as base platforms. Each of the 8 currently supported kernels
differs with regard to features, security, hardware support, complexity, and
resource management. Even though different applications call for different
kernel properties, through Genode, those properties can be leveraged using a
unified API. The growing number of supported base platforms, however, poses two
challenges, which are the comprehension of the large diversity of tools and
boot concepts, and capturing of the semantic differences of all the kernels.
With the version 11.08, the framework mitigates the former challenge by
introducing a unified way to download, build, and use each of the
kernels with Genode's user-level infrastructure. The new tools empower users of
the framework to instantly change the underlying kernel without the need to know
the peculiarities of the respective kernels. Using microkernels has never been
easier.
The second challenge of translating each kernel's specific behaviour to the
framework's unified API longs for an automated testing infrastructure that
systematically exercises all the various facets of the API on all base
platforms. The new version introduces the tooling support especially designed
for conducting such quality-assurance measures. These tools largely remove the
burden of manual testing while helping us to uphold the stability and quality
of the framework as it grows in terms of functional complexity and number of
base platforms.
Speaking of functional enhancements, the work on version 11.08 was focused
on our block-device infrastructure and ARM support. The block-device-related
work is primarily motivated by our fundamental goal to scale Genode to a
general-purpose computing platform. The additions comprise new drivers for
SD-cards, IDE, SATA, USB storage as well as a new partition server. All those
components provide Genode's generic block interface, which is meant to be used
as back end for file systems. On file-system level, a new libc plugin utilizes
libffat to enable the straight-forward use of VFAT partitions by libc-using
programs.
The current release comes with far-reaching improvements with respect to
ARM-based platforms. The paravirtualized L4Linux kernel has been updated to
Linux version 2.6.39 running on both x86_32 and ARM. Also, Qt4 including Webkit
has become functional on ARMv6-based platforms.
Among the further improvements are many new examples in the form of
ready-to-use run scripts as well as a comprehensive documentation update.
Originally, we had planned to complement the Noux runtime environment to
support interactive command-line applications by the time of the current
release. However, we realized that the current users of the framework would
value the new streamlined tooling support, the enhanced documentation, and the
new quality-assurance infrastructure over such a functional addition. Hence, we
prioritized the topics accordingly. Even though you will find the first bits of
interactive GNU application support in this release, we deferred working on
this topic in full steam to the upcoming version 11.11.
Blurring the boundaries between different kernels
#################################################
Before the Genode project was born, each microkernel carried along its own
userland. For example, the L4/Fiasco kernel came with the L4 environment, the
OKL4 kernel came with Iguana, or the L4ka::Pistachio kernel came with a small
set of example components. Those user-level counterparts of the kernel
complemented their respective kernels with a runtime for user-level
applications and components while exposing significant parts of the kernel
interface at API level. Consequently, most if not all applications developed
against these APIs were tied to a particular kernel. On the one hand, this
approach enabled developers to fine-tune their programs using kernel-specific
features. On the other hand, much effort was wasted by duplicating other
people's work. Eventually, all of the mentioned userlands stayed limited to
special purposes - for the most part the purposes of operating-systems
researchers. Consequently, none of the microkernels gained much attention in
general-purpose computing. Another consequence of the highly fragmented
microkernel community was the lack of a common ground to compare different
kernels in an unbiased way because each userland provided a different set of
components and libraries.
Different application areas call for different kernel features such as
security mechanisms, scheduling, resource management, and hardware support.
Naturally, each kernel exhibits a specific profile of these parameters
depending on its primary purpose. If one microkernel attempted to accommodate
too many features, it would certainly sacrifice the fundamental idea of being
minimally complex. Consequently, kernels happen to be vastly different. During
the past three years, however, Genode has demonstrated that one carefully
crafted API can target highly diverse kernels, and thereby enables users of
the framework to select the kernel that fits best with the requirements
dictated by each application scenario individually. For us Genode developers,
it was extremely gratifying to see that kernels as different as Linux and NOVA
can be reconciled at the programming-interface level. Still, each kernel comes
with different tools, configuration mechanisms, and boot concepts. Even though
Genode programs can be developed in a kernel-independent way, the deployment of
such programs still required profound insights into the peculiarities of the
respective kernel.
With the current release, we introduce a fundamentally new way of using
different microkernels by unifying the procedures of downloading and building
kernels as well as integrating and running Genode programs with each of them.
Existing Genode application scenarios can be ported between kernels in an
instant without the need for deep insights into the kernel's technicalities. As
a teaser, consider the following commands for building and running Genode's
graphical demo scenario on the OKL4 microkernel:
! # check out Genode
! svn co https://genode.svn.sourceforge.net/svnroot/genode/trunk genode
!
! # download the kernel, e.g., OKL4
! make -C genode/base-okl4 prepare
!
! # create Genode build directory
! genode/tool/create_builddir \
! okl4_x86 BUILD_DIR=build
!
! # build everything and execute the interactive demo
! make -C build run/demo
The same principle steps can be used for any of the OKL4, NOVA,
L4/Fiasco, Fiasco.OC, L4ka::Pistachio, or Codezero kernels. You should
nevertheless consult the documentation at 'base-<platform>/doc/' before
starting to use a specific kernel because some base platforms require
the installation of additional tools.
Under the hood, this seamless way of dealing with different kernels is made
possible by the following considerations:
:Repository preparation:
Each kernel comes from a different source such as a Git/SVN/Mercurial
repository or a packaged archive. Some kernels require additional patches. For
example, OKL4 needs to be patched to overcome problems with modern tool chains.
Now, each 'base-<platform>' repository hosts a 'Makefile' that automates the
download and patch procedure. To download the source code of a kernel,
issue 'make prepare' from within the kernel's 'base-<platform>' directory. The
3rd-party source code will be located at 'base-<platform>/contrib/'.
:Building the kernel:
Each kernel has a different approach when it comes to configuration and
compilation. For example, NOVA comes with a simple 'Makefile', OKL4 relies on a
complex SCons-based build system, L4ka::Pistachio uses CML2 and autoconf (for
the userland tools). Furthermore, some kernels require the setting of specific
configuration values. We have streamlined all these procedures into the Genode
build process by the means of a 'kernel' pseudo target and a 'platform' pseudo
library. The kernel can be compiled directly from the Genode build directory by
issuing 'make kernel'. The 'platform' pseudo library takes care of making the
kernel headers available to Genode. For some kernels such as OKL4 and NOVA, we
replaced the original build mechanism with a Genode target. For other kernels
such as L4ka::Pistachio or Fiasco.OC, we invoke the kernel's build system.
:Genode build directory:
Genode build directories are created via the 'tool/create_builddir' tool.
This tool used to require certain kernel-specific arguments such as the
location of the kernel source tree. Thanks to the unified way of preparing
kernels, the need for such arguments has vanished. Now, the only remaining
arguments to 'create_builddir' are the actual platform and the location
of the build directory to create.
:System integration and booting:
As diverse the build systems of the kernels are, so are the boot concepts. Some
kernels rely on a multiboot-compliant boot loader whereas others have special
tools for creating boot images. Thankfully, Genode's run concept allows us to
hide the peculiarities of booting behind a neat and easy-to-use facade. For
each platform we have crafted a dedicated run environment located at
'base-<platform>/run/env', which contains the rules for system integration and
booting. Therefore, one and the same run script can be used to build and
execute one application scenario across various different kernels. For an
illustrative example, the 'os/src/run/demo.run' script can be executed on all
base platforms (except for base-mb) by issuing 'make run/demo' from within the
build directory.
Emerging block-device infrastructure
####################################
Since version 10.08, Genode is equipped with a block-session interface. Its
primary use cases so far were the supply of the paravirtualized OKLinux kernel
with backing store, and the access of the content of a bootable Live CD.
However, for our mission to use Genode as general-purpose computing platform,
disk device access is crucial. Therefore, we dedicated our attention to
various aspects of Genode's block-device infrastructure, reaching from
programming APIs for block drivers, over partition handling, to file-system
access.
:Block session interface:
The glue that holds all block-device-related components together is the generic
block interface 'os/include/block_session'. It is based on the framework's
packet-stream facility, which allows the communication of bulk data via shared
memory and a data-flow protocol using asynchronous notifications. The interface
supports arbitrary allocation schemes and the use of multiple outstanding
requests. Hence, it is generally suited for scatter-gather DMA and the use of
command queuing as offered by the firmware of modern block-device controllers.
(albeit the current drivers do not exploit this potential yet)
:Block component framework:
Our observation that components implementing the block session interface share
similar code patterns prompted us to design a framework API for implementing
this family of components. The set of classes located at 'os/include/block'
facilitate the separation of device-specific code from application logic.
Whereas 'component.h' provides the application logic needed to implement the
block service, the 'driver.h' is an abstract interface to be implemented by the
actual device driver. This new infrastructure significantly reduces code
duplication among new block-device drivers.
:Device-driver implementations:
The new block-device drivers introduced with the current release address
common types of block devices:
* By adding ATA read/write support to the ATAPI driver ('os/src/drivers/atapi'),
this driver can be used to access IDE disks now.
* The new fully-functional SD-card driver ('os/src/drivers/sdcard') enables the
use of SD-cards connected via the PL180 controller.
* The USB storage driver ('linux_drivers/src/drivers/usb') has been adapted
to the block-session interface and can be used on PC hardware.
* The new AHCI driver ('os/src/drivers/ahci') enables the access of disks
connected via SATA on PC hardware.
Because all drivers are providing the generic block-session interfaces, they
can be arbitrarily combined with components that use this interface as back
end, for example, the partition server and file systems.
:Partition manager as resource multiplexer:
The new partition manager ('os/src/server/part_blk') multiplexes one back-end
block session to multiple block sessions, each accessing a different partition.
Its natural role is being "plugged" between a block-device driver and a file
system.
:File-system access:
Even though a session interface for file systems does not exist yet, we
enabled the use of VFAT partitions through a libc plugin. This libc plugin uses
the ffat library to access files stored on a block device. An
application using this plugin can be directly connected to a block session.
New documentation
#################
The new way of dealing with different kernels motivated us to revisit and
complement our exiting documentation. The following documents are new or
have received considerable attention:
:[http://genode.org/documentation/developer-resources/getting_started - Getting started]:
The revised guide of how to explore Genode provides a quick way to
test drive Genode's graphical demo scenario with a kernel of your
choice and gives pointers to documents needed to proceed your
exploration.
:[http://genode.org/documentation/developer-resources/build_system - Build system manual]:
The new build-system manual explains the concepts behind Genode's
build system, provides guidance with creating custom programs and
libraries, and covers the tool support for the automated integration
and testing of application scenarios.
:[http://genode.org/documentation/components - Components overview]:
The new components-overview document explains the categorization of
Genode's components and lists all components that come with the framework.
:[http://genode.org/documentation/developer-resources/init - Configuration of the init process]:
The document describes Genode's configuration concept, the routing of
service requests, and the expression of mandatory access-control policies.
:[http://genode.org/community/wiki - Wiki]:
The platform-specific Wiki pages for L4/Fiasco, L4ka::Pistachio, NOVA,
Codezero, Fiasco.OC, and OKL4 have been updated to reflect the new flows of
working with the respective base platforms.
Base framework
##############
The RPC API for performing procedure calls across process boundaries
introduced with the version 11.05 was the most significant API change
in Genode's history. To make the transition from the old client-server
API to the new RPC API as smooth as possible, we temporarily upheld
compatibility to the old API. Now, the time has come to put the old
API at rest. The changes that are visible at API level are as follows:
* The old client-server API in the form of 'base/server.h' is no more.
The functionality of the original classes 'Server_entrypoint' and
'Server_activation' is contained in the 'Rpc_entrypoint' class provided
via 'base/rpc_server.h'.
* When introducing the RPC API, we intentionally left the actual session
interfaces as unmodified as possible to proof the versatility of the new
facility. However, it became apparent that some of the original interfaces
could profit from using a less C-ish style. For example, some interfaces used
to pass null-terminated strings as 'char const *' rather than via a dedicated
type. The methodology of using the new RPC API while leaving the original
interfaces intact was to implement such old-style functions as wrappers
around new-style RPC functions. These wrappers were contained in
'rpc_object.h' files, e.g. for 'linux_dataspace', 'parent', 'root',
'signal_session', 'cpu_session'. Now, we have taken the chance to modernise
the API by disposing said wrappers. Thereby, the need for 'rpc_object.h'
files has (almost) vanished.
* The remaining users of the old client-server API have been adapted to the
new RPC API, most prominently, the packet-stream-related interfaces such as
'block_session', 'nic_session', and 'audio_session'.
* We removed 'Typed_capability' and the second argument of the 'Capability'
template. The latter was an artifact that was only used to support the
transition from the old to the new API.
* The 'ipc_client' has no longer an 'operator int'. The result of an IPC can
be requested via the 'result' function.
* We refined the accessors of 'Rpc_in_buffer' in 'base/rpc_args.h'. The
'addr()' has been renamed to 'base()', 'is_valid_string()' considers the
buffer's capacity, and the new 'string()' function is guaranteed to return a
null-terminated string.
* We introduced a new 'Rm_session::Local_addr' class, which serves two
purposes. It allows the transfer of the bit representation of pointers across
RPC calls and effectively removes the need for casting the return type of
'Rm_session::attach' to the type needed at the caller side.
* The 'Connection' class template has been simplified, taking the session
interface as template argument (rather than the capability type). This change
simplified the 'Connection' classes of most session interfaces.
* The never-used return value of 'Parent::announce' has been removed. From the
child's perspective, an announcement always succeeds. The way of how the
announcement is treated is entirely up to the parent. The client should never
act differently depending on the parent's policy anyway.
* The new 'Thread_base::cap()' accessor function allows obtaining the thread's
capability as used for the argument to CPU-session operations.
Operating-system services and libraries
#######################################
Dynamic linker
==============
As a follow-up to the major revision of the dynamic linker that was featured
with the previous release, we addressed several corner cases related to
exception handling and improved the handling of global symbols.
The dynamic linker used to resolve requests for global symbols by handing out
its own symbols if present. However, in some cases, this behaviour is
undesired. For example, the dynamic linker contains a small set of libc
emulation functions specifically for the ported linker code. In the presence of
the real libc, however, these symbols should never be considered at all. To
avoid such ambiguities during symbol resolution, the set of symbols to be
exported is now explicitly declared by the white-list contained in the
'os/src/lib/ldso/symbol.map' file.
We changed the linkage of the C++ support library ('cxx') against dynamic
binaries to be consistent with the other base libraries. Originally, the 'cxx'
library was linked to both the dynamic linker and the dynamic binary, which
resulted in subtle problems caused by the duplication of cxx-internal data
structures. By linking 'cxx' only to the dynamic linker and exporting the
'__cxa' ABI as global symbols, these issues have been resolved. As a positive
side effect, this change reduces the size of dynamic binaries.
C++ exception handling in the presence of shared libraries turned out to be
more challenging than we originally anticipated. For example, the
'_Unwind_Resume' symbol is exported by the compiler's 'libsupc++' as a hidden
global symbol, which can only be resolved when linking this library to the
binary but is not seen by the dynamic linker. This was the actual reason of why
we used to link 'cxx' against both dynamic binaries and shared libraries
causing the problem mentioned in the previous paragraph. Normally, this problem
is addressed by a shared library called 'libgcc_s.so' that comes with the
compiler. However, this library depends on glibc, which prevents us from using
it on Genode. Our solution is renaming the hidden global symbol using a
'_cxx__' prefix and introducing a non-hidden global wrapper function
('__cxx__Unwind_Resume' in 'unwind.cc'), which is resolved at runtime by the
dynamic linker.
Another corner case we identified is throwing exceptions from within the
dynamic linker. In contrast to the original FreeBSD version of the dynamic
linker, which is a plain C program that can never throw a C++ exception,
Genode's version relies on C++ code that makes use of exceptions. To support
C++ exceptions from within the dynamic linker, we have to relocate the
linkers's global symbols again after having loaded the dynamic binary. This
way, type information that is also present within the dynamic binary becomes
relocated to the correct positions.
Block partition server
======================
The new block-partition server uses Genode's block-session interfaces as both
front and back end, leading to the most common use case where this server will
reside between a block driver and a higher level component like a file-system
server.
At startup, the partition server will try to parse the master boot record (MBR)
of its back-end block session. If no partition table is found, the whole block
device is exported as partition '0'. In the other case, the MBR and possible
extended boot records (EBRs) are parsed and offered as separate block sessions
to the front-end clients. The four primary partitions will receive partition
numbers '1' to '4' whereas the first logical partition will be assigned to '5'.
The policy of which partition is exposed to which client can be expressed
in the config supplied to the 'part_blk' server. Please refer to the
documentation at 'os/src/server/part_blk/README' for further details. As an
illustration of the practical use of the 'part_blk' server, you can find a run
script at 'os/run/part_blk.run'.
Skeleton of text terminal
=========================
As part of the ongoing work towards using interactive text-based GNU software
on Genode, we created the first bits of the infrastructure required for
pursuing this quest:
The new terminal-session interface at 'os/include/terminal_session/' is the
designated interface to be implemented by terminal programs.
After investigating the pros and cons of various terminal protocols and
terminal emulators, we settled for implementing a custom terminal emulator
implementing the Linux termcap. This termcap offers a reasonable small set of
commands while providing all essential features such as function-key support
and mouse support. Thanks to Peter Persson for pointing us to the right
direction! The preliminary code for parsing the escape sequences for the Linux
termcap is located at 'gems/include/terminal/'.
We have created a simplistic terminal service that implements the
terminal-session interface using a built-in font. Please note that the
implementation at 'gems/src/server/terminal/' is at an early stage. It is
accompanied by a simple echo program located at 'gems/src/test/terminal_echo'.
Device drivers
##############
USB HID and USB storage
=======================
We replaced the former DDE-Linux-based USB-related driver libraries (at the
'linux_drivers/' repository) by a single USB driver server that offers the
'Input' and 'Block' services. This enables us to use both USB HID and USB
storage at the same time. The new USB driver is located at
'linux_drivers/src/drivers/usb/'.
For using the USB driver as input service (supporting USB HID), add the
'<hid/>' tag to the 'usb_drv' configuration. Analogously, for using the driver
as block service, add the '<storage/>' tag. Both tags can be combined.
For testing the USB stack, the 'linux_drivers' repository comes with the run
scripts 'usb_hid.run' and 'usb_storage.run'.
ATA read/write support
======================
The ATAPI driver has been extended to support IDE block devices for both
read and write transactions. To use the new facility, supply 'ata="yes"'
as XML attribute to the config node of 'atapi_drv'. Please note that this
driver was primarily tested on Qemu. Use it with caution.
SATA driver
===========
The new SATA driver at 'os/src/drivers/ahci/' implements the block-driver
API ('os/include/block'), thus exposing the block-session interface as
front-end. AHCI depends on Genode's PCI driver as well as the timer server. For
a usage example see: 'os/run/ahci.run'.
Limitations and known issues
----------------------------
Currently, the server scans the PCI bus at startup and retrieves the first available
AHCI controller, scans the controller ports and uses the first non-ATAPI port
where a device is present.
On real hardware and on kernels taking advantage of I/O APICs (namely NOVA and
Fiasco.OC) we still lack support for ACPI parsing and thus for interrupts,
leading to a non-working driver.
SD-card driver
==============
The first fragments of our SD-card driver that we introduced with the previous
release have been complemented. The new SD-card driver located at
'os/src/drivers/sd_card/' implements the block-session interface by using
MMC/SD-cards and the PL180 controller as back end. Currently the driver
supports single-capacity SD cards. Therefore, the block file for Qemu should
not exceed 512 MB. Because the driver provides the generic block-session
interface, it can be combined with the new 'libc_ffat' plugin in a
straight-forward way. To give the driver a quick spin, you may give the
'libports/run/libc_ffat.run' script on the 'foc_pbxa9' platform a try.
ARM Realview PL011 UART driver
==============================
The new PL011 UART driver at 'os/src/drivers/uart/' implements the LOG session
interface using the PL011 device. Up to 4 UARTs are supported. The assignment
of UARTs to clients can be defined via a policy supplied to the driver's config
node. For further information, please refer to the README file within the
'uart' directory.
Libraries and applications
##########################
Hello tutorial
==============
The 'hello_tutorial/' repository contains a step-by-step guide for building
a simple client-server scenario. The tutorial has been rewritten for the new
RPC API and is now complemented by a run script for testing the final scenario
on various base platforms.
C and C++ runtimes
==================
:Support for standard C++ headers:
Triggered by public demand for using standard C++ headers for Genode applications,
we introduced a generally usable solution in the form of the 'stdcxx' library
to the 'libc' repository. The new 'stdcxx' library is not a real library. (you
will find the corresponding 'lib/mk/stdcxx.mk' file empty) However, it comes
with a 'lib/import/import-stdcxx.mk' file that adds the compiler's C++ includes
to the default include-search path for any target that has 'stdcxx' listed in
its 'LIBS' declaration.
:Libc back end for accessing VFAT partitions:
The new 'libc_ffat' libc plugin uses a block session via the ffat library. It
can be used by a Genode application to access a VFAT file system via the libc
file API. The file-system access is performed via the 'ffat' library. To
download this library and integrate it with Genode, change to the 'libports'
repository and issue the following command:
! make prepare PKG=ffat
For an example of how to use the libc-ffat plugin, please refer to the run
script 'libports/run/libc_ffat.run'. The source code of the test program can be
found at 'libports/src/test/libc_ffat/'.
Qt4
===
Qt4 version 4.7.1 has been enabled on ARMv6-based platforms, i.e., PBX-A9 on
Fiasco.OC. The support comprises the entire Qt4 framework including qt_webcore
(Webkit).
L4Linux
=======
L4Linux enables the use of one or multiple instances of Linux-based operating
systems as subsystems running on the Fiasco.OC kernel. The Genode version of
L4Linux has seen the following improvements:
:Kernel version: has been updated to Linux 2.6.39.
:ARM support: The L4Linux kernel can be used on ARM-based platforms now.
The PBX-A9 platform is supported via the 'l4linux.run' script as found
at 'ports-foc/run/'. Please find more information at 'ports-foc/README'.
:Genode-specific stub drivers outside the kernel tree:
The stub drivers that enable the use of Genode's services as virtual
devices for L4Linux have been moved outside the kernel patch, which
makes them much easier to maintain. These stub drivers are located
under 'ports-foc/src/drivers/'.
Platform support
################
All base platforms are now handled in a unified fashion. Downloading 3rd-party
source code is performed using the 'prepare' rule of the 'Makefile' provided by
the respective kernel's 'base-<platform>' repository. Once, the platform's base
repository is prepared, the kernel can be built directly from the Genode
build directory using 'make kernel'. All base platforms are now supported by
Genode's run mechanism that automates the tasks of system integration and
testing. For more details about each specific kernel, please revisit the
updated documentation within the respective 'base-<platform>/doc/' directory.
:L4/Fiasco:
The kernel has been updated to revision 472, enabling the use of recent
GNU tool chains.
:Fiasco.OC:
The kernel as been updated to revision 36, which remedies stability problems
related to interaction of the IPC path with thread destruction. The new version
improves the stability of highly dynamic workloads that involve the frequent
creation and destruction of subsystems. However, we experienced the new kernel
version to behave instable on the x86_64 architecture. If you depend on x86_64,
we recommend to temporarily stick with Genode 11.05 and Fiasco.OC revision 31.
:L4ka::Pistachio:
The kernel has been updated to revision 803, enabling the use of recent
versions of binutils.
:OKL4:
OKL4v2 is showing its age. Apparently, the use of the original distribution
requires tools (i.e., python 2.4) that do not ship with current Linux
distributions anymore. This makes it increasingly difficult to use this kernel.
Still, we find ourselves frequently using it for our day-to-day development. To
streamline the use of OKL4v2, we have now incorporated the kernel compilation
into the Genode build system and thereby weakened the kernel's dependency on
ancient tools. However, we decided to drop support for OKL4/ARM for now. We
figured that the supported GTA01 platform is hardly used anymore and hard to
test because it is unsupported by Qemu. Newer ARM platforms are supported by
other kernels anyway.
:Codezero:
Even though B-Labs apparently abandoned the idea of developing the Codezero
kernel in the open, we adapted Genode to the kernel's most recent Open-Source
version that is still available at the official Git repository. Furthermore,
the kernel is now fully supported by Genode's new 'make prepare' procedure and
run environment. Therefore, run scripts such as 'run/demo' can now easily be
executed on Codezero without the need to manually configure the kernel.
Note that, for now, we have disabled Codezero's capabilities because they do
not allow the assignment of device resources. Consequently, 'sys_map' fails for
MMIO regions when performing the capability check (calling 'cap_map_check').
Furthermore, the current version of the kernel requires a workaround for a
current limitation regarding the definition of a thread's pager. At some point,
Codezero abandoned the facility to define the pager for a given thread via the
exregs system call. Instead, the kernel hard-wires the creator of the thread as
the thread's pager. This is conflicting with Genode's way of creating and
paging threads. In the current version of Genode for this kernel, all threads
are paged by one thread (thread 3 happens to be the global pager) within core.
As a workaround to Codezero's current limitation, we define thread 3 to be the
pager of all threads. The patch of the upstream code is automatically being
applied by the 'make prepare' mechanism.
Build system and tools
######################
In addition to the major change with respect to the integration of the various
base platforms, Genode's tool support received the following incremental
improvements:
Build system
============
:Simplification of 'create_builddir' tool:
The 'create_builddir' tool has been relocated from
'tool/builddir/create_builddir' to 'tool/create_builddir' to make it more
readily accessible. Furthermore, we simplified the usage of the tool by
removing the mandatory 'GENODE_DIR' argument. If not explicitly specified, the
tool deduces 'GENODE_DIR' from the its known location within the Genode source
tree.
:Booting from USB sticks:
For most x86-based base platforms, their respective run environments execute
Genode from an ISO image via Qemu. Naturally, such an ISO image can be burned
onto a CD-ROM to be used to boot a real machine. However, booting from CD-ROM
is slow and optical drives are becoming scarce. Therefore we changed the
procedure of creating ISO images to support writing the resulting images to a
USB stick. Under the hood, the boot mechanism chain-loads GRUB via ISOLinux.
The files to implement the boot concept are located at 'tool/boot/'.
:Support for source files in target sub directories:
Until now, the 'SRC_*' declarations in target description files contained
a list of plain file names. The location of the files within the directory
tree had to be defined via 'vpath'. This led to inconveniences when building
3rd-party code that contains files with the same name at different subdirectories.
To resolve such an ambiguity, the target had to be decomposed into multiple
libraries each building a different set of subdirectories. To make the
build system more convenient to use, we have now added support for specifying
source codes with a relative pathname. For example, instead of using
! SRC_CC = main.cc addon.cc
! vpath addon.cc $(PRG_DIR)/contrib
we can now use
! SRC_CC = main.cc contrib/addon.cc
Automated testing across multiple kernels
=========================================
To execute one or multiple test cases on more than one base platform, we
introduced a dedicated tool located at 'tool/autopilot'. Its primary purpose is
the nightly execution of test cases. The tool takes a list of platforms and a
list of run scripts as arguments and executes each run script on each platform.
The build directory for each platform is created at
'/tmp/autopilot.<username>/<platform>' and the output of each run script is
written to a file called '<platform>.<run-script>.log'. On stderr, autopilot
prints the statistics about whether or not each run script executed
successfully on each platform. If at least one run script failed, autopilot
returns a non-zero exit code, which makes it straight forward to include
autopilot into an automated build-and-test environment.

1008
doc/release_notes-11-11.txt Normal file
View File

File diff suppressed because it is too large Load Diff