diff --git a/doc/release_notes-20-08.txt b/doc/release_notes-20-08.txt new file mode 100644 index 0000000000..7c843a64c6 --- /dev/null +++ b/doc/release_notes-20-08.txt @@ -0,0 +1,1015 @@ + + + =============================================== + Release notes for the Genode OS Framework 20.08 + =============================================== + + Genode Labs + + + +There are two overarching themes of Genode version 20.08: Increasing the +weight of native work loads, and strengthening the system's resilience against +driver failures. + +With native work loads, we are speaking of software executed directly on +Genode without relying on virtual machines. Compared to static systems or +hypervisor scenarios, such work loads are brutally unforgiving when it comes +to the quality of the POSIX runtime, the performance of Genode's protocol +stacks, and the economics of the porting of software. By bringing the +*Chromium* web engine alive, we expose Genode to one of the most heavy-weight +commodity software stacks in existence. We are thrilled to report in Section +[Improved Qt5 integration and work flows] that the Chromium-based Falkon web +browser can be hosted on Sculpt OS now. The collateral effects of this work +are at least as valuable as the particular application: Improved work flows +for porting large software projects, and covering many formerly hidden corner +cases of the C and C++ runtimes. + +With [https://genode.org/download/sculpt - Sculpt OS], the resilience of +Genode in the event of failing graphics or input drivers came into focus. +Section [The GUI stack, restacked] describes a complex surgery that puts +the *low-level GUI stack* upside down, paving the ground for exciting features +like swapping out or updating drivers on the fly without reboot. + +Besides these main topics, the current release features the continuation of of +two long-term projects, namely the CBE block encrypter and the profound +support of 64-bit ARM devices. +Section [Consistent Block Encrypter] describes the CBE's new *pluggable +crypto* and trust anchor facilities. +Section [Device drivers] goes into detail about the steadily emerging *driver +landscape on 64-bit ARM*. + + +The GUI stack, restacked +######################## + +The current release breaks free from several limitations of Genode's low-level +GUI stack. This architectural change was preceded by a two-years planning and +exploration phase, spinning the following goals in our heads: + +* Achieving resilience against failing device drivers. Graphical applications + on Genode shall be able to survive driver failures. + We strive to keep overly complex device drivers out of the trusted computing + base of security-critical applications. + +* Enabling the principal support for screen capturing and remote desktop + scenarios without compromising the security and privacy of the user. + +* Paving the ground for sophisticated multi-head scenarios in the future, + e.g., using more than one graphics card. + +* Starting, killing, and updating drivers at runtime without rebooting the + system. + +* Optimizing the throughput of the pixel data path and the input latency, + aiming at a highly responsive interactive experience without tearing + artifacts. + +The key idea and plan for this change has been laid out in the dedicated article +[https://genodians.org/nfeske/2020-06-23-gui-stack - Restacking the GUI stack]. +We executed the steps outlined in the article just in time for the current +release. + + +Consistent naming +================= + +The starting point of Genode's GUI stack was the nitpicker GUI server, +introduced in 2006 and based on prior research of the Genode developers +([https://www.genode-labs.com/publications/secure-gui-2009.pdf - PDF]). +Back then, the name of the implementation somehow proliferated throughout +Genode from the API level, over the configuration level, up to the naming of +related components. Over the years, the jargon crept into the project without +much thought. This had to stop and now was the right time. The renaming +affects the following areas. + +:Source tree: + + The _include/nitpicker_session_ headers moved to _include/gui_session_. + Those headers are not tied to the specific nitpicker implementation but are + much more general. For example, the window manager provides an alternative + implementation of the same interface. + + The change also affects the corresponding depot API archive accordingly. + +:Configuration and run scripts: + + The "Nitpicker" service is now called "Gui" service. The upper/lower casing + follows the convention for services provided outside of Genode's core + component. + +:Packaged runtime definitions: + + The '' nodes in runtime files are now named ''. + +:API level: + + The C++ namespace 'Nitpicker' has been renamed to 'Gui'. + +:Component names: + + The nit_fb server has become the gui_fb server. + The nit_fader has become the gui_fader. + + +Streamlined pixel format +======================== + +Genode's framebuffer session interface was traditionally based on the RGB565 +pixel format. We have now changed the format to 32-bit XRGB where the X part +is ignored. We updated all affected libraries, graphical applications, and +device drivers accordingly. + +This change also affects the users of the drivers_interactive packages. Now, +we generally assign 64 MiB RAM and 1500 caps to the drivers subsystem, which +is sufficient for covering high resolutions at 32 bits per pixel and to +accommodate multi-component USB HID input stacks. + + +New "Capture" and "Event" session interfaces +============================================ + +Traditionally, Genode's driver model for framebuffer and input drivers +followed a layered architecture. Each framebuffer driver naturally provided a +"Framebuffer" service and each user-input driver provided an "Input" service. +At the next layer, the nitpicker GUI server would use those services as a +client and, in turn, provide a higher-level "Nitpicker" service (which has now +been renamed to "Gui"). In this architecture, higher levels naturally depend +on all lower levels, specifically the framebuffer and input drivers at the +bottom. The interplay of the components looks as follows. The arrows denote +client-server relationships where the arrow points from the client to the +server. The drivers and the nitpicker GUI server are displayed in red because +they are servers. The GUI applications at the right are nitpicker clients. +Nitpicker, in turn, is a client of the framebuffer and input drivers. + +[image nitpicker_orig] + +This architecture is flawed because it makes the trusted low-complexity GUI +server (nitpicker) depend on high-complexity and potentially flaky device +drivers. If a driver fails, the entire GUI stack - up to the applications - +suffers. To put the risk into perspective: The binary of the nitpicker GUI +server incorporates about 11,000 lines of code, which includes the Genode API. +The actual nitpicker code is about 3000 lines of plain C++. +In contrast, the Intel framebuffer driver comprises more than 122,000 lines of +code (sloccount). Code complexity equals risk. The dependency of nitpicker on +the Intel framebuffer driver is indefensible. For input devices like USB HID, +the situation looks very similar. + +To solve this dilemma, we had to turn the client-server relationship between +the GUI server and the drivers upside down. This approach is enabled by two +new service interfaces. The new "Event" session replaces the former input +session. The new "Capture" session replaces the former framebuffer session. +The information that is flowing through those interfaces remains the same - +input events and pixels - but the direction is reversed. An event client +propagates input events to the server. A capture client obtains pixel data +from the server. + +[image nitpicker_restacked] + +In the new architecture, nitpicker remains as the only server, providing the +"Event", "Capture", and "Gui" interfaces. It does no longer depend on the +well-being of any driver. This is reflected by the node of nitpicker +when integrating the GUI server into an init configuration: + +! +! +! /provides> + +To retain support of cascaded usage scenarios of the GUI server, nitpicker can +still be instructed to request a framebuffer session or input session by +specifying the '' attributes 'request_framebuffer="yes"' and +'request_input="yes"'. If specified, nitpicker requests sessions for an +"Input" and a "Framebuffer" service upon startup and uses those as +input/output back end. Those services are usually provided by the gui_fb +component. Note that this facility may eventually be replaced by a +'request_gui' attribute (requesting a GUI session), eliminating the notion of +framebuffer and input services altogether. + + +Turning all framebuffer and input drivers into nitpicker clients +================================================================ + +Following the introduction of the new capture-session interface, we replaced +the use of the "Framebuffer" session interface by the new "Capture" session +interface in all framebuffer drivers. The new versions of all drivers had to +be tested on the respective hardware. Generally, the drivers have become +simpler. + +Analogously to the graphics drivers, all input drivers had to be reworked to +operate as event clients instead of input servers. This came down to adjusting +and testing the variety of PS/2 drivers and USB HID drivers on all supported +platforms. As a minor side note, the server/acpi_input pseudo driver has +turned into the app/acpi_event application. Similar to the framebuffer +drivers, the transition from input servers to event clients generally made the +drivers simpler. + +The drivers_interactive packages for various boards have changed their public +interface. The drivers subsystem no longer provides "Framebuffer" and "Input" +services but needs a valid route to the "Capture" and "Event" services +provided by nitpicker. It goes without saying that this change affected all +run scripts that make use of nitpicker. + + +Replacing the input filter with an event filter +=============================================== + +The transition from the "Input" session to the "Event" session is not limited +to the drivers only but also affected Genode's input-filtering mechanism. The +functionality of the input_filter is now provided by the event_filter. The +event filter requests only one "Event" session as destination for the filter +results, which is usually routed to the nitpicker GUI server. It provides an +"Event" service to which any number of event sources can connect. + +The configuration of the filter chain remains almost the same. Only the +declaration of the '' nodes is no longer needed. Instead, the +configuration must specify '' nodes, which define the mapping of +"Event" clients (event sources) to the inputs used in the filter chain. + + +Improved Qt5 integration and work flows +####################################### + +Streamlined build-system integration +==================================== + +Up to now, Qt libraries and applications for Genode had been built using the +Genode build system. The Qt library build files in the _libports/lib/mk_ +directory were in part generated with a shell script from the GNU make files +which the _qmake_ tool had generated when building Qt for Linux. Also there +was some rudimentary support for building Qt applications from qmake project +files by just interpreting the qmake project files as GNU make files and +translating the relevant qmake variables to corresponding Genode build-system +variables. + +But this approach was not feasible anymore when building the 'qtwebengine' +module (the successor of the deprecated 'qtwebkit' module), which is based on +the Chromium web engine source code and mostly built with the Ninja build +system on officially supported platforms instead of qmake. + +We also wanted to make it easier for Genode users to build Qt applications, in +particular by using the [https://github.com/nfeske/goa - goa] tool. It would +be better to be able to actually process qmake project files with the qmake +tool instead of interpreting them as GNU make files and aborting when a +project file contains qmake-specific code not understood by GNU make. + +For these reasons, we reworked the Qt build process such that it now uses +qmake with a tailored Genode platform configuration for building both Qt +libraries and applications. + +If you have already built a Qt application for Genode and want to continue +using it with the new release and you cannot use the goa tool yet (for example +because additional Qt libraries cannot be built with goa or because other +architectures than x86_64 are not supported by goa currently), you need to +update the 'target.mk' file (and any run scripts) of your project. For the +details, please have a look at the updated Qt example projects in the +'libports' repository and feel free to ask on the Genode mailing list if any +problems arise with the conversion. It is also necessary to rebuild the Qt +tool chain with the 'tool/tool_chain_qt5' script to get the qmake tool built +and installed. + +While converting the run scripts of the Qt example projects, we switched to +usage of the 'drivers_interactive' package, which is the preferred way to +start drivers for interactive scenarios nowadays. A downside might be that +network or storage drivers are not loaded this way. If these features are +needed for a Qt project, the suggested solution is to use goa or to run the +scenario with Sculpt OS instead, if possible. + +Another Qt-related feature in this release is that the 'qt5_component' library +now extracts command line arguments and environment variables from the +'config' ROM, like the 'posix' library already did for non-Qt applications. + + +Chromium engine and Falkon web browser +====================================== + +Since the previously ported 'Arora' web browser has not been maintained +upstream for years and the 'qtwebkit' Qt module it depended on has also been +deprecated in the meantime, it was eventually time to try to port the newer +Chromium-based 'qtwebengine' module to Genode as well as to find a new and +maintained qtwebengine-based web browser which could replace Arora. At first +glance, porting qtwebengine to Genode appeared to be quite a challenging task, +since officially only Linux, Windows, and macOS are supported and most of the +qtwebengine code, which also contains a huge amount of third-party libraries, +is not built with qmake for the most part like other Qt modules, but with the +Ninja build system. + +Fortunately, the situation turned out to be less problematic than expected, at +least with regard to the initial goal of getting a lightweight web browser +running and displaying regular web sites comparably to Arora. One reason for +that is that there already existed a qtwebengine port for FreeBSD, which +resolved most of the possible libc compatibility problems and thus served as +the basis for the Genode port. There were still some operating system +specifics in the code, which had to be dealt with, though, like supporting +shared memory with 'mmap' or cache maintenance for JIT-compiled JavaScript +code. Advanced web-engine features like multimedia support are disabled at the +moment, also some security-related features like the verification of server +certificates (requires a port of libnss), multi-processing, or sandboxing +remain unused at the current time. + +The build system issue could eventually be resolved by adding a Genode +platform back end for qmake and passing the correct compiler options to the +Chromium build system. + +As qtwebengine-based web browser, we chose the [https://www.falkon.org - Falkon] +browser, which was previously known as 'QupZilla'. Even though the current +performance and memory footprint clearly call for further optimization work on +our side, it looks quite promising so far: + +[image falkon_screenshot] + +For the upcoming release of the a Sculpt OS in September, Falkon will be +offered as a ready-to-use package. + +The Arora web browser and qtwebkit module are no longer available with this +release. + + +Base framework and OS-level infrastructure +########################################## + +Base API/ABI changes +==================== + +Synchronization primitives +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +With the current release, we wrap up the +[https://genode.org/documentation/release-notes/20.05#Migration_from__Lock__to__Mutex__and__Blockade_ - migration] +of the code base from the 'Lock' to the 'Mutex' and 'Blockade' types to +improve diagnostics and clear the path towards future performance +optimizations. + + +Moved atexit handling from base library to C runtime +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Traditionally, Genode's base library took care of the execution of 'atexit' +handlers as this mechanism was regarded as fundamental to both C and C++. In +particular, the C++ compiler automatically generates calls to the +'__cxa_atexit' function of the C++ ABI for objects instantiated as local +static variables so that the destructors of such objects are called at the +exit of the program. + +With the transition to Genode's modern component API (introduced in version +[https://genode.org/documentation/release-notes/16.05#New_component_API - 16.05]), +this mechanism became irrelevant for plain Genode components. But for +higher-level components that rely on the C runtime, in particular POSIX +applications, this mechanism remained crucial. Hence, we did not question the +presence of the atexit mechanism in Genode's base library for a long time. + +However, we ultimately realized that the atexit functionality must be +moved from the base library into the C runtime to fully adhere the +[https://genode.org/documentation/release-notes/17.02#New_execution_model_of_the_C_runtime - execution model] +of the C runtime. Atexit handlers may perform I/O such as the closing and +syncing of files after all. + +This change has the positive side effect of reducing the complexity of the +base library. Furthermore, it overcomes the limitations of the formerly +statically dimensioned array of atexit handlers, which was wasteful for plain +Genode components, yet insufficient for complex POSIX applications. + +Note that this change deliberately renders the atexit mechanism for plain +Genode components (without C runtime) ineffective. Genode components that used +to rely on the 'genode_exit' function for calling cleanup code must be changed +to explicitly call cleanup code. + +This change also removes the implicit call of 'genode_exit(1)' by 'abort', +which is triggered by, i.e., uncaught exceptions. Scenarios that relied on +the former behavior for component monitoring should be changed to use Genode's +[https://genode.org/documentation/release-notes/18.11#Component_health_monitoring - heartbeat monitoring] +mechanism instead. + +As another minor but still noteworthy detail, shared-library destructors are +no longer called via the atexit mechanism but explicitly by the dynamic +linker. This slightly changes the call order of destructors: Functions marked +as destructors are called after the libc-managed atexit handlers now. + + +Removal of deprecated interfaces +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Unsafe 'Xml_node' methods +------------------------- + +The unsafe 'Xml_node' methods that were marked as deprecated in the +[https://genode.org/documentation/release-notes/20.05#Deprecation_of_unsafe__Xml_node__methods - previous release] +are removed now. + + +Cancel-blocking interface +------------------------- + +Closely related to the revised synchronization primitives mentioned above, we +removed the outdated 'Cancelable_lock' API along with the underlying +cancel-blocking mechanism, which had become obsolete by now. In past times, +this mechanism was employed to cleanly wind down threads that are blocked at +their destruction time. The move of Genode's API design away from blocking RPC +to asynchronously working components marginalized the problem to the point +where it ceased to exist. + + +Delayed dispatch of 'Rpc_entrypoint' +------------------------------------ + +Similar to the cancel-blocking mechanism, the ability to delay the dispatching +of RPC requests until an explicit call of 'Rpc_entrypoint::activate' is an API +relic from a time when each RPC server was multi-threaded. The modern flow of +execution has become much simpler, eliminating the need for this feature. + + +C runtime +========= + +Serialized VFS access +~~~~~~~~~~~~~~~~~~~~~ + +Unsurprisingly, the VFS is an integral part of the C runtime as _everything is +a file_ in the UNIX philosophy. The basis for the libc VFS is the VFS library +and its versatile plugin concept. As the libc (and other users of the library) +may employ multiple threads, which could use the VFS, the integrity of data +structures, esp. those of more complex plugins like the network stack in +_vfs_lwip_, must be protected from concurrent access. From the beginning, we +planned to move the serialization of accesses out of the plugins into code +that uses the VFS library but missed to completely achieve this goal with the +libc execution model. With Genode 20.08, we transform the lower parts of the +libc into an execution monitor that ensures the aspired serialized processing +of VFS operations. This paves the way towards the removal of synchronization +measures in all VFS plugins with the upcoming release. + + +Removal of libc_pipe LibC plugin +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The VFS pipe plugin was introduced in +[https://genode.org/documentation/release-notes/19.11#VFS_plugin_for_emulating_POSIX_pipes - Genode 19.11] +and matured into a suitable replacement for the aged libc_pipe plugin library. +Now, we finalize this process by removing the old libc plugin from Genode and +adapt all run scripts (most prominently those for Qt5 examples). Users can +migrate to the VFS plugin with the following changes to their component +configurations. + +* The libc must be configured to access the pipe mechanism from a VFS + path by the 'pipe="..."' configuration attribute like follows. + ! + +* The pipe plugin must be instantiated in the VFS at the desired path. + ! ... + ! + ! + +* Run scripts no longer need to integrate _libc_pipe.lib.so_ into the + boot modules but _vfs_pipe.lib.so_. + + +Placing pthreads on CPUs +~~~~~~~~~~~~~~~~~~~~~~~~ + +Up to now, pthreads experienced no affinity configuration on creation time and +ran implicitly on the first CPU available to the component only. With this +release, we add a placement strategy and configuration option to the C runtime +to enable the tuning of CPU placement for pthreads of libc components. The +default placement strategy is 'all-cpus', which means pthreads are assigned to +the available CPUs. round-robin. + +The new '' libc configuration sub-node can be used +to manually place pthreads according to their 'id' on 'cpu' as follows. + +! +! +! +! . +! . +! . +! . +! +! + +Note, the current placement-strategy implementation is limited to 32 pthreads. +If more pthreads are running simultaneously further threads are placed on CPU +0. + + +Standard C++ library +==================== + +We enabled support for thread-related features like 'std::thread' or +'std::mutex' in the stdcxx library. + + +Ada/SPARK support +================= + +With the development of the [Consistent Block Encrypter], Genode's Ada/SPARK +support was used in the context of a rather complex software project for the +first time. Therefore, we felt the necessity to make up our minds about a +generally accepted Ada/SPARK coding-style as long as this step still doesn't +entail the painstaking adaption of large amounts of code. + +Fortunately, the GNAT compiler flags assist developers a lot in this affair. +That said, we added the consideration of two new environment Make-variables to +the Genode tool chain: 'CC_ADA_WARN' and 'CC_ADA_WARN_STRICT'. Analogously to +the C++ pendants ('CC_CXX_WARN' and 'CC_CXX_WARN_STRICT'), the variable +'CC_ADA_WARN' denotes the common warning-configuration of the compiler for +Ada/SPARK whereas 'CC_ADA_WARN_STRICT' applies additional configuration for a +much higher level of strictness by default. This strict mode might be +deactivated locally for code which is not subject to the Genode coding style - +e.g., ported third party code - by adding the following line to the +corresponding build description files. + +! CC_ADA_WARN_STRICT = + +The common, less strict warning mode for Ada/SPARK merely enables all warning +messages with the '-wa' flag. The strict mode, in addition to it, causes GNAT +to treat warnings like errors and to refuse compiling affected code. In +addition to that, the strict mode activates a broad variety of checks for +potential pitfalls and style consistency using the configuration +'-gnatyyBdSux', which is compatible to the GNAT standard style. +A comprehensive description of all the checks would be beyond the scope of +this document. Please have a look at the +[https://gcc.gnu.org/onlinedocs/gnat_ugn/Style-Checking.html - GNAT style-checking reference] +instead if you're interested. + +Besides the style enforcement through compiler flags, we decided to generally +promote "Camel-Snake Casing" for Ada/SPARK code. The only exceptions from that +are abbreviations of initials that are completely upper case and language +keywords that are completely lower case. This would be a conform snippet of +code: + +! function T1_Node_XML_Tag_Open ( +! Node : Type_1_Node_Type; +! Node_Idx : Type_1_Node_Block_Index_Type; +! Show_Hsh : Boolean; +! VBA : Virtual_Block_Address_Type) +! return String; + +Another minor improvement of Genode's Ada/SPARK runtime consists in the added +support for the exponential operator on integers. + + +Operating-system components +########################### + +Consistent Block Encrypter +========================== + +With the Genode 20.05 release, our +[https://genode.org/documentation/release-notes/20.05#Feature-completeness_of_the_consistent_block_encrypter - Consistent Block Encrypter] +(CBE) reached feature completeness by adding support for +[https://genodians.org/m-stein/2020-06-07-cbe-rekeying - online rekeying] and +[https://genodians.org/m-stein/2020-06-08-cbe-resizing - online resizing]. +Furthermore, we were able to run a Linux virtual machine using a CBE device as +system partition. During the last three months, we mainly reviewed and +consolidated the code but also improved the integration in a notable way. + +One important step forward is that we implemented a new _reference_ trust +anchor. The trust anchor (TA) is a separate entity (might be a smartcard, a +USB dongle, or a piece of software) that the CBE relies on for key management +and for storing superblock hashes. In contrast to the dummy TA we used for +testing so far, the reference TA stores private key and superblock hashes +persistently through a file-system back-end and requires the user to enter a +passphrase to be unlocked. This enables the CBE to actually verify that the +CBE device is in the same state that it was left in the last time and that +only authorized users can access its data. + +Furthermore, both the crypto and the TA back end of the CBE are now integrated +through VFS plugins. This change gives the user of the CBE more flexibility in +two ways. On one hand, one can switch between integrating those entities +locally (inside the CBE driver) or remotely (in a VFS server instance isolated +from the CBE driver) by merely altering the system configuration. + +[image cbe_integration] Three different ways to integrate the CBE + +On the other hand, the implementation of the crypto entity and the trust +anchor can now be swapped out through the means of configuration alone. +For instance, let us take a look at the VFS server in the CBE test scenario +_repos/cbe/run/vfs_cbe.run:_ + +! +! +! +! +! ... +! +! ... +! +! ... +! + +We may replace the plugin employing an AES-CBC ESSIV block-cipher in this test +with one using plain memcopy (in other words: disabling block encryption) by +changing one line only: + +! +! +! +! +! ... +! +! ... +! +! ... +! + +The memcopy example may in fact be useful for contexts where only the +abilities of the CBE to manage snapshots and to ensure consistency are of +interest and encryption would only generate unnecessary overhead. However, the +primary motivation for this high modularity is the easy integration of other +cipher algorithms in the future. + +In order to make things even more approachable, the VFS-plugin-related code +between CBE, crypto entity, and trust anchor relies on generic interfaces. +This simplifies supporting other existing cryptographic algorithms and TA +types as enabling them boils down to the implementation of a handful of well +defined hook functions. + +As for the CBE demo scenario that we published with the previous Genode +release: The corresponding packages moved from the package source 'cnuke' to +'mstein'. Furthermore, the CBE demo is now based on the unaltered Genode +master branch and the new master branch in the CBE repository itself. Besides +that, for the most part, the procedure for reproducing the demo described in +the [https://genodians.org/m-stein/2020-06-12-cbe-linux-vm - demo article] and +the [https://genodians.org/m-stein/2020-06-17-cbe-linux-vm-video - demo video] +remained the same. But be prepared to find minor discrepancies. + + +Virtual file system (VFS) +========================= + +We extended the 'zero' VFS plugin to limit the amount of data - the number of +zero bytes - it will provide on plain sequential reads. The file size can be +determined by setting the 'size' configuration attribute limiting readable +seek offsets to the configured size and resulting in EOF beyond that offset. +The attribute is parsed as 'Number_of_bytes', which caps the amount to the +maximum of the addressable memory on the platform. + +This addition comes in handy when needing to provide a synthetic empty file +with a fixed size: + +! +! +! +! +! +! + + +Furthermore, we added an option to the 'terminal' VFS plugin to ignore control +characters. Usually this plugin is used in interactive configurations where +reacting to control characters, e.g. ^C, is vital. That being said, the plugin +lends itself as general transport mechanism for low-traffic data transfers. +Those transfers, however, may contain arbitrary data that could very well be +interpreted as control characters. Therefore, we introduced the 'raw' +configuration attribute. Setting the attribute instructs the plugin to pass +all data through as-is without further interpretation. + +! +! +! +! +! + + +Block server backed by VFS content +================================== + +This release adds a new component that provides access to a VFS file via the +block-session interface. Block requests will be translated to VFS requests +operating directly on the file. + +The client access is configured by setting the attributes of the 'policy' node +appropriately. The assumed block size of the underlying file must be specified +via the 'block_size' attribute. It defaults to 512 bytes. + +The block count is determined by querying the file and dividing its size by +the block size. Pseudo file systems, which do not return a proper size in +their 'stat' implementation will therefore not work. The 'writeable' attribute +denotes if the block session is allowed to perform write requests. However, +should the underlying file be read-only, such requests will nonetheless fail. +The default value is 'no'. + +The following snippet shows a configuration where the 'vfs_block' component +grants the component 'the_client' writeable access to the 'ext2_disk.img' +file. In return, this file is accessed via a file-system session. + +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! + +Since the 'vfs_block' is - by using the VFS library - flexible enough to cover +the use-cases of components like 'ram_block' and 'rom_block', we were able to +remove those components. + +For example a typical 'rom_block' use-case is implemented by using the +following configuration: + +! +! +! +! +! +! +! +! +! +! +! +! +! +! +! + +On a technical note, the server currently allows a single session only and +processes requests sequentially. + + +Device drivers +############## + +System management privilege +=========================== + +The PD session interface of the base library got extended by a +system-management call: + +! Managing_system_state managing_system(Managing_system_state const & state) + +It is used to call privileged system-management functionality, like power +management or clock settings that cannot be implemented in user-land +components. The transferred state is equivalent to the common general-purpose +CPU registers. Currently, the function is implemented by base-hw's core +component on ARMv8 to call the ARM Trusted Firmware on behalf of the PD +session client. All other kernel and architectures leave the 'managing_system' +call empty. + +A special privilege is necessary to enable a component to use the new call. +Therefore, the init component's configuration provides a new attribute in the +start node: + +! +! ... + +The new call and privilege got introduced in order to enable a platform driver +to implement power management on modern ARM SoC. + +The formerly existent, special privilege 'constrain_phys', which was used by +the platform driver to constrain physical memory to be allocated for DMA to a +specific region, is now implicit part of the 'managing_system' privilege. +Configurations must be adapted accordingly as the 'constrain_phys' attribute +is no longer supported. + +That means configurations that so far used the "constrain_phys" attribute in +the RAM resource description, like the following: + +! +! +! ... +! + +have to be changed to the following to be effective: + +! +! +! ... +! + +Platform driver +=============== + +Configuration changes +~~~~~~~~~~~~~~~~~~~~~ + +The new platform driver API that was introduced with the +[https://genode.org/documentation/release-notes/20.05#New_platform_driver_for_the_ARM_universe - previous release] +for the ARM architecture received a minor subsequent improvement, namely the +device resource description within the configuration of the platform driver: + +! +! +! +! + +is now shortened to the following: + +! +! +! +! + + +Power and clocks for i.MX 8M Quad SoC +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +To drive peripherals of the i.MX 8M Quad Evaluation Kit and the +[https://www.crowdsupply.com/mnt/reform - MNT Reform 2] appropriately, we +introduce a new platform driver specific for the i.MX 8M Quad SoC used by both +platforms. + +The new driver supplements the API of the generic ARM platform driver +introduced in release 20.05 with power and clock management specifically for +this SoC. It comprises all power and clock settings available in the SoC. +Single power domains and clocks can be assigned to devices. Whenever the +client of the platform driver acquires a device, its power domains and clocks +are enabled implicitly, and all clocks are set accordingly. When the device +gets released, all power-domains and clocks are disabled again. + +The following USB host-controller example for the i.MX 8M Quad EVK illustrates +the additional configuration options: + +! +! +! +! +! +! +! +! +! +! +! +! + +Most clock settings can be derived directly by looking at device tree sources +for the corresponding board and peripheral, e.g., in the Linux kernel tree. +But be aware that the clock names are aligned to the i.MX 8M Quad Applications +Processors Reference Manual and not the Linux code. + +As a side effect of the clock settings work, the CPU cores of the i.MX 8M Quad +SoC are now running with the full-rate of 1.5 GHz. + + +New platform driver for Raspberry Pi +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +As some driver components like the USB host-controller driver share one code +base across a lot of different boards, SoCs, and architectures. We like to +unify the API they are dependent on, including the platform driver's API. +Therefore, the already existent platform driver for Raspberry Pi 1 got +modernized to take advantage of the new platform driver API for ARM introduced +with the previous release. + +However, we did not achieve to change all drivers at once. Therefore, we kept +the old platform driver still used by some drivers for now. To prevent name +clashes, the new driver is named "rpi_new_platform_drv" until the legacy +driver "rpi_platform_drv" vanishes. + +During this work, we unified the formerly different enumeration of interrupts +between the Fiasco.OC and base-hw kernels. + + +USB host-controller driver +========================== + +The USB host-controller driver got modified to iterate over all device +resources the platform driver provides to it, and registers those in +corresponding Linux kernel structures of the ported driver code. +This board-specific information was hard-coded beforehand. Now, the driver +uses the platform driver for x86, as well as all ARM architectures. + + +Interactive drivers support for Arndale and Panda boards +======================================================== + +In the Genode world repository the almost abandoned support for Pandaboard and +the Arndale board got revived by drivers_interactive depot packages for both +platforms. Therewith, it is now easy again to use interactive run-script +scenarios on these platforms. + + +Libraries and applications +########################## + +Migration of ported software to Genode-World +============================================ + +In the previous releases, we continuously relieved the Genode main repository +from features that receive no continuous maintenance. Rather than dropping +such features, they find a new home at the +[https://github.com/genodelabs/genode-world - Genode-World] repository. +We foster this distinction to avoid wrong expectations. Whereas the code of +the Genode main repository is subjected to rigid quality assurance at Genode +Labs, the content of the World repository is tested and maintained in a rather +ad-hoc fashion. + +The current release moves the Solo5 runtime (originally added in version +[https://genode.org/documentation/release-notes/18.11#Genode_as_a_platform_for_Mirage-OS_unikernels - 18.11]) +and the stubby DNS tool to the Genode-World repository. + + +New ported 3rd-party software in Genode-World +============================================= + +Port of SDL 2.0.14 +------------------ + +As more and more software requires SDL2, it became necessary to support it as +well in addition to the already existing support for SDL1. We therefore ported +the up-to-date SDL version to Genode. + +For the time being, its functionality is limited when compared to the SDL1 +port. At the current time, only input and non-accelerated video are supported. + +Even in its limited state, it suffices for +[https://www.genodians.org/alex-ab/2020-07-02-scrcpy - Streaming Android to Genode using Scrcpy]. + + +Port of gRPC +------------ + +_This work is a contribution by Pirmin Duss of_ +_[https://gapfruit.com - Gapfruit]. Thanks for this welcome update._ + +At gapfruit, we wanted to provide an efficient way to combine a Genode system +with existing cloud environments. For that, we ported the gRPC/protobuf +libraries and tool chain to Genode. For this release, we simplified the tool +chain so it does not depend on any packages of your Linux distribution. +Additionally, we fixed a memory leak and updated the port to gRPC version +1.28.1 and protobuf version 3.11.2. + + +Removal of orphaned 3rd-party ports +=================================== + +The dde_zircon device-driver environment was added in +[https://genode.org/documentation/release-notes/18.08#Experimental_runtime_for_Zircon-based_drivers - 18.08] +as an experiment for executing device drivers of Google's Fuchsia OS as Genode +components. However, since the initial import of the feature, it remained +unused and received no further development. As Fuchsia largely diverged from +the APIs implemented by dde_zircon in the meantime, we decided to remove the +feature. + + +Sculpt OS improvements +###################### + +The Genode-based [https://genode.org/download/sculpt - Sculpt] operating +system has been adapted to the new Genode version without major user-visible +changes. Given the fact that this release turned the low-level GUI stack +upside down (Section [The GUI stack, restacked]), this must be read as *good* +news. The GUI-stack changes have several nice side effects: + +* The elimination of color-space conversions in the pixel-data path has + noticeably improved the interactive latency. Sculpt OS has never been + as snappy as now. + +* As Sculpt's base system no longer depends on the successful initialization + of the framebuffer and user-input devices, the administrative user interface + can be started even before the low-level drivers are ready, increasing the + parallelism at boot time. Yes, Sculpt OS boots even faster now. + +* The now ubiquitous use of 32-bit pixels throughout the whole GUI stack has + largely eliminated the need for dithering, thereby improving the visual + clarity of the user interface. Dithering remains in use in some situations + though. In particular to reduce quantization artifacts for the alpha channel + of the leitzentrale overlay. But that is a feature now, not a limitation. + + +Interactive CPU affinity configuration +-------------------------------------- + +The component-configuration dialog gained the new ability to restrict the +component to a subset of CPUs. This option is available under the new +"Resource assignment" sub menu. + +[image sculpt_affinity] + +The configuration dialog shows a matrix of CPU cores where the x-axis denotes +the physical cores and the y-axis the hyperthreads. By default, all available +CPU cores are selected. The user can toggle the nodes by clicking on them. +The dialog ensures that the selection is always a rectangular area. So one +click may affect nodes other than only the clicked one. + + +Fail-safe handling of unsupported partition tables +-------------------------------------------------- + +Sculpt now uses Genode's heartbeat-monitoring mechanism to detect the failure +of part_block instances during storage discovery. This situation can happen +when a storage device contains an unsupported partitioning-table format. +In this case or other error cases, the discovery would fail to complete. +The new version of Sculpt responds to this situation by removing the +unresponsive part_block instance automatically, releasing the corresponding +device. This leaves the device available for alternative uses, e.g., for +assigning it to a virtual machine. +