Structuring the System Layer with Dataspaces (2021)
Could dataspaces be a suitable system layer foundation, perhaps replacing software like systemd and D-Bus?
An NLnet Foundation-sponsored project, part of the NGI Zero PET programme.
In this project, syndicated-actor theory will be co-developed with new system layer software in the setting of the user interface to and management of a cellular telephone.
The expected project outcomes are:
- A capability-based security model for syndicated actors
- A proof-of-concept dataspace-based system layer for a mobile phone
- A qualitative evaluation of the suitability of the Syndicated Actor model as structuring principle for system layers
Abstract and Hypothesis
The system layer (Rice 2019; Corbet 2019) is an essential but often-ignored and under-theorized part of an operating system, mediating between user-facing programs and the kernel.
Its importance lies in its role as the technical foundation for many qualities1 relevant to system security, resilience, connectivity, maintainability and usability. Despite its importance, the concept has only been recently recognised and has not received a great deal of attention. Existing implementations are thus ad-hoc, lacking solid theoretical footing.
The Syndicated Actor model of concurrency and communication combines a small number of concepts to yield succinct expression of ubiquitous system-layer features such as service naming, presence, discovery and activation; security mechanism and policy; subsystem isolation; and robust handling of partial failure. The model offers an understandable “conversational” metaphor: programs meet and converse in virtual locations, building common understanding of the state of their shared tasks.
This project will evaluate the hypothesis that
the Syndicated Actor model provides a suitable theoretical and practical foundation for system layers,
since a well-founded system layer is a necessary part of any vision of secure, securable, resilient networked personal computing.
From two layers to two-and-a-half
What is a system layer?
The term “system layer” was coined2 by Benno Rice in a 2019 talk. Here’s an excerpt from the relevant portion of Rice’s talk:3
… dynamic DHCP, IPv6 auto config, all these kinds of things are more dynamic. Time is more dynamic. Some aspects of device handling, you know, all of these things are a lot more dynamic now, and we need a way of strapping these things together so we can manage them that doesn’t involve installing 15 different packages that all behave differently.
[15:08] And so what that ends up becoming, is what I term the system layer. Which is a bunch of stuff which might be running in user space or might be running in kernel space but is providing systemic level stuff as opposed to the stuff that you’re writing or using directly. So this could include things like NetworkManager, and udev, and a whole bunch of things.
Systemd as a project ends up complementing the Linux kernel by providing all of this user space system layer.
(It’s a really good talk.) The system layer idea seems to have been latent for a long time, and only recently to have been given a name.
As I write this, I’m running Debian Linux with XFCE4. Looking at the processes running in my system, I classify (at least) the following as part of the system layer for this machine:
- xfce4 applets
It’s a rough classification; one of the expected outcomes of the project is a more rigorous analysis.
What does a system layer do?
There are some strong commonalities among the system layer programs in the list above:
- supplying services in response to static or dynamic demand
- DBus services started by service name on demand
- systemd services started by socket activation
- statically-enabled systemd services
- virtual machine and container lifecycles
- monitoring and reacting to changes in system state
- service supervision hierarchies with service restarting
- monitoring and reacting to device hotplugging
- monitoring and reacting to changes in system metrics
- monitoring and reacting to network connectivity changes
- setup and naming of devices and network routes
- giving higher-level perspectives on system state and resources
- high-level audio interface
- DBus interfaces to various resources
- centralized system configuration database
- access control
- resource allocation services
- ACL-based access control for system services and DBus objects
- inter-process communication
- DBus as a program-to-program communication bus
- email for use by system services
- naming services
- publishing names for intra-machine services on this system
- publishing names for LAN services on this system
- resolving names of intra-machine services on this system
- resolving names of services on other systems4
The Syndicated Actor model seems like a great fit!
The tasks listed above are all core strengths of syndicated actors.
Syndicated-actor programs communicating via a dataspace:
- are all about publishing eventually-consistent replication of
(such as facts about battery levels, connected devices, services needed, services available, volume levels, available networks, etc. etc.)
- offer naming and directory services, as a corollary of state replication;
- naturally increase/decrease supply of services in response to demand;
- implicitly and naturally topologically sort service dependencies at startup time;
- control access to resources via delegable, attenuable object-capabilities; and
- converse via a flexible, high-speed IPC bus.
This is what led me to propose the project.
The aim of this project is to re-express features of existing system layers using a coherent underlying vocabulary.
The goal is to demonstrate (a path to) rough parity of ability with existing systems, with improved security, understandability, resilience, etc., rather than to innovate in terms of functionality.
Syndicated actors at the system layer of a mobile phone
Alternative (i.e. non-Android) free software operating systems for mobile phones need new system-level design ideas suited to the form factor. Such operating systems are not yet well served by a robust system layer.
Therefore, I aim to build a running, if minimal, prototype system layer atop PostmarketOS for mobile phones that supports day-to-day operation of the phone and illustrates the benefits of a dataspace-style model.
Prior work on networked and federated dataspaces and on Smalltalk-based mobile user interfaces will aid in cross-language integration of portions of the system.
Putting theory into practice, and theorizing existing practice
The Syndicated Actor model is abstract, independent of application domain. Concrete interaction protocols reflecting any given area of application must be developed and debugged. Conversely, extant system layers are incompletely specified using a motley of approaches because no unifying theory has thus far been a suitable fit.
I will survey existing open systems such as the following, in order to map the landscape before venturing into it:
- systemd/udev/D-Bus/NetworkManager/dhcpcd/etc., as sketched above
- init/inetd/crond/etc., the traditional Unix system layer
- daemontools/runit/s6: service supervision software
- OpenRC/s6-rc: service manager and supervisor used in Alpine
- Android architecture components
- Erlang’s OTP, the system layer for the Erlang virtual operating system
I will then design dataspace-based interaction protocols that realize this functionality. These protocols will form the heart of the system: each component will perform one or more roles as described.5
At the same time, the protocol descriptions will serve as internal and external APIs and API documentation for the system layer. The project’s thesis predicts that dataspace protocol descriptions will be at the correct level to effectively capture the concepts intrinsic to a system layer.
Ensuring security and privacy
Existing system layers rely on single-machine approaches to security and securability that do not scale well: for example, Unix ACLs and user- and group-ID-based permissions. The theory of object capabilities (“ocaps”), exemplified in languages such as E and programming models such as Actors, offers a fine-grained approach that can be made to scale further than a single machine. However, ocaps only control access to shared programs. Access controls for shared data are left implicit. In addition, ideas of location and system boundary are left implicit in ocap systems.
I will adapt ocaps to syndicated actors. Because the Syndicated Actor model includes a first-class notion of shared data as well as a layered conception of locations and location boundaries, syndicated capabilities will reflect these ideas directly. I will generalize the Syndicated Actor model’s existing notions of place, connecting capabilities not to individual actors but to individual places and the data held therein. I will draw on existing ocap literature, including in particular the recent notion of Macaroons (Birgisson, Politz, Erlingsson, Taly, Vrable and Lentczner 2014) and older ideas from SPKI/SDSI (Ylonen, Thomas, Lampson, Ellison, Rivest and Frantz 1999; Ellison 1999).
The following artifacts are expected to realise the outcomes of the project:
- Documents describing the theory of capabilities as applied to the Syndicated Actor model
- Implementations reflecting this theory
- Documents analysing the elements of a selection of existing system layers
- Protocols capturing a synthesis of system layer behaviours, based on the analysis
- A system layer implementation making use of syndicated actors and dataspaces
- Associated demos and case studies
- A website gathering together all project material
- Software releases and developer-focused documentation
Q. Why aren’t projects like systemd, NetworkManager, DBus tackling these issues?
A. Existing system layer realizations cross-cut so many projects: postmarketOS, Alpine Linux, NetworkManager, GNOME, DBus, systemd, and many others. Each project has its own role in the overall system layer, and none takes a strong stance on the overall architecture that results from their combination. This project touches on responsibilities currently handled at each of these levels.
Q. Why not aim at a system layer for NixOS instead of a mobile phone?
A. Systems like Nix and NixOS are really interesting and could make an excellent foundation for a personal computing system. However, there are a group of basic “runtime” concepts involved in a system layer that transcend distributions, relating to issues of IPC, discovery, and whole-machine and application state management more than specifically to package management and system configuration.
Starting with a minimal Linux system, namely Alpine Linux (in the form of postmarketOS) seems to me to be the way to go, since it can be configured to have almost no “flavour” of its own and to allow me to concentrate on the “runtime” aspects of a system layer.
I want to establish the basics at this level first - atop little more than a kernel, init, /proc, /sys and busybox - and then to incorporate system configuration and management ideas from Nix, Genode and friends, building atop the principles previously established.
That is, a future phase could combine the outputs of this project with approaches to system management such as Nix.
Q. How do you feel dataspaces would most enhance privacy or trust?
A. Capability technology offers strong, flexible control over access to any given dataspace without getting lost in the weeds of identity management: identity is an application-local, application-private concern.
Dataspaces default to being closed, “invite-only” networks, meaning casual observation of activity in a dataspace is not possible. But the necessary extension of the capability model to handle the data-sharing aspects of dataspaces gives benefit in terms of privacy and trust that goes beyond the already considerable benefits a traditional capability model offers.
Traditional capabilities directly control access to behavioural objects, and only indirectly control access to data held within such objects. Syndicated capabilities, by contrast, directly control access to shared data held within a space - changes to which may trigger activity in “objects” participating in the dataspace.
In other words, traditional capabilities encode data access controls in terms of object access controls; syndicated capabilities, vice versa.
This ability to directly express access to shared data gives system designers a powerful tool for thinking about permitted information flows, including questions of privacy. Furthermore, attenuating the authority of syndicated capabilities before passing them on to some other principal allows for strong partitioning of access within a dataspace, offering fine-grained, local, compositional decisions about access to shared data. Finally, it becomes possible to expose capabilities to end-users (roughly analogous to URLs), putting that power in their hands also.
I should also mention that dataspaces can scale from managing activity within a single OS process up to coordinating activity between machines around the world. A distributed dataspace could be an excellent foundation for collaborative applications, where privacy concerns come to the forefront. In effect, a dataspace can become a richly-structured “VPN”, containing application-specific shared data and with application- or schema-specific access controls.
Bass, Len, Paul Clements, and Rick Kazman. Software Architecture in Practice. Addison-Wesley, 1998.
Birgisson, Arnar, Joe Gibbs Politz, Úlfar Erlingsson, Ankur Taly, Michael Vrable, and Mark Lentczner. “Macaroons: Cookies with Contextual Caveats for Decentralized Authorization in the Cloud.” In Network and Distributed System Security Symposium. San Diego, California: Internet Society, 2014.
Clements, Paul, Rick Kazman, and Mark Klein. Evaluating Software Architectures: Methods and Case Studies. Addison-Wesley, 2001.
Corbet, Jonathan. “Systemd as Tragedy.” LWN.Net, January 28, 2019. https://lwn.net/Articles/777595/.
Ellison, Carl. SPKI Requirements. Request for Comments 2692. RFC Editor, 1999. https://doi.org/10.17487/RFC2692.
Rice, Benno. “The Tragedy of Systemd.” Conference Presentation presented at the linux.conf.au, Christchurch, New Zealand, January 24, 2019. https://2019.linux.conf.au/schedule/presentation/156/.
Ylonen, Tatu, Brian Thomas, Butler Lampson, Carl Ellison, Ronald L. Rivest, and William S. Frantz. SPKI Certificate Theory. Request for Comments 2693. RFC Editor, 1999. https://doi.org/10.17487/RFC2693.
I wrote to Benno Rice to ask him about the term. He replied that he doesn’t know of any earlier use of “system layer” for this particular bundle of ideas. Quoted (with permission) from his email to me:
I’m not going to claim to be the first who thought of the idea but the name was something I came up with to describe the services that run in userspace but provide system-level services. I’m happy to own it if nobody else had the idea first. 🙃
Like you say I think it’s been kicking around in a latent form for a long time. I suspect it’s one of those things that escaped definition for a long time simply because the way UNIX “thinks” doesn’t make it easy to conceptualise. It’s one of the reasons I brought up Windows in the talk because Windows has always had that kind of systemic service concept in a way that UNIX only started to later on.
It looks to me, then, like the term originated with him in 2019. ↩
I cut and pasted the automated YouTube transcript of the talk, and then cleaned it up. (Emphasis mine.) ↩
The resolver built in to libc plays the major part in this; but things like dnsmasq play a role too, especially when (as here) combined with virtual machines running within a host. ↩
The resulting analysis of system-layer concepts will be useful on its own, in addition to forming a “menu” of features available for implementation in the project’s own software. ↩