One promising application of dataspaces is dependency tracking for orderly service startup.
The problem of service startup appears at all scales. It could be services cooperating within a single program and process; services cooperating as separate processes on a single machine; containers running in a distributed system; or some combination of them all.
Syndicate programs are composed of multiple services running together, with dependencies on each other, so it makes sense to express service dependency tracking and startup within the programming language.
In the following, I’ll sketch service dependency support for cooperating modules within a single program and process. The same pattern can be used in larger systems; the only essential differences are the service names and the procedures for loading and starting services.
Let’s imagine we have the following situation:
A program we are writing depends on the “tcp” service, which in turn depends on the “stream” service. Separately, the top-level program depends on the “timer” service.
Describing the data and protocol
A small protocol for services and service activations describes the data involved:
RequireService = <require-service @service-name any>. ServiceRunning = <service-running @service-name any>.
RequireService record indicates demand for a running
instance of the named service; an asserted
indicates presence of the same; and interest in a
implies assertion of a
A library “service manager” process, started alongside the top level
program, translates observed interest in
RequireService, and then translates observed
assertions into service startup actions and provision of matching
1 2 3 4 5 6 (during (Observe (:pattern (ServiceRunning ,(DLit $service-name))) _) (assert (RequireService service-name))) (during/spawn (RequireService $service-name) ;; ... code to load and start the named service ... )
Putting these pieces together, we can write a program that waits for a
'some-service-name to be running as follows:
1 (during (ServiceRunning 'some-service-name) ...)
When the service appears, the facet in the ellipsis will be started, and if the service crashes, the facet will be stopped (and restarted if the service is restarted).
Services can wait for their own dependencies, of course. This automatically gives a topologically sorted startup order.
Modules as services, and macros for declaring dependencies
In the Syndicate/rkt implementation, a few standard macros and functions implement the necessary protocols.
First, services can be required using a
1 2 3 4 (with-services [syndicate/drivers/tcp syndicate/drivers/timer] ;; ... body expressions ... )
Second, each Racket module can offer a service named after the
module by using a
provide-service macro at module toplevel. For
example, in the
syndicate/drivers/tcp Racket module, we find the
1 2 3 4 5 (provide-service [ds] (with-services [syndicate/drivers/stream] (at ds ;; ... set up tcp driver subscriptions ... )))
Finally, the main entry point to a Syndicate/rkt program can use a
standard-actor-system macro to arrange for the startup of the
“service manager” process and a few of the most frequently-used
1 2 3 4 (standard-actor-system [ds] ;; ... code making use of a pre-made dataspace (ds) and ;; preloaded standard services ... )