Atom feedFixing up protocol mismatches on-the-fly
I’ve been fleshing out the syndicate-rkt Racket implementation based
on the novy-syndicate TypeScript sketch. I just reached a milestone
of TCP-based interoperability between the two implementations (yay!),
but there’s an interesting little side track involved that I thought
I’d write about.
The novy-syndicate code had a placeholder “dataspace” implementation
that had extremely limited pattern-matching. It was only able to
offer subscribers the ability to select (1) record assertions having
(2) a user-selected, constant label.
For example, a subscriber could elect to receive all records labelled
with Present; or with Says. Subscribers were not able to even
specify arity of matched records. It really was a placeholder for a
proper implementation to come later (ported across from a previous
syndicate/js implementation).
By contrast, the syndicate-rkt code has a full-fledged dataspace
able to index assertions according to quite sophisticated patterns:
1
2
3
4
5
6
7
8
9
10
11
12
13
# Dataspace patterns: a sublanguage of attenuation patterns.
Pattern = DDiscard / DBind / DLit / DCompound .
DDiscard = <_>.
DBind = <bind @name symbol @pattern Pattern>.
DLit = <lit @value any>.
DCompound = @rec <compound @ctor CRec @members { int: Pattern ...:... }>
/ @arr <compound @ctor CArr @members { int: Pattern ...:... }>
/ @dict <compound @ctor CDict @members { any: Pattern ...:... }> .
CRec = <rec @label any @arity int>.
CArr = <arr @arity int>.
CDict = <dict>.
Now, I managed to get the novy-syndicate example programs to talk to
the full syndicate/rkt dataspace - without changing the code!
The way I did it was to rewrite assertions travelling between the programs on the fly.
And the way I did that was to include “rewrite” statements in the
capability I gave to the novy-syndicate client to allow it to
connect to the syndicate/rkt server.
The idea was to rewrite assertions-of-interest (subscriptions) from
the simple label-only pattern of novy-syndicate to the equivalent
full-dataspace pattern of syndicate/rkt, and to rewrite the
responses from the dataspace from the arbitrary-arity responses of
syndicate/rkt to the simple unary responses of novy-syndicate.
Here’s the rewrite specification,1 which ultimately appears embedded as a “caveat” inside the Macaroon-style capabilities that Syndicate uses:2
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
[ <or [
<rewrite
# Step 1:
<compound <rec Observe 2> {0: <bind label Symbol>, 1: <bind observer Embedded>}>
<compound <rec Observe 2> {
# Step 1(a):
0: <compound <rec bind 2> {
0: <lit assertion>
1: <compound <rec compound 2> {
0: <compound <rec rec 2> {0: <ref label>, 1: <lit 1>}>,
1: <compound <dict> {}>
}>
}>
# Step 1(b):
1: <attenuate <ref observer> [
<rewrite
<compound <arr 1> {0: <bind v <_>>}>
<ref v>>
]>
}>>
# Step 2:
<rewrite <bind n <_>> <ref n>>
]> ]
It reads:
-
try matching
<ObserveL C>, where L is a symbol and C an embedded capability; if it does not match, skip the remainder of this step; otherwise, rewrite it into<Observe <bind assertion ⌜P⌝>f(C)>, where-
P is a pattern matching records of the form
<L_>, and the quotation operator⌜·⌝quotes a pattern over assertions into a term conforming to thePatternschema above; and -
f(C) “attenuates” C by attaching rewrites to it. Any assertion sent to C is required to be of the form
[V], and is rewritten into just V.
-
-
if the rewrite in step 1 didn’t apply then match anything; call it
n; and rewrite it to itself.
The net effect is that when the simple chat example from
novy-syndicate asserts
<Observe Present #!C>
the syndicate-rkt server actually sees
<Observe <bind assertion ⌜<Present _>⌝> #!f(C)>
and when syndicate-rkt replies with an actual concrete presence
record, for example3
[<Present "Tony">]
the novy-syndicate client will actually receive just
<Present "Tony">
Cool huh?
Now, this works great for Present, which is unary, but not so well
for the client’s subscription to Says, which is binary: <Says who
what>. So our interoperability is limited here: the client only
sees presence information from its peers, and the actual utterances
sent get dropped on the floor for lack of an appropriate pattern at
the syndicate-rkt dataspace. To fix this, we could include a more
complex rewrite specification that treated Presence and Says
subscriptions separately and explicitly, with the correct arity for
each. But I’m done for now, and will focus on getting a proper
dataspace implementation into novy-syndicate instead.
-
We need a DSL for these rewrite specifications! I’m working on it. It’ll probably look like the existing Syndicate DSL syntax for patterns. ↩
-
Here’s the whole capability, including an “oid” identifying the service to be accessed, the sequence of “caveats” rewriting and attenuating information flowing through the capability, and the signature proving the capability’s validity:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
<ref "syndicate" [[<or [ <rewrite <compound <rec Observe 2> { 0: <bind label Symbol>, 1: <bind observer Embedded> }> <compound <rec Observe 2> { 0: <compound <rec bind 2> { 0: <lit assertion>, 1: <compound <rec compound 2> { 0: <compound <rec rec 2> { 0: <ref label>, 1: <lit 1> }>, 1: <compound <dict> {}> }> }>, 1: <attenuate <ref observer> [<rewrite <compound <arr 1> {0: <bind v <_>>}> <ref v>>]> }>>, <rewrite <bind n <_>> <ref n>> ]>]] #[1oCXyvdXylgpWRhgg0w+iw==]> -
The single-element list is there because the rewritten pattern included a single binding named
assertion, so there’s a single value in the list of potentially-many values sent back to the subscriber. The simplifiednovy-syndicatepatterns included exactly one implicit whole-assertion binding, and so the list wrapper is also implicit in thenovy-syndicatevariation, which is why it has to be explicitly removed to get interoperability here. ↩