I’ve been fleshing out the syndicate-rkt Racket implementation based on the novy-syndicate TypeScript sketch. I just reached a milestone of TCP-based interoperability between the two implementations (yay!), but there’s an interesting little side track involved that I thought I’d write about.

The novy-syndicate code had a placeholder “dataspace” implementation that had extremely limited pattern-matching. It was only able to offer subscribers the ability to select (1) record assertions having (2) a user-selected, constant label.

For example, a subscriber could elect to receive all records labelled with Present; or with Says. Subscribers were not able to even specify arity of matched records. It really was a placeholder for a proper implementation to come later (ported across from a previous syndicate/js implementation).

By contrast, the syndicate-rkt code has a full-fledged dataspace able to index assertions according to quite sophisticated patterns:

1
2
3
4
5
6
7
8
9
10
11
12
13
# Dataspace patterns: a sublanguage of attenuation patterns.
Pattern = DDiscard / DBind / DLit / DCompound .

DDiscard = <_>.
DBind = <bind @name symbol @pattern Pattern>.
DLit = <lit @value any>.
DCompound = @rec  <compound @ctor CRec  @members { int: Pattern ...:... }>
          / @arr  <compound @ctor CArr  @members { int: Pattern ...:... }>
          / @dict <compound @ctor CDict @members { any: Pattern ...:... }> .

CRec = <rec @label any @arity int>.
CArr = <arr @arity int>.
CDict = <dict>.

Now, I managed to get the novy-syndicate example programs to talk to the full syndicate/rkt dataspace - without changing the code!

The way I did it was to rewrite assertions travelling between the programs on the fly.

And the way I did that was to include “rewrite” statements in the capability I gave to the novy-syndicate client to allow it to connect to the syndicate/rkt server.

The idea was to rewrite assertions-of-interest (subscriptions) from the simple label-only pattern of novy-syndicate to the equivalent full-dataspace pattern of syndicate/rkt, and to rewrite the responses from the dataspace from the arbitrary-arity responses of syndicate/rkt to the simple unary responses of novy-syndicate.

Here’s the rewrite specification,¹ which ultimately appears embedded as a “caveat” inside the Macaroon-style capabilities that Syndicate uses:²

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
[ <or [
    <rewrite

     # Step 1:
     <compound <rec Observe 2> {0: <bind label Symbol>, 1: <bind observer Embedded>}>
     <compound <rec Observe 2> {

       # Step 1(a):
       0: <compound <rec bind 2> {
         0: <lit assertion>
         1: <compound <rec compound 2> {
           0: <compound <rec rec 2> {0: <ref label>, 1: <lit 1>}>,
           1: <compound <dict> {}>
         }>
       }>

       # Step 1(b):
       1: <attenuate <ref observer> [
         <rewrite
          <compound <arr 1> {0: <bind v <_>>}>
          <ref v>>
       ]>

     }>>

    # Step 2:
    <rewrite <bind n <_>> <ref n>>

  ]> ]

It reads:

1. try matching <Observe L C>, where L is a symbol and C an embedded capability; if it does not match, skip the remainder of this step; otherwise, rewrite it into <Observe <bind assertion ⌜P⌝> f(C)>, where

   1. P is a pattern matching records of the form <L _>, and the quotation operator ⌜·⌝ quotes a pattern over assertions into a term conforming to the Pattern schema above; and

   2. f(C) “attenuates” C by attaching rewrites to it. Any assertion sent to C is required to be of the form [V], and is rewritten into just V.

2. if the rewrite in step 1 didn’t apply then match anything; call it n; and rewrite it to itself.

The net effect is that when the simple chat example from novy-syndicate asserts

<Observe Present #!C>

the syndicate-rkt server actually sees

<Observe <bind assertion ⌜<Present _>⌝> #!f(C)>

and when syndicate-rkt replies with an actual concrete presence record, for example³

[<Present "Tony">]

the novy-syndicate client will actually receive just

<Present "Tony">

Cool huh?

Now, this works great for Present, which is unary, but not so well for the client’s subscription to Says, which is binary: <Says who what>. So our interoperability is limited here: the client only sees presence information from its peers, and the actual utterances sent get dropped on the floor for lack of an appropriate pattern at the syndicate-rkt dataspace. To fix this, we could include a more complex rewrite specification that treated Presence and Says subscriptions separately and explicitly, with the correct arity for each. But I’m done for now, and will focus on getting a proper dataspace implementation into novy-syndicate instead.