PsEPR Documentation: Protocol Guide

Contents


1 Overview

As described in the Architecture Guide , a PsEPR application is built of loosely coupled clients. The clients stay coupled by generating events and listening for event on channels. PsEPR implements these channels through a system of routers which create an efficient connection mesh to move the events from event sources to event sinks.

This design leads to two levels of protocols. The first is the low level PsEPR "internal protocols" which PsEPR uses to set up communication pipes between the clients and routers and between the routers themselves. This internal protocol creates the channels that the clients use to communicate. Once the channels are in place, there can be "service protocols" which the clients implement. This distinction is brought up here because there is not a hard line between these two levels. For instance, the routers will use a protocol described in this manual to communicate but information on the topology of the network (and thus the information that the routers must share to create an optimal mesh network) is distributed as events on a channel. This simplifies the internal protocol and allows for possible future expansion.

Several guiding priniciples guide the design of these internal PsEPR protocols:

  • internal protocols implement a minimum of functionality
  • state maintainance is kept to a minimum
  • failures happen (expect failures and indeterminant states)
  • lots more

2 Theory of Operation

2.1 Channel Abstraction

Loose coupling. Error recovery.

2.2 Channel Name Space

A channel name consists of a naming authority name followed by a hiearchy of channel subdivisions all separated by the character slash ("/"). The "naming authority" is the domain name of the entity managing the top level of the channel space and is used to provide a globally unique top to the hiearchy of channel names.

Some examples of channel names are:

/psepr.org/ns/planetlab/node/planetlab1.nbgisp.com/
/psepr.org/ns/planetlab/nodelist/
/cs.utexas.edu/channels/cities/Austin/

Note that channel names always begin and ends with a slash. There is currently no wildcard'ing but various constructs are reserved for future expansion -- the character "*" and an empty field ("//"), in particular.

The hiearchy of channel names can be used since PsEPR allows subscribing to all of the channels below a certain point in the hiearchy. For instance, getting a lease on the channel /psepr.org/ns/planetlab/node/ will return all of the events on all of the channels named as a subset of that channel (/psepr.org/ns/planetlab/node/planetlab1.nbgisp.com/, etc).

All channel names are "rooted". The leading MUST be present. All channel names also MUST have a slash at the end.

A detailed definition of channel is:

ChannelName ::= "/" ChannelNameAuthority "/" ChannelPart "/"

ChannelNameAuthority ::= ChannelElement

ChannelPart ::= ChannelElement [ "/" ChannelPart ]

ChannelElement ::= 1*(%x21-2e / %x30-3F / %x41-FE )
                  ; any octets that map to the printable
                  ; UTF-8 encoded Unicode 2.0 characters except
                  ; octet '0x2F' (which corresponds to the 
                  ; ASCII character '/').
                

2.3 Client to Router Connection Management

The client must connect to a Jabber server and over time, services will be developed so a client connects to the optimal server. But, as a connection fallback, the initial installation of PsEPR client installs the file /usr/local/planetlab/etc/psepr-servers which contains a list of potential Jabber servers.

This file contains multiple lines with each line of the form:

INAME@ISERVER/IINSTANCE
              
INAME required the Jabber name of the Router on the server
ISERVER required the DNS name of the Jabber server to connect to for communication with the Router.
IINSTANCE optional The instance name of the router. This is optional and, if not specified, the separating slash is not included.

The PsEPR installation guarantees that at least one of the servers listed in this file is operational. A client should walk through the entries in this file in random order until an operating server is discovered.

Once a connection is made to the PsEPR infrastruction, other services with information on connectivity can be consulted for a more optimal connection.

2.4 Sending One Event

There is a non-XMPP connection that a client can make which is avaiable for the simple sending of events. The client creates a socket connection to the Router and sends a block of XML which contains both the authentication and the event. If the authentication is accepted, the event is routed through PsEPR. If authentication is not accepted, the event is lost and no indication is returned to the sender.

The format of this message is specified in a later section and client libraries will have helper functions like the one in the Java client library .

3 Client to Router Message Reference

3.1 General Format

Events between the client and the router, have the following format:

<message to='REGISTRY@JABBER-SERVER/REGISTRY-INSTANCE'>
    <x seconds="EXPIRATION-SECONDS" xmlns="jabber:x:expire"/>
    <event xmlns="http://psepr.org/xs/event">
      <to>
          <channel>TO-CHANNEL-NAME</channel>
          <service>TO-SERVICE-NAME</service>
          <instance>TO-INSTANCE-NAME</instance>
      </to>
      <distribution>DISTRIBUTION</distribution>
      <from>
          <service>FROM-SERVICE-NAME></service>
          <instance>FROM-INSTANCE-NAME</instance>
      </from>
      <payload xmlns="PAYLOAD-NAMESPACE">
      </payload>
  </event>
</message>
            

The <event> element can include any XML content. The format of the content is specified by the namespace on the <event> element and the ones handled by the PsEPR system are explained below.

Several of the files have distinct values:

REGISTRY required Always "psepr_registry". This is the name of the jabber account that the registry runs under.
JABBER-SERVER required the jabber server to talk to to reach the registry
REGISTRY-INSTANCE optional this would address the particular registry but an unspecified instance will get it to one of the registries which is sufficient
EXPIRATION-SECONDS optional PsEPR relies on packets going away after a while. Well, 'relies' is may be too strong a term -- it likes the them to die off as it makes error recovery work better. A good value here is "30" for 30 seconds.
TO-CHANNEL-NAME required Name of the channel message is to/from. Refer to the channel name space section for the format and functionality of the channel name.
TO-SERVICE-NAME optional If not specified, the event will be delivered to all services listening to the channel. Giving a service name allows some limiting of the event delivery.
TO-INSTANCE-NAME optional If not specified, the event is delivered to all instances of the service. Giving an instance allows addressing of the event to a particular receiver. This field is ignored if no TO_SERVICE-NAME is given.
DISTRIBUTION optional This specifies the distribution of the message. This is either:
all send to all listeners
one send to at most one listener
If distribution is not specified, the default is all (send to all listeners) .
FROM-SERVICE-NAME required This field is filled in by PsEPR and is the autheniticated name of the sender.
FROM-INSTANCE-NAME required This field is filled in by PsEPR and specifies the instance ID of the sender. This string is generated by the sender and is only as unique as the sender makes it.
PAYLOAD-NAMESPACE required This specifies the format of the event payload. Follow XML namespace rules in creating this string. Users specify their own namespaces for the payload. Note that the payload element itself is part of this namespace.

3.2 Payload

General payload.

<message to='REGISTRY@JABBER-SERVER/REGISTRY-INSTANCE'>
    <x seconds="EXPIRATION-SECONDS" xmlns="jabber:x:expire"/>
    <event xmlns="http://psepr.org/xs/event">
      <to>
          <channel>TO-CHANNEL-NAME</channel>
          <service>TO-SERVICE-NAME</service>
          <instance>TO-INSTANCE-NAME</instance>
      </to>
      <distribution>DISTRIBUTION</distribution>
      <from>
          <service>FROM-SERVICE-NAME></service>
          <instance>FROM-INSTANCE-NAME</instance>
      </from>
      <payload xmlns="http://psepr.org/xs/payload">
        XML-DATA
      </payload>
  </event>
</message>
            

3.3 Attribute Payload

A defined payload for a simple attribute/value pair. The bodies of both the field and value elements are text which follows the xml text inclusion rules.

<message to='REGISTRY@JABBER-SERVER/REGISTRY-INSTANCE'>
    <x seconds="EXPIRATION-SECONDS" xmlns="jabber:x:expire"/>
    <event xmlns="http://psepr.org/xs/event">
      <to>
          <channel>TO-CHANNEL-NAME</channel>
          <service>TO-SERVICE-NAME</service>
          <instance>TO-INSTANCE-NAME</instance>
      </to>
      <distribution>DISTRIBUTION</distribution>
      <from>
          <service>FROM-SERVICE-NAME></service>
          <instance>FROM-INSTANCE-NAME</instance>
      </from>
      <payload xmlns="http://psepr.org/xs/payload/attribute">
          <field>ATTRIBUTE-NAME</field>
          <value>ATTRIBUTE-VALUE</value>
          <expire>ATTRIBUTE-EXPIRE</expire>
          <serial>ATTRIBUTE-SERIAL</serial>
      </payload>
  </event>
</message>
            
ATTRIBUTE-NAME required Text specifying the attribute name. The format is specific to the server.
ATTRIBUTE-VALUE required Text giving the value for the attribute. The format is specific to the server.
ATTRIBUTE-EXPIRE optional Number of seconds until this value expires. This is from the point of view of the sender and the receiver can take it as the amount of time the sender thinks the value is valid.
ATTRIBUTE-SERIAL optional A uniquifying value for this instance of the attribute.

3.4 Lease

A client requests to receive events on a channel by sending a lease event to that channel. The lease event specifies that this client wishes to receive events for some period in the future.

A client knows that PsEPR has noticed the lease by a lease message with a LEASE-OPERATION lease with the LEASE-IDENTIFIER if the origional register event appearing on the channel. PsEPR might not grant the whole requested duration. In that case, the lease message that is received will contain the granted duration.

Do note that a client can receive events on a channel before it sees the lease event. The lease event is best used to manage lease duration -- if a lease is not received, the client should renew the lease with another register event.

A client renews a lease by sending another register event. A client can undo a lease by sending a unregister event with the LEASE-IDENTIFIER of the origional lease.

<message to='REGISTRY@JABBER-SERVER/REGISTRY-INSTANCE'>
    <x seconds="EXPIRATION-SECONDS" xmlns="jabber:x:expire"/>
    <event xmlns="http://psepr.org/xs/event">
      <to>
          <channel>TO-CHANNEL-NAME</channel>
          <service>TO-SERVICE-NAME</service>
          <instance>TO-INSTANCE-NAME</instance>
      </to>
      <distribution>DISTRIBUTION</distribution>
      <from>
          <service>FROM-SERVICE-NAME></service>
          <instance>FROM-INSTANCE-NAME</instance>
      </from>
      <payload xmlns="http://psepr.org/xs/payload/lease">
          <identifier>LEASE-IDENTIFIER</identifier>
          <duration>LEASE-DURATION</duration>
          <op>LEASE-OPERATION</op>
          <type>LEASE-MESSAGE-TYPE</type>
      </payload>
  </event>
</message>
            
LEASE-IDENTIFIER required A string that uniquely identifies this lease request. It is generated by the client for a register lease message and that same identifier is used later in the unregister and returned in lease messages.
LEASE-DURATION required Duration of the lease in seconds
LEASE-OPERATION required This field has one of the following values:
register Request by the client to receive messages on the channel for LEASE-DURATION time.
unregister Request by the client to stop receiving messages on the channel. The LEASE-IDENTIFIER is the one from the origional register event.
lease Specifies that a lease register has been noticed by PsEPR . The LEASE-IDENTIFIER is the one from the corresponding register and the LEASE-DURATION is the period that the lease has been granted for.
LEASE-MESSAGE-TYPE optional If specified, this says this lease is for only events who's PAYLOAD-NAMESPACE matches this value. This allows a receiver to only listen for messages of a specific format/type.

3.5 One Event

Clients can send an event by opening up a port on the PsEPR server and sending a block of XML which contains the authenication and the event.

<inputEvent>
  <authentication>
    <passwordAuth>
      <service>JABBER-ACCOUNT</service>
      <password>JABBER-PASSWORD</password>
    </passwordAuth>
  </authentication>
    <oneEvent>
      <event xmlns="http://psepr.org/xs/event">
        <to>
            <channel>TO-CHANNEL-NAME</channel>
            <service>TO-SERVICE-NAME</service>
            <instance>TO-INSTANCE-NAME</instance>
        </to>
        <distribution>DISTRIBUTION</distribution>
        <from>
            <service>FROM-SERVICE-NAME></service>
            <instance>FROM-INSTANCE-NAME</instance>
        </from>
        <payload xmlns="my:payload:namespace">
          XML-DATA
        </payload>
      </event>
    </oneEvent>
</inputEvent>
            
JABBER-ACCOUNT required A string of the Jabber account that one out log into for Jabber.
JABBER-PASSWORD required A string of the Jabber account password.

3.6 Examples

Here are several examples of messages and exchanges.

A service ('pldb) asks for a lease for 900 seconds on the channel /psepr.org/ns/planetlab/list/nodes/:

<message to='psepr_router@jabber.services.planet-lab.org'>
  <x seconds='60' xmlns='jabber:x:expire'/>
  <event xmlns='http://psepr.org/xs/event'>
    <to>
      <channel>psepr.org/ns/planetlab/list/nodes/</channel>
      <service>psepr_router</service>
      <instance></instance>
    </to>
    <from>
      <service>pldb</service>
      <instance>planetlab6.nbgisp.com/list/nodes/</instance>
    </from>
    <payload xmlns='http://psepr.org/xs/payload/lease'>
      <duration>900</duration>
      <intifier>psepr-1104515112</identifier>
      <op>register</op>
      <type>http://psepr.org/xs/payload/pldbdeadbeat</type>
    </payload>
  </event>
</message>
          

The PsEPR registry responds to the lease message with a lease confirmation:

<message 
    from='psepr_registry@jabber.services.planet-lab.org/loudmouth-test-registry'
    to='pldb@jabber.services.planet-lab.org/planetlab6.nbgisp.com/list/nodes/'
    xmlns='jabber:client'>
  <x seconds='60' xmlns='jabber:x:expire'/>
  <event xmlns='http://psepr.org/xs/event'>
    <to>
      <channel>psepr.org/ns/planetlab/list/nodes/</channel>
      <service>psepr_router</service>
      <instance/>
    <to>
    <from>
      <service>pldb</service>
      <instance>planetlab6.nbgisp.com/list/nodes/</instance>
    <from>
    <payload xmlns='http://psepr.org/xs/payload/lease'>
      <duration>900</duration>
      <identifier>psepr-1104515112</identifier>
      <op>lease</op>
      <type>http://psepr.org/xs/payload/pldbdeadbeat</type>
    </payload>
  </event>
</message>
            

4 Client to Router Messages

This section describes in detail the messages the raw messages the client sends to the server. The goal is to allow people to write new client libraries for PsEPR.

The connection between the client and the server is usually over a TCP/IP socket connection and there are serveral formats: an XMPP like format, a single event format and a few others taylored to specific formats.

4.1 Default Server Ports

As a default, the PsEPR server uses the following port assignments. Check the configuration files, though, since port assignments can always be over-ridden.

Port Use
23340 Reserved
23341 Server to server communication. This port is used by the servers to pass messages and control between themselves when implementing the LeasePeer model.
23342 Reserved
23343 Incoming (from client to server) One Event messages.
23344 Reserved for future incoming WS-Notification events.
23345 Incoming XMPP-like connections which are the basis for the basic lease model. Clients connect here, authenticae themselves and then keep a connection up to make leases and receive requested events.
23346 Reserved
23347 Reserved

4.2 XMPP-like Protocol

The origional design of PsEPR thought that using an existing messaging interface for events would allow leveraging existing tools and applications. XMPP seemed like a good starting point. It didn't work out as well as expected: the existing open source XMPP server implementations either died under heavy load or had features that were too tied to it's instant messaging roots.

The outcome of this initial decision, thought, is that the protocol to access the PsEPR server is XMPP-like. I use the term "XMPP-like" because no attempt was made to make it XMPP compliant. The servers only accept enough XMPP messages to allow some of the existing XMPP client libraries to talk to PsEPR servers. No attempt has been made to do compatability testing so client library implementors should just follow the messages described herein.

It is possible that advances in XMPP standards and implementations may have rendered the basic pub/sub functionality of PsEPR obsolete, PsEPR was designed as a research vehicle for eventing infrastructures. Different semantics for subscription, filtering and routing are all potential extenstions to the basic functions currently implimented in PsEPR.

The XMPP protocol has two classes of messages: 'messages' which are used by PsEPR for event, and 'queries' which are used for authentication.

There are several attributes and elements that XMPP can include in the messages. These are ignored by the PsEPR server and do not create errors.

4.2.1 XMPP Authentication Query

An empty authentication query asks the server which authentication protocols it supports. The current PsEPR server only support password authentication so the server responds to such a message with a response saying so.

If the server receives a message like:

<iq id="IDENT">
  <query xmlns="jabber:iq:auth">
  </query>
</iq>
              
the server will respond with this message which specifies password authentication:
<iq id="IDENT">
  <query xmlns="jabber:iq:auth">
    <password/>
  </query>
</iq>
              
the 'IDENT' in the responding message will be identical as the one passed in the query request.

Other elements which may occur in the query ('digest' and 'resource', for instance) are ignored.

4.2.2 XMPP Authentication

To log in to the server with a password, the client will send the following message:

<iq id="IDENT">
  <query xmlns="jabber:iq:auth">
    <username>USERNAME</username>
    <password>PASSWORD</password>
    <resource>RESOURCE</resource>
  </query>
</iq>
              
to which, the server will reply with the following message if the user and password are accepted:
<iq id="IDENT">
  <query xmlns="jabber:iq:auth">
    <username>USERNAME</username>
    <resource>RESOURCE</resource>
  </query>
</iq>
              
where the 'IDENT', 'USERNAME', and 'RESOURCE' are the same as in the query.

If the username and password are not accepted, no response is sent. You'll just have to time out.


Revision History

1.4 2007-06-10 RA Changed R2R section to C2R. Added ports and description of the protocol to the server.
1.3 2006-04-10 RA Removed R-to-R sections to focus this document on the C-to-R protocol. Added sections on OneEvent. Corrected error in description of payload namespace.
1.2 2005-04-15 RA Added 'expire' and 'serial' to definition of PayloadAttribute.
1.1 2005-03-02 RA Added field descriptions to the event and lease field tables. Removed the 'Route Info' events since they don't exist.
1.0 2005-02-01 RA Initial release