PsEPR Documentation: jClient Guide

Contents


1 Overview

The Java package org.psepr.jClient provides and interface to the PsEPR event system. This document describes how to use the extend the jClient package. Javadoc interface documentation is also available.

2 Quick Start

This library presents a set of objects for the pieces of the PsEPR protocol : classes for a connection, lease, event, payload, etc.

First a connection is made to the PsEPR system. This requires an authenticated service ID so one creates a PsEPRServiceIdentity object. This is passed to PsEPRConnection. In most cases, only one connection is needed for an application as multiple leases can use one connection.

Sending events is done by creating an event, adding the payload to the event and then sending it. A code sample for sending an "attribute" event looks like:

// create and populate the attribute payload
PayloadAttribute myPA = new PayloadAttribute();
myPA.setAttributeName("projectName");
myPS.setAttributeValue("Cute Service");

// create the event
PsEPREvent myEV = new PsEPREvent();
myEV.setDistribution(PsEPREvent.distributionOne);
myEV.toChannel("/example.com/services/cuteservice/status/");
myEV.toService(myService);    // optional
myEV.toInstance(myInstance);  // optional
myEV.setPayload(myPA);
myPConn.sendEvent(myEV);
        

To receive events, a lease on a channel must be aquired. This is done by specifying the type of the event to receive and the routine to receive the event.

Here is a generic code sample to create a lease:

import org.psepr.jClient.*;
...
PsEPRServiceIdentity myID = new PsEPRServiceIdentity("service", "password");
PsERPConnection pConn = new PsEPRConnection(myID);
...
PayloadParser myPP = new myClassImplementingPayloadParser();
EventReceiver myPR = new myClassImplementingEventReceiver();
myPLease = myPCon.getLease(channelName, myPP, myQM);
...
myPConn.releaseLease(myPLease);
        
(Note that a lease helper class explained later in this document.)

When starting to use PsEPR, one of the first problems is connecting to the servers. The current list of servers is specified in the configuration file and the most current version can be fetched from http://psepr.org/PsEPRServer/jClient.xml . There is usually a delay while an application is searching for a server. You can watch this searching process (helpful while debugging) by turning on the debug flags in the configuration file. Add the following section to your user configuration file:

<jClient>
  <debug>
    <filterLevel>
      0x0062
    </filterLevel>
  </debug>
</jClient>          
        
This will cause server searching and lease aquisition to be output to standard out.

3 Service Accounts

Any client sending events on PsEPR has a service ID that is authenticated with the PsEPR infrastructure. For the moment, you must contact the PsEPR maintainer (Robert dot Adams at intel dot com) to get an ID.

The service ID authenticates the service. The service ID is in the from/service element in any received event. There can be many instances of a service and all of these should authenticate as the same service. That is, if your service is running on 1000 nodes, all will authenticate as the same service (use the same PsEPRServiceIdentity parameters. A receiver can tell the separate instances of the service apart since each will have a random instance code generated. This instance code is seen in the from/instance element of received events. [Note: an application can specify an instance string if necessary. Normal usage is to use the default random string.]

4 Sending Events

Events can be sent at any time onto any channel. To send an event, one creates a payload, creates an PsEPREvent, adds the payload to it and then sends the event on the previously opened PsEPRConnection.

PsEPREvent has methods for the fixed event values (to and from fields) and an instance of a payload. The payload is encapsulated in an object that is derived from Payload.

The code for sending an event which contains an attribute payload is:

// create and populate the attribute payload
PayloadAttribute myPA = new PayloadAttribute();
myPA.setAttributeName("projectName");
myPS.setAttributeValue("Cute Service");

// create the event
PsEPREvent myEV = new PsEPREvent();
myEV.setDistribution(PsEPREvent.distributionOne);
myEV.toChannel("/example.com/services/cuteservice/status");
myEV.toService(myService);    // optional
myEV.toInstance(myInstance);  // optional
myEV.setPayload(myPA);
myPConn.sendEvent(myEV);
          

Note that you do not specify the from fields of the event. These are set by the PsEPR infrastructure based on the authenticated identity that was used to create the connection.

4.1 Sending One Event

If you are sending only a few events and are not expecting to receive any events, there is a special interface on the service and an invocation that will quickly send one event.

The process is to create a PsEPREvent and send it. The authentication is sent with the even. Additionally, there is no response on whether the authentication is accepted or whether the event is well formed so, it is suggested, that you test things out with a regular lease connection. An event that doesn't work will just be thrown away by this interface with no indication of failure.

The calling sequence is:

SendOneEvent soe = new SendOneEvent();
PsEPREvent pe = new PsEPREvent();
PsEPRServiceIdentity id = new PsEPRServiceIdentity(service, password);
pe.setChannel(channel);
... initialize the PsEPREvent
String myPayload = "";
Payload pl = new PayloadGeneric();
pl.setXML(myPayload);
pe.setPayload(pl);
soe.sendOneEvent(i, pe);
              
Note that the payload contains the <payload> element. Because of the way that the namespace is defined, they payload element is part of my:name:space.

5 Receiving Events

To receive and event, one must get a lease on a channel for messages of a particular type. The lease request

5.1 Payload Class

The variable part of a PsEPR event is the payload. The payload is represented by an object that is derived from the Payload class. The payload must be parsed and this is done by a class that implements the PayloadParser interface. To keep all of the code for a payload together, this is usually both done in one class. An example is PayloadAttribute which handles payloads of the /psepr.org/ns/payload/attribute type.

A class that handles a payload looks like:

import org.psepr.jClient.*;
class PayloadMyPayload extends Payload implements PayloadParser {
  public PayloadMyPayload() {
    super("myPayloadNamespace");
  }

  // get and set varaibles that access the data in the payload
  ...

  // method that parses the payload
  public Payload parsePayload(String ns, XmlPullParser parser) {
    // check to make sure the name space matches.
    if (ns.equalsIgnoreCase(super.getNamespace())) {
      // parse the payload leaving the parser on the ending payload element
      myPayload = new PayloadMyPayload();
      ...
      return myPayload;
    }
    return null;   // null to denote that we didn't parse the payload
  }

  // method that outputs an XML representation of the payload
  //   note that is includes the beginning and ending payload elements
  public String toXML() { }
}
              

5.1.1 Payload Attribute

PayloadAttribute is included in the jClient jar as an example of a simple payload receiver. It processes payloads of the namespace /psepr.org/ns/payload/attribute. The payload contains two elements: field which is the name of the attribute, and value which is it's value.

5.1.2 Payload Generic

PayloadGeneric is a generic payload receiver -- it receives a payload of the specified namespace and just captures the pure XML of the payload.

The use of this payload type would be:

PayloadParser myPP = new PayloadGeneric("/my/name/space");
myPLease = myPCon.getLease(channelName, myPP, this);
...
public boolean receiveEvent(PsPEREvent ev) {
  if (ev.getPayload() != null) {
    if (ev.getPayload() instanceOf PayloadGeneric) {
      System.out.println("Body of payload in XML = "
              +((PayloadGeneric)ev.getPayload()).getXML());
      }
    }
  }
}
                

The generic parsing routine tries to copy all of the contents of the payload including name space specification and element prefixes. Real fancy contents (like processing instructions) are not copied although XML comments are copied.

5.2 Event Queue Receiver

The class EventQueue implmements the EventReceiver interface and creates a simple queue of events. It is thread safe and it can be passed to multiple leases to create a single queue for incoming events:

EventQueue eQueue = new EventQueue();
PsEPRLease lease1 = pConn.getLease(chan1, parser1, eQueue);
PsEPRLease lease2 = pConn.getLease(chan2, parser2, eQueue);
...
              
After this, a thread can call eQueue.nextEvent() for a blocking reader for all the events coming into both of these leases.

5.3 Lease Management

A client asks for a lease on a PsEPR channel. These leases are for specific message types (payload namespaces) on a channel (and subchannels) and the lease is for a fixed duration. A client that wants a long term lease, has to renegotiate the duration of the lease as and end of the last lease approaches.

It can take several seconds for a lease to be granted. It is even possible for the lease grant response to be lost and the calling routine could start receiving events before it thinks the lease has been granted. The status of the lease is found by querying the lease manager. The method pLease.getLeaseManager().getLeaseActive() will return true if the lease has been granted (PsEPR has responded to the lease request).

The jClient library includes the SimpleLeaseManager who simply gets the lease for a specific time period and then sends a new lease when the end of the lease is approaching.

You can create the SimpleLeaseManager yourself or, if you don't specify a lease manager, the PsEPRConnection.getLease will create one. The forms are:

// user supplies their own lease manager
PayloadParser mpp = new ClassThatImplmentsPayloadParserInterface();
EventReceiver mer = new ClasstheImplementsPayloadReceiverInterface();
LeaseManager mem = new ClassDerivedFromLeaseManager
PsEPRLease lease1 = pConn.getLease(chan1, mpp, mer, mem);

// use the SimpleLeaseManager for a lease of 300 seconds
PayloadParser mpp = new ClassThatImplmentsPayloadParserInterface();
EventReceiver mer = new ClasstheImplementsPayloadReceiverInterface();
LeaseManager mem = new SimpleLeaseManager(300);
PsEPRLease lease1 = pConn.getLease(chan1, mpp, mer, mem);
                
// use the SimpleLeaseManager for a lease of 300 seconds
PayloadParser mpp = new ClassThatImplmentsPayloadParserInterface();
EventReceiver mer = new ClasstheImplementsPayloadReceiverInterface();
PsEPRLease lease1 = pConn.getLease(chan1, mpp, mer, 300);
                
// use the SimpleLeaseManager for the default period
PayloadParser mpp = new ClassThatImplmentsPayloadParserInterface();
EventReceiver mer = new ClasstheImplementsPayloadReceiverInterface();
PsEPRLease lease1 = pConn.getLease(chan1, mpp, mer);
              
The last case will be the most common usage.

SimpleLeaseManager has additional methods for viewing the state of a lease. In particular, getLeaseState which will report the detailed state of the lease:

SimpleLeaseManager.LEASE_INACTIVE The lease has not been started. No events will be received (they are thrown away).
SimpleLeaseManager.LEASE_PENDING A lease request has been send and a response is being waited for.
SimpleLeaseManager.LEASE_ACTIVE the lease is active and running
SimpleLeaseManager.LEASE_EXPIRED the lease has expired. Normally, an extension on the lease is requested before it expires. This state happens when a response to an extenstion is never reveived.
SimpleLeaseManager.LEASE_RENEGOTIATING An extension to the lease has been sent and the lease is waiting for a confirmation. Events will continue to be received.
SimpleLeaseManager.LEASE_RELEASE The lease has be released. No events will be received (they are thrown away).

There are several parameters for the SimpleLeaseManager in the .PSEPRjClient configuration file:

<SimpleLeaseManager>
    <MaximumRetires>3</MaximumRetires>
    <-- default duration in seconds -->
    <DefaultLeaseDuration>120</DefaultLeaseDuration>
    <-- seconds to wait for lease confirmation -->
    <ShortTimeoutAbsolute>20</ShortTimeoutAbsolute>
    <-- percent of duration to wait before renegotiating the lease -->
    <LongTimeoutPercent>80</LongTimeoutPercent>
</SimpleLeaseManager>
              

5.3.1 Lease Management Helper Class

New in jClient 2.6 is a helper class that opens a lease and then watches it to make sure it stays active. Because ofthe dynamic configuration of the PsEPR servers, connection can go down. This requires a client to keep watch on the connection and remake it if it fails.

The LeaseHandler class will create a lease and start a thread that will continually check the health of the lease. If the lease fails, it will call jClient to renegotiate the connection to the servers and renegotiate the lease. If both of these renegotiations fail, an optional callback routine is called which the application can use to report connectivity failure.

The use of LeaseHandler usually looks like:

LeaseHandler inLease = null;
try {
  PayloadParser xParser = new PayloadGeneric();
  EventQueue eQueue = new EventQueue();
  String myChannel = "/edu/usgsr/grad/topoWatch/status/";
  // pConn is an open PsEPRConnection
  // note the use of 'null' for the LeaseWatcher callback
  inLease = new LeaseHandler(pConn, myChannel,  xParser, eQueue, null);
  // leases are not instantanious so we wait for it to be granted
  inLease.waitForLease();
}
catch (PsEPRException e) {
  System.out.println("GetLease failed:"+e.toString());
  pConn.close();
  return;
}
catch (Exception e) {
  // I don't know why this would happen
  System.out.println("GetLease exception:"+e.toString());
  pConn.close();
  return;
}
                

The last parameter to LeaseHandler is an instance of a class that implements the interface org.psepr.jClient.LeaseWatcher. The main feature of that class is the method public void LeaseStateChange(PsEPRLease theLease, String reason);.

5.4 Processing of a Received Event

This section describes in detail the internal processing of the reception of an event. The goal is to make explicit the separation of parsing and processing.

When a message is received from XMPP, it passes through two phases: parsing and processing. The steps are:

  1. When a <message> is received, it is parsed and EventExtension is called for the <event> sub-element;
  2. EventExtension parses the standard portions of the event ( <to>, <from>, <distribution>);
  3. when the <payload> element is encountered, the namespace is extracted and EventExtension calls LeaseCollection.parsePayload();
  4. LeaseCollection loops through each lease in the collection (all of the leases that are on the channel) and it calls PsEPRLease.parsePayload() on each;
  5. PsERPLease checks if a lease manager exists and, if so, it is passed to the LeaseManager for parsing;
  6. if not a lease message, the payload parser is called.
  7. the payload parser checks to see if the message is if it's a type it knows how to parse. If so, the parser parses the payload and returns a Payload object for the payload contents.
  8. the payload object is returned in the EventExtension where it is packaged into an Event object;
  9. the XMPP packet is eventually passed to PsEPRConnection.processPacket(). The event object is extracted from the message packet extension and LeaseCollection.receiveEvent() is called.
  10. This calls receiveEvent for each of the leases.

6 XML Pull Parser

The parsing of the XML is done with a "pull parser". The whole message is not parsed initially into DOM objects nor are events passed for the elements (al la SAX), but a parser moves through the XML and the application looks at the elements as they pass by.

The pull parser used in jClient is available at http://xmlpull.org/ where there are the jar files, an introduction to the library and Java API documentation.

The general form for parsing payloads is the following (using the attribute payload as an example):

public Payload parsePayload(String ns, XmlPullParser parser) {
  PayloadAttribute myPA = null;
  if (ns.equalsIgnoreCase(attributeNamespace)) {
    try {
      myPA = new PayloadAttribute();
      boolean done = false;
      while (!done) {
        int pI = parser.next();
        if (pI == XmlPullParser.START_TAG) {
          String elementName = parser.getName();
          if (elementName.equals("field")) {
            myPA.setAttributeName(PsEPRService.cleanXMLText(parser.nextText()));
          }
          if (elementName.equals("value")) {
            myPA.setAttributeValue(PsEPRService.cleanXMLText(parser.nextText()));
          }
        }
        else if (pI == XmlPullParser.END_TAG) {
          if (parser.getName().equals("payload")) {
            done = true;
          }
        }
      }
    }
    catch (Exception e) {
      // if the parser exploads, we have to bail
      throw new PsEPRException("PayloadLease parser failed: "+e.toString());
    }
  }
  return myPA;
}
          
Some unordered comments about the above example:
  • The parser routine is entered pointing at the payload start element so attributes can be fetched;
  • The routine must exit with the parser pointing at the payload END_TAG
  • the parser is advanced by calling next() or nextText(). Validation is not done and parsing can be pretty free form (note that this doesn't check for the end elements for field or value, for instance);
  • the cleanXMLText() utility routine cleans out the end-of-line characters and removes white space around the value. This could result in a zero length string or a null (if there was no text). But, in general, what happens is what is expected;
  • if anything fails in the parsing, this throws an Exception. Because there is no way to back out of parsing, the event is lost and no further processing will happen after this Exception.

7 jClient Configuration File

There are some environmental parameter defaults that can be set with a configuration file. The configuration file is named .PsEPRjClient.xml and is looked for in the following directories in the following order:

Linux Windows
System /usr/local/psepr/etc/psepr/jClient.xml c:\usr\local\psepr\etc\psepr\jClient.xml
User $HOME/.psepr/jClient.xml c:\Documents and Settings\USER\.psepr\jClient.xml
Application current directory/jClient.xml current directory\jClient.xml
The configuration file is read from each of these locations and newer values over write older values. Thus, the configuration is the union of all of the configuation files with the user files overwriting the system files. So, it is recommended that global information (routers and timing defaults) be kept in the system locations and user specific information (default service/password, specific debugging settings) should be in user specific configuration files.

The configuration file is an XML file. A full blown, fully specified file looks like:

<jClient>
    <router>
        <!-- the number of times to retry the connectin to the router -->
        <reconnectionRetries>5</reconnectionRetries>
        <!-- milliseconds to wait for XMPP connection and login responses-->
        <connectionTimeout>10000</connectionTimeout>
        <!-- expiration seconds for jabber messages -->
        <defaultExpiration>30</defaultExpiration>
        <!-- optional default SOCKS service to use -->
        <!-- if specified, this SOCKS service is used for all routers below -->
        <socks>
            <server>5222</server>
            <port>5222</port>
        </socks>
        <r001>
            <name>psepr_router</name>
            <server>jabber.services.planet-lab.org</server>
            <instance/>
            <!-- JID address of routers: usually combination of above -->
            <address>psepr_router@jabber.services.planet-lab.org</server>
            <!-- optional port specification -->
            <port>5222</port>
            <!-- optional per router SOCKS connection specification -->
            <socks> 
                <server>5222</server>
                <port>5222</port>
            </socks>
        </r001>
        <r002>
            <!-- the minimun that must be specified -->
            <server>planetlab2.cs.purdue.edu</server>
        </r002>
    </router>
    <!-- optional section the just specifies the name of the registry -->
    <registry>
        <r001>
            <name>psepr_registry</name>
        </r001>
    </registry>
    <!-- optional serviceID information used as a default by some tools -->
    <!-- As mentioned above, this should be in user local config files. -->
    <default-service>
      <service>myservice</service>
      <password>nooneknowsit</password>
    </default-service>
    <!-- parameters for the default simple lease manager -->
    <SimpleLeaseManager>
        <MaximumRetires>3</MaximumRetires>
        <!-- default duration in seconds -->
        <DefaultLeaseDuration>120</DefaultLeaseDuration>
        <!-- seconds to wait for lease confirmation -->
        <ShortTimeoutAbsolute>20</ShortTimeoutAbsolute>
        <!-- percent of duration to wait before renegotiating the lease -->
        <LongTimeoutPercent>80</LongTimeoutPercent>
    </SimpleLeaseManager>
    <debug>
        <!-- 'on' if to output all sent and received packets -->
        <detailedPacketPrintout>off</detailedPacketPrintout>
        <!-- controls the output of the debug logger -->
        <filterLevel>
            <!-- default filter output level -->
            0x0000
            <!-- filter flags for individual modules -->
            <modulename>0xFFFE</modulename>
        </filterLevel>
    </debug>
</jClient>
          

The specification of debug output is a bit array. The value given outside any modulename block is the default value that will be used everywhere. If a modulename block is given, the debug value applies only to that logging module. The bits have the following meaning:

Bits Code Meaning
0x0001
0x0002 IO general IO operations
0x0004 IODETAIL gory details about IO operations
0x0008 INVOCATION messages when key routines are entered
0x0010 DETAIL
0x0020 BADERROR messages about conditions that should not occur
0x0040 LEASE general messages about lease requests and status
0x0080 LEASEDETAIL detailed messages about lease processing
0x0100 PARAMDETAIL detailed info about parameter processing
0x0200 PARSEDETAIL detailed info on payload parsing. Can be very verbose and thus best done within a module block.
All of the debug messages are output by the library to standard out.

Because most of the values default, the absolute minimum configuration file only needs to list the routers to connect to:

<jClient>
    <router>
        <r001>
            <server>jabber.services.planet-lab.org</server>
        </r001>
        <r002>
            <server>planetlab2.cs.purdue.edu</server>
        </r002>
        <r003>
            <server>router.example.com</server>
        </r003>
        <r004>
            <server>router.example.de</server>
        </r004>
    </router>
</jClient>
          

7.1 User Specified Configuration Files

The configuration file search mentioned above can be replaced by a list of user specified files. Before the PsEPRConnection is used, a caller can set an array of files to load for configuration information. The user passes an array of filenames to PsEPRService.setConfigFiles() before the PsEPRConnection is created. This list of files replaces the redefined list of configuration files.

Some sample code for an application which takes configuration files as a parameter looks like:

ArrayList<String> userConfigs = new ArrayList<String>();
...
for (int jj=0; jj<args.length; jj++) {
    ...
    if (args[jj].equalsIgnoreCase("--config") {
        userConfigs.add(args[++jj]);
    }
    ...
}
...
if (userConfigs.size() > 0) {
    PsEPRService.setConfigFiles((String[])userConfigs.toArray());
}
...
PsEPRConnection pConn = new PsEPRConnection(...);
...
              

The configuration files are in exactly the same format as described above. If multiple files are specified, they are read in order from index 0 up and later configuration files overlay the values specified in earlier files.

8 Router Selection

The configuration file lists several PsEPR Routers that the library can connect to. When a new PsEPRConnection is created, one of these Routers is chosen. What follows is a description of how that is done within the library and how it can be extended.

When a new PsEPRConnection is created, it calls PsEPRService.getRouterManager().getNextRouterToTry(). This routine, a method of the RouterManager interface, returns an instance of the class PsEPRRouter. PsEPRConnection tries to connect to the specified Router and if it fails, it calls PsEPRServcice.getRouterManager().routerFailed() with the router that didn't work. It then calls getNextRouterToTry() to get another router.

In this way, PsEPRConnection steps through all the potential Routers to find one that is working.

If a RouterManager is not set before the first creation of a PsEPRConnection, an instance of SimpleRouterManager is created. SimpleRouterManager implements a weighting of the routers: successful connections add to the weight and time decreases the weight. The current values of the weights are saved in the file /usr/local/psepr/etc/jClientRouterInfo.xml or \Documents and Settings\USER\.psepr\jClientRouterInfo.xml depending on the OS. The routers listed in jClient.xml use the saved weight valued from these files to probablistically choose the routers to return as getNextRouterToTry() is called. Nothing bad happens if these files are missing -- it just might take longer to connect next time.

This selection process can be extended by providing a user created class which implements the RouterManager interface. Before the first PsEPRConnection is created, the calling routine can set a manager by calling PsEPRService.setRouterManager() passing a class which implements the RouterManager interface. The user's manager then returns Routers to PsEPRConnection based on any criteria it wishes (locality to calling host, existance of existing connections, etc).

9 Schemas

Schemas are available for some of the PsEPR XML items. Here is the current list:

Item Schema URL
Event
Namespace: http://org.psepr/xs/event
http://dsmt.org/schema/psepr/event-1.0.xsd
Attribute Payload
Namespace: http://org.psepr/xs/payload/attribute
http://dsmt.org/schema/psepr/payload/attribute-1.0.xsd
Lease Payload
Namespace: http://org.psepr/xs/payload/lease
http://dsmt.org/schema/psepr/payload/lease-1.0.xsd

Revision History

1.7 2007-06-10 RA Changed URL to the list of active PsEPR servers to be at psepr.org rather than at dsmt.org .
1.6 2006-09-29 RA Removed pointer to PsEPR account signup. Refer people to PsEPR maintainer for accout aquisition.
1.5 2006-04-10 RA Added SendOneEvent section.
1.4 2005-11-22 RA Updated schema information to include namespace. Added section on LeaseHandler.
1.3 2005-09-20 RA Added schema information. Corrected comments on where configuration file comes from. Added section of user specified config files.
1.2 2005-04-15 RA Added description of how new SimpleRouterManager works and location of the weight files.
1.1 2005-02-13 RA Update to configuration file description.
1.0 2005-01-01 RA Initial release