(back to SPIRO Homepage)

SPIRO Manual

Introduction

SPIRO - Simple Python Interface to Remote Objects - is a framework that allows objects to be shared between n different processes.

It is aimed particularly at providing CPython programs to access java libraries, functions and objects, via Jython, in a way that doesn't require a brittle compilation process.

The obvious advantage to this being that you get to have your cake and eat it too; you can stay in CPython, with its speed and access to binary python modules, yet you can interoperate with Java and take advantage of the huge repository of Java libraries.

In fact, SPIRO makes it possible for you to distribute CPython programs, along with a self-contained .jar file, such that the user only needs to have Python plus a Java runtime environment installed - no Jython needed.
(If targetting at windows, you can compile your python to a .EXE, build the SPIRO server-side to a self-contained .jar file, and toss in a java runtime environment, for a fully self-contained program distribution that doesn't require Joe Sixpack Windows User to install any third-party stuff at all).
This brief manual should be (at least just) enough to get you up and running with SPIRO. You should try the examples herein, and get a hands-on feel for how things work. This will give you a quick grasp of SPIRO, and how it can help you.

Prerequisites

To develop CPython->Java code using SPIRO, you will need: For your users to run SPIRO-using programs you've written, they will need: Note that if you're targetting your program at Windows, you can release the user of these requirements by:
  1. Building the Python client-side of your programs into a Windows EXE file, with the Py2EXE or the McMillan Installer.
  2. Including a Java Runtime Environment with your program

Installation

Assuming you've already installed Python, Java SDK and Jython, all you need to do is unpack the SPIRO distribution tarball and start writing your client program.

To test your setup, cd into the directory where you've unpacked the SPIRO tarball, then:
  1. Open one terminal window, and type:
    jython spiroserver.py
  2. Wait till you see a message indicating that the server is ready for connections
  3. Open another terminal window, and type:
    python spiroclient.py
  4. In the client terminal window, you should see something like:
    $ python spiroclient.py
connecting to Spiro server...
Importing java package
Grabbing java's sqrt() function...
Representation of this function:
<java function sqrt at 8225050>
Executing sqrt(16)
got 4.0
$

Basic Concepts

These are some of the underlying concepts behind SPIRO, in no particular order:

Getting Started - quick tutorial

While learning to use SPIRO (or trying out new stuff), it's a good idea to run a server session in its own window, and an interactive cPython session in another.

Within the interactive python session, instantiate a server connection: 
      $ python
      >>> import spiroclient
      >>> c = spiroclient.SpiroClient("fred")
      >>>
Now, a namespace has been created for you on the server. You can look at it by: 
      >>> c.dir()
      ()
      >>>
There are a couple of magic methods of server connection objects. .dir() is one of them, which lists out the names in the server-side namespace. As you can see in the above example, the namespace is currently empty.

So let's stick something into the namespace - for instance, let's do a remote import of the java environment: 
      >>> c.do("import java")
      >>> c.dir()
      ('java',)
      >>>
Here we see another 'magic method' - .do(). This method causes arbitrary python statements to be executed within your namespace on the server. You'll also see that the namespace now contains a symbol called 'java'.

Now, let's take a ref to the java module, and display the ref: 
      >>> java = c.java
      >>> java
      <java package java at 17705039>
      >>>
That's right - we've got a local ref to a remote object, in this case, a module object.

Now let's actually do something. 
      >>> java.lang.Math.sqrt(25)
      5.0
      >>>
As you can see, we've called a remote function just as if the function existed locally.

Similarly, you can instantiate remote classes, set/get their attributes, call their methods to your heart's content.

Now, let's try another trick - importing a server-side package directly onto the client: 
      >>> org = c.doimport("org")
      >>> org
      <java package org at 7607272>
      >>>
Note that this .doimport() method doesn't import the module into the server-side namespace, the module is instead returned directly to the client (or, more correctly, wrapped into a proxy object on the client).

Lastly, you can stick stuff into the server-side namespace by setting attributes of your connection object. For example: 
      >>> c.dir()
      ('java',)
      >>> c.alice = 33
      >>> print c.alice
      33
      >>> c.bob = "hello"
      "hello"
      >>> c.dir()
      ('java', 'alice', 'bob')
      >>> c.fred = ["one", "two"]
      >>> c.dir()
      ('java', 'alice', 'bob', 'fred')
      >>> c.fred
      ["one", "two"]
      >>> c.fred.append("three")
      >>> c.fred
      ["one", "two", "three"]
      >>>
Note in this example that the list object assigned to 'fred' in the server namespace actually resides on the namespace. That's because it's a mutable object.

When we typed the command, c.fred, the local python interpreter invoked the __repr__ method of c.fred, or more precisely, our client-side proxy object for c.fred. In this case, SPIRO transparently fetched fred's __repr__ attribute from the server, then placed a call to the server on the __repr__ method, and got back a string representation.

Similarly, the statement, c.fred.append("three"), actually causes the following: But you don't need to worry too much about this - it all happens in the background.

Please note - you should not try to assign functions, methods or class objects, in fact anything that can't be pickled, directly onto the server. For example: 
      >>> def mary():
      ...   return "this is mary"
      >>> c.mary = mary
will cause Really Bad Things to happen. The way around this is to set up a callback (see below).
One last thing for this intro - if you want, you can have two or more clients sharing the same namespace on the server. All they have to do is instantiate a server connection, using the same session name.

What About Callbacks?

Callbacks work - you can have objects on the server, which call client-side functions or methods. You can even have objects on one client calling functions/methods on another client.

There is a trick though - to make your local thing callable from the server-side, you have to wrap it with the connection object's .callback() method: 
      >>> def myfunc():
      ...   return "this is myfunc"
      >>> c.myfunc = c.callback(myfunc)
Then, objects on the server can call your client-side callables. This is particularly useful when you need to pass callables on the fly as callback args to java functions/methods.

What About Exceptions?

Server->Client Exceptions are supported, but only partially. When you do something on the client that causes something to happen on the server which triggers an exception, the server will send an exception message to the client, including the exception name, args and a text stack backtrace. All this info will be wrapped on the client into an exception object of the same name, with the same arguments, but also the server-side stacktrace added on.

What About Subclassing?

This is a more arcane area, which deals with esoteric internals of what actually happens when you subclass something. I am a long way away from mastering this, so SPIRO at this stage cannot support class interitance relationships that cross server-client boundaries.

This being said, though, there's nothing to stop you from building client-side classes that contain server-side objects, and/or vice-versa.

Deployment Issues

As all developers know, it's one thing to get a program working on one's own box, but it's a whole new set of issues to package the program so that users can readily install it on their machines without undue pain and complication. In this section, we discuss ways of making your SPIRO-using applications easier for users to set up.

Server-Side

You have two basic choices with the server-side code:
  1. Package up the whole server side into a single .jar file. The beauty of this option is that the entire server-side can be launched with a single java -jar myspiroserver.jar command. Also, it will likely reduce the download size for the user, since this .jar file will only contain what it really needs.
  2. A-la-carte - create one .jar file with the spiro server, and possibly other .jar files with any jython code you want to import (if any). This requires you to package the core jython.jar file, plus .jar files for any java libraries which your code uses. Also, it doesn't allow launching via the java -jar command; instead, to launch the server, your prog needs a script which sets the CLASSPATH to include your own jars, jython.jar plus all library jars, then specify the right jar to start with.
We'll assume here that you watn to go with the first option.

There's a few simple steps for creating a standalone .jar file for the server-side of your program:
  1. Create a Jython module (use myserver.py as a template if you want) which imports spiroserver, instantiates a SpiroServer object, and runs it with required options
  2. At the start of your file, explicitly import all java (and/or jython) modules and packages your client-side progs will need. This is crucial - if you don't do this, the java packages won't be importable to your client.
  3. Take a copy of Makefile.myserver, read the directions in the comments, and edit it to your needs.
  4. type make -f <your-makefile> to build your server.

Over to You

Well, this has been a really quick walkthrough of SPIRO. But it should have been enough to whet your appetite, and give you at least a sense of what you can and can't do with SPIRO.

I wrote SPIRO because I wanted an easy way to interface between real Python and java.

Jython is great, but comes at a harsh price - you lose access to python packages that contain binary code. Also, since Jython consists of a VM running atop another VM, it is not exactly, er, fast.

I tried some JNI-based wrapper frameworks such as JPE, but found the build to be very brittle, and nowhere stable enough for deploying any apps I write - a high percentage of users would have trouble building the JPE layer, and give up on installing my app.

So I wanted something which would allow my cPython apps to use Java libraries, and be able to run on a user's box, where the user only has to install python and a java jre, or (in the case of windows, neither of these).

As you can see from the command/response sequence above, there's a lot of socket communication happening, especially with compound attribute lookups and calls. SPIRO can get very slow, but you can speed it up by saving attributes locally. Also, you can speed it up by importing/creating stuff on the server side.

With a bit of trial and error, you'll find a scheme for storing object refs that gives you the fastest possible access to java's facilities.
Lastly, in conclusion, have a read through the SPIRO source. I've tried hard to make (at least most of it) reasonably clear and easy to understand.
David McNab
Last modified: Sun Apr 11 14:28:22 NZST 2004