samskivert
samskivert » code

January 24, 2010

“Duplicate field expected”

In my recent programming adventures, I separately encountered two perplexing error messages:

  • RuntimeException: Variable is used without definition!
  • IllegalArgumentException: Duplicate field expected

Inspection of the underlying code brought clarity:

  • throw new RuntimeException("Variable " + m + " used without definition!");
  • throw new IllegalArgumentException("Duplicate field " + fieldName);

The former was reporting a problem with a variable declared as InputStream is, the latter a problem with JUnit’s @Test annotation which defines a parameter java.lang.Class<? extends java.lang.Throwable> expected.

Tip from the pros: always wrap identifiers in quotes when including them in error messages.

Posted to code by mdb at 4:33 pm | Comments (1)       

November 4, 2009

Thank Gosling

The following is not legal Java:

    public static void main (String[] args) {
        int[] foo = new int[1];
        int[] bar = new int[1];
        ((args.length == 0) ? foo[0] : bar[0]) = 1;
    }

That makes my life easier.

Posted to code by mdb at 5:25 pm | Comments (2)       

October 9, 2009

GWT AsyncGen

The back-breaking straw landed on this camel earlier today when adding some methods to a GWT RemoteService interface and then remembering that, “Oh yeah, I have to go add those manually to the async mirror interface that goes with my service.”

First I poked around on the web. I found some sort of Maven plugin that in theory automatically generates these extra services, but I don’t use Maven and I’m not about to start now. Plus it had some bugs filed against it that made me think it was written by a barrel full of monkeys. There was some sort of IDE plugin, but I’m definitely not going to start using some random IDE. I also found some crazy old command line tool that harkened back to the @gwt.typeArgs days. No good.

So I did what I should have done a couple of years ago and wrote an ant task to DWIFN and to do it properly. So if you find yourself using GWT presently or in the future, run, don’t walk, to the following URL and spend the two minutes it will take to save yourself a bunch of annoying typing:

http://code.google.com/p/gwt-asyncgen/

Posted to code by mdb at 6:57 pm | Comments (0)       

June 17, 2009

A technique for type-safe, concise actors in Java

Actors are great. They provide a concurrency model that is not too taxing to reason about and an organizing principle that tends to jive with how you would naturally structure your code. I recently put actors to use in a real project (implemented in Java), and in the process had to overcome some aesthetic difficulties with a technique that seemed worth sharing.

The Itch

The requirements I was trying to satisfy are as follows (note: I realize that most people use the term messages to describe what actors process but I prefer to say that actors process actions, so hopefully you can bear with my rogue nomenclature):

  1. I want the declaration of my actor’s public interface to be like declaring a set of public methods. The declaration should be concise and should be easily documented to communicate to the caller what each action does.
  2. I don’t want the act of posting an action to an actor to be dramatically syntactically more verbose than just calling a method on the actor.
  3. Conversely, I want posting an action to an actor to be visibly distinct in the code. The programmer needs to know that the operation will be performed asynchronously, so it should not look exactly like calling a normal method.
  4. When implementing my actors, I want to simply define methods that are invoked with actor semantics (these methods are only ever executed by a single thread at any particular time). I don’t want an explosion of anonymous inner classes.
  5. My actions may take arguments.
  6. I want as much help from the compiler as possilble.

I am using the Functional Java actor framework, which provides a QueueActor that implements the actor semantics I desire given an ExecutorService. Using the QueueActor is pretty straightforward:

class Action { /* some action class */ }
Strategy<Unit> strategy = Strategy.executorStrategy(someExecutor);
QueueActor<Action> actor = QueueActor.queueActor(strategy, new Effect<Action>() {
    public void e (Action action) {
        // handle your action, blissfully ignore concurrency
    }
});
actor.act(new Action(...));

However, you can imagine that using this building block to meet my requirements was going to require some effort. After trying and discarding a number of unsatisfactory approaches involving large quantities of boilerplate and/or a lack of type safety, I eventually arrived at a technique that avoids both. The key is to use generics and Java’s wee bit of type inference to our advantage.

The Scratch

To see how this all hangs together, let’s start by looking at an implementation of a basic actor using the technique. As you can see below, we have created an AbstractActor class and our actor implementations derive from that.

public class Thingy extends AbstractActor<Thingy>
{
    /** Instructs the thingy to initialize itself.
      * Param: File - the directory in which the thingy operates. */
    public static final Action1<Thingy, File> INIT =
        newAction(Thingy.class, "init", File.class);

    /** Instructs the thingy to shut itself down. */
    public static final Action<Thingy> SHUTDOWN =
        newAction(Thingy.class, "shutdown");

    public Thingy (Strategy<Unit> executor) {
        super(executor);
    }

    /** Handles the {@link #INIT} action. */
    protected void init (File dir) { ... }

    /** Handles the {@link #SHUTDOWN} action. */
    protected void shutdown () { ... }
}

To send this actor an action, we do the following:

thingy.act(Thingy.INIT, new File(somedir));
thingy.act(Thingy.SHUTDOWN);

Going back to my requirements, we can see that a few of them are already in good shape:

  1. We define the actor’s actions with fairly concise constant objects which also serve as a convenient place to provide documentation for the actions. The actions are essentially the public interface of the actor.
  2. Calling a method on an actor via the act() method is pretty concise. We do have to repeat the actor class name when supplying the action constant but that’s not a horrible burden. You could even use static imports to lessen that burden, but I don’t advocate such craziness.
  3. act() makes it pretty clear, without being obtrusive, that we’re sending an action to an actor and not just calling a normal method.
  4. Actions are implemented directly by protected (or private if you swing that way) methods in the actor class. There is no manual wiring and it’s pretty easy to see what’s going on in the actor code.
  5. It’s very easy to indicate that actions take arguments and to pass those arguments to a call to act(). No special classes are needed to wrap up the arguments to an action, and the arguments show up just where you want them as parameters on the method that implements the action.
  6. It’s not entirely obvious without seeing the code for AbstractActor, but all of this is completely type-safe. If you try to pass a Thingy action to a Whatsit actor’s act() method you get a compile error. If you try to pass a String to an action that takes a File, you get a compile error. If you neglect to pass an argument to an action that takes an argument you get a compile error.

The Gory Details

Let’s take a look behind the curtain and see how it works. First let’s look at what newAction() does:

public abstract class AbstractActor<T extends AbstractActor>
{
    // ...
    protected static <A extends AbstractActor> Action<A> newAction (Class<A> clazz, String name) {
        return new Action<A>(resolveMethod(clazz, name));
    }

    protected static <A extends AbstractActor, A1> Action1<A, A1> newAction (
        Class<A> clazz, String name, Class<A1> arg1) {
        return new Action1<A, A1>(resolveMethod(clazz, name, arg1));
    }
    // ...
}

Here’s where we start to get into some type gymnastics. newAction() returns an Action class (or subclass) parameterized by the types of the actor class and the arguments to that action. So when we call newAction(Thingy.class, “init”, File.class) above, we get back an Action1<Thingy, File> object. This is critical to propagating the type information to the act() method and enforcing the correct types and count for the action arguments. Let’s take a look at act() now:

public abstract class AbstractActor<T extends AbstractActor>
{
    // ...
    public void act (Action<T> action) {
        enact(action);
    }

    public <A1> void act (Action1<T, A1> action, A1 arg1) {
        enact(action, arg1);
    }
    // ...
}

There are a couple of things going on here. First, you can see that AbstractActor is parameterized in the type of itself, or rather of the type of the class that extends abstract actor. So we have Thingy extends AbstractActor<Thingy>. This then requires that all act() methods take Action objects that are typed Action<Thingy> and makes it a compile error to supply an Action typed for some other AbstractActor derived class.

To enforce the correct number of arguments, we simply have unique Action derivations for every arity. Action for zero-argument methods, Action1 for one-argument methods, and so on. In the examples I’ve only provided guts for zero and one argument actions, but you can trivially extend this to as many arguments as you desire.

Finally to enforce the type of the arguments, we use Java’s limited, but useful, generic type inference. By declaring our second act() method with type <A1>, which is inferred from the type of the Action1<T, A1> that is supplied when you call the method, we define A1 via the first parameter and the compiler requires that the subsequent “A1 arg” parameter match. So the type that was provided back in our newAction() call is properly preserved and enforced any time that Action is passed to an act() method.

Note that I’m glossing over argument covariance here for clarity. What you really want is:

    public <A1, B1 extends A1> void act (Action1<T, A1> action, B1 arg1) {

This allows you to declare a method that takes Number, for example, and pass an Integer to it. Java methods are covariant in their argument’s primary types, so we probably want actor actions to be the same.

Now let’s look at how we wire up and dispatch the action implementation methods internally. As you may suspect, we use reflection to do this. Normally one gives up compile time type checking when making use of reflection, but we’ve fortunately already handled that. One downside is that reflective method invocation is slower than a non-reflective method invocation. However, later I’ll show you how you can do away with the reflective dispatch for specific situations where you’ve discovered that performance is actually critical. This comes at the cost of some boilerplate which is why we prefer the reflective dispatch by default.

Now then, back to the implementation. You’ll notice that newAction() above calls a method “resolveMethod()” and passes its results into the Action constructor. This method is pretty simple. newAction() takes the name of the method and the types of its parameters, and Class provides a mechanism to look up a method given that exact information. We’d just call it directly except that unfortunately NoSuchMethodException is a checked exception.

public abstract class AbstractActor<T extends AbstractActor>
{
    // ...
    protected static Method resolveMethod (Class<?> clazz, String name, Class<?>... ptypes) {
        try {
            return clazz.getDeclaredMethod(name, ptypes);
        } catch (NoSuchMethodException nsme) {
            throw new IllegalArgumentException("Unable to resolve actor method: " + name, nsme);
        }
    }
    // ...
}

There’s another downside to using reflection. We specify the name of the reflected method to be called as a String passed to newAction(). What if you misspell that? What if you pass the argument classes in the wrong order? Since newAction() is called to initialize static fields, it will be called as soon as the Thingy class is resolved. This means that as soon as the Thingy class is referenced (a class that invokes actions on Thingy is resolved, or an instance of Thingy is created, etc.), the programmer finds out that they made a mistake in an action declaration. This is not delayed until the program actually sends the action in question to the actor. So while the class still has to be loaded, which could conceivably not happen immediately upon program startup, we’re still very likely to discover any problems as soon as possible during the development process.

The remainder is pretty simple. The Action class simply reflectively invokes the method when it is acted upon and AbstractActor’s enact() method simply queues up a Runnable to invoke the Action using appropriate actor semantics. I’ll include the entirety of AbstractActor here so that you can see how it all hangs together.

import java.lang.reflect.Method;

import fj.Effect;
import fj.Unit;
import fj.control.parallel.QueueActor;
import fj.control.parallel.Strategy;

public abstract class AbstractActor<T extends AbstractActor>
{
    public void act (Action<T> action) {
        enact(action);
    }

    public <A1> void act (Action1<T, A1> action, A1 arg1) {
        enact(action, arg1);
    }

    protected void enact (final Action<T> action, final Object... args) {
        _actor.act(new Runnable() {
            public void run () {
                @SuppressWarnings("unchecked") T self = (T)AbstractActor.this;
                action.invoke(self, args);
            }
        });
    }

    protected AbstractActor (Strategy<Unit> execor) {
        _actor = QueueActor.queueActor(execor, new Effect<Runnable>() {
            public void e (Runnable action) {
                action.run();
            }
        });
    }

    protected static <A extends AbstractActor> Action<A> newAction (Class<A> clazz, String name) {
        return new Action<A>(resolveMethod(clazz, name));
    }

    protected static <A extends AbstractActor, A1> Action1<A, A1> newAction (
        Class<A> clazz, String name, Class<A1> arg1) {
        return new Action1<A, A1>(resolveMethod(clazz, name, arg1));
    }

    protected static Method resolveMethod (Class<?> clazz, String name, Class<?>... ptypes) {
        try {
            return clazz.getDeclaredMethod(name, ptypes);
        } catch (NoSuchMethodException nsme) {
            throw new IllegalArgumentException("Unable to resolve actor method: " + name, nsme);
        }
    }

    protected static class Action<A extends AbstractActor> {
        public Action (Method method) {
            _method = method;
        }

        public void invoke (A actor, Object... args) {
            try {
                _method.invoke(actor, args);
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }

        protected Method _method;
    }

    protected static class Action1<A extends AbstractActor, A1> extends Action<A> {
        public Action1 (Method method) {
            super(method);
        }
    }

    protected QueueActor<Runnable> _actor;
}

The Fine Print

I mentioned above that I’d indicate how to avoid the reflective call. You can probably figure this out yourself now that you’ve seen all the code, but just in case you’re conserving brain power for more important projects, I’ll show you an example of how to do that here:

public class Thingy extends AbstractActor<Thingy>
{
    public static final Action1<Thingy, String> ZIP = new Action1<Thingy, String>(null) {
        public void invoke (Thingy actor, Object... args) {
            actor.zip((String)args[0]);
        }
    };

    /** Implements the {@link #ZIP} action. */
    protected void zip (String zoom) { ... }
}

Note that you don’t have to worry about checking that args has one argument or that the argument is a String, because the compiler is ensuring that you can only call act(Thingy.ZIP, …) with a single string argument. It is worth noting that if you want primitive arguments you have to worry about both boxing and unboxing and that someone could do something stupid like: actor.act(Thingy.FOO, (Integer)null), but fortunately that brand of stupidity usually doesn’t happen accidentally.

You also might be wondering about actors sending responses to other actors and callers magically blocking until they receive their response. Functional Java’s actor library doesn’t support blocking on a response from an act() call, so that’s out. I don’t think it would be incompatible with this technique were it to be supported. In terms of sending responses at all: you could pretty easily pass a concrete actor to another actor and have that actor send an action in response. The trickier thing would be if you wanted to support sending a response to, say, any actor that supported the appropriate response action. Without having actually tried it, I think it would probably be possible using some clever definition of interfaces and more type gymnastics. I’ll leave that as an exercise for the reader or for myself and a future blog post.

Posted to code by mdb at 1:19 pm | Comments (0)       

November 5, 2006

Java Puzzlers (3)

An excellent compendium of misleading and non-intuitive constructions in the Java language. A small few are contrived, but most ring true with my ten years of experience with the language. I have discovered more than a few of them “the hard way”. Highly recommended for anyone who writes a lot of Java code.

Posted to books| code by mdb at 5:05 pm | Comments (1)       

August 6, 2006

Scala

Scala has piqued my interest more than any language to come down the pike in a long time. Type inference makes me cry out with joy, Java-compatibility, sensible mix-in support (made all the more useful by their very sophisticated type system), first-class functions, continuations (mostly), very SML-like type pattern matching, list comprehensions. One could get into some serious trouble with this language.

I was then even more excited when I saw a paper by the language designer describing how he was able to easily implement an elegant stackless threads system as a library using only the functionality provided by the language (and the small compromise of requiring a continuation passing style, but the elegance of the language construct being used makes that hardly noticable).

I wish I wasn’t mired in a bastard mashup of Java, JavaScript and ActionScript at the moment. I’ll have to wait a bit before giving Scala a thorough run around the block.

Posted to code| geek by mdb at 7:15 pm | Comments (0)       

July 7, 2004

Random bitching

What kind of error message is this?

[11:20pm] baltic:~ > java -jar foo.jar
Exception in thread "main" java.util.zip.ZipException: No such file or directory
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.(ZipFile.java:112)
        at java.util.jar.JarFile.(JarFile.java:127)
        at java.util.jar.JarFile.(JarFile.java:65)

It doesn’t even tell me what file or directory is missing. You might think it’s obvious because I just ran the command, but frequently java is run from a shell script and all I see is this baffling crap about zip files.

I was about to suggest that it be more like the error message one gets when specifying a class that doesn’t exist:

[11:24pm] baltic:~ > java nonexistent.Class
Exception in thread "main" java.lang.NoClassDefFoundError: nonexistent/Class

However, that largely sucks as well. At least its not six calls deep into classloader code, which would be on par with the zip business, and it does tell me which class it couldn’t find, but it’s still a sorry excuse for an error message. Is it that difficult to catch the most common exceptions and report something sensible?

Posted to code by mdb at 11:31 pm | Comments (0)       

MDB

Bits
Ludography
Histoire

samskivert.com ©2001 - 2009 Michael Bayne <mdb@samskivert.com>