samskivert » code

August 25, 2010

Shoot me now

I’m foolishly trying to write a javac annotation processor in Scala. This has already been a road fraught with gratuitous obstacles, but the latest wrinkle has driven me to vent in public.

When I run my annotation processor thusly:

javac -processor FooProcessor -processorpath fooproc/classes:blah/scala-library.jar SomeFile.java

I get “java.lang.IllegalStateException: zip file closed” when my processor tries to load Scala classes. But if I unpack scala-library.jar into fooproc/classes then it works. Joy! Somehow the scala-library.jar file causes javac to choke.

I of course tried repacking the jar file, in case perhaps something in the stock scala-library.jar was weird, but using the repacked version still failed with “zip file closed.” What’s even more awesome is that javac 1.6 does not have a problem with scala-library.jar, only javac 1.7.

Posted to code by mdb at 3:30 pm | Comments (0)       

August 12, 2010

An emacs on every desktop

I am a heavy user of virtual desktops. I generally have a couple of shell windows, an emacs window, possibly a browser, and whatever target program I’m working on, on each of a few virtual desktops. When I’m interrupted from one project, I flip to the next virtual desktop, open a shell window, open an emacs window and start hacking away on said interruption. Often the stack will get three or four virtual desktops deep, and I’ll find myself flipping back and forth between the various in-progress projects.

I generally try to open new files from within emacs, rather than from a terminal, but occasionally the terminal is more expedient, except that it results in an entirely new emacs process and window being created on that virtual desktop. The additional virtual memory consumption isn’t a big deal, but the fact that it screws up my ability to alt-tab between windows is a major annoyance. Plus my brain naturally assumes there’s one emacs process on each virtual desktop and that process contains all the buffers that have been opened thereon. So when I go to switch to a buffer that I know is open only to discover that it’s in a different emacs process whose window is hidden behind the one I’m currently using, it harshes my mellow.

I have played with emacsclient a bit in the past, but it assumes that you use one emacs to rule them all. So if I have an emacs open on virtual desktop one, and run emacsclient on virtual desktop three, it will happily open the requested file on the completely invisible and completely inappropriate emacs instance on virtual desktop one. Not useful. Further, it assumes that you’re the sort of developer that started emacs back in the late seventies and have kept that instance running ever since. So if you run emacsclient when there’s no emacs instance running, it happily spits out seven or eight lines of error message and does nothing especially useful.

For whatever reason (probably me subconsciously avoiding the tedious work that I have to get done before the end of this weekend), the camel’s back broke today (even though this has been bothering me since I started using FVWM in like 1995). I scoured the interwebs for a way to find out what virtual desktop was active (wmctrl FTW) and wrote the necessary scripts and elisp to cause “emacs filename” executed on the command line to always do the Right Thing. If there’s no emacs instance on the active virtual desktop, one is started with the requested file(s). If an instance is running on that virtual desktop, it is instructed to open the requested file(s). And as an added bonus, since I use “emacs -nw” for $EDITOR jobs, if the script sees -nw on the command line, it just forks off a whole new emacs with the supplied args, knowing that it will remain safely ensconced in the terminal.

If you too are an emacs user whose workflow resembles mine, feel free to reap the fruits of my labor. Add the following to your .emacs file:

(defun cur-desk ()
  "Returns the numeric identifer of the current desktop."
  (replace-regexp-in-string "\\(^[[:space:]\n]*\\|[[:space:]\n]*$\\)" ""
    (shell-command-to-string "wmctrl -d | grep '\*' | awk '{ print $1 }'"))
  )
(if (string= "x" window-system)
    (progn (setq server-name (format "server%s" (cur-desk)))
           (server-start)))

and create a shell script in your favorite location with the following contents:

#!/bin/sh

EMACS=/usr/bin/emacs

# if we have -nw on the command line, invoke a separate emacs instance
for ARG in $*; do
    if [ $ARG = "-nw" ]; then
        exec $EMACS $*
    fi
done

# otherwise send the file(s) to the emacs instance on this virtual desktop
CURDESK=`wmctrl -d | grep '\*' | awk '{ print $1 }'`
emacsclient -s "server$CURDESK" --no-wait $* > /dev/null 2>&1
if [ $? != 0 ]; then
    # no instance running on this virtual desktop, so start one
    $EMACS $* &
fi

There are probably better ways to accomplish both of the above bits of code. My bash and elisp skills have both perpetually lingered in the “good enough to get the job done” realm.

Side note: I can’t believe the SyntaxHighlighter WordPress plugin doesn’t have support for Lisp. They support actionscript3, bash, coldfusion, cpp, csharp, css, delphi, erlang, fsharp, diff, groovy, javascript, java, javafx, matlab, obj-c, perl, php, text, powershell, python, ruby, scala, sql, vb, xml, and have the audacity to claim that “most widely used languages are supported”. Perhaps Lisp no longer rates as widely used. Ah well, I just used the Ruby highlighter. Ruby is just an accessible Lisp right?

Posted to code by mdb at 9:27 pm | Comments (2)       

June 9, 2010

Riddle me this

I don’t expect anyone to know the answer to this question, but I’m hoping that bitching about it will improve my mood sufficiently to prepare me for the painful foray into the guts of javac that will be needed to find the answer myself.

There’s a very sophisticated pile of code in com.sun.tools.javac.comp.Infer that infers the type of a universally quantified method application, given its signature and actual parameter types. To clarify what that means, let’s look at some code. Suppose you have an interface like the following and a universally quantified method which uses it:

    public static interface Predicate<T> {
        boolean apply (T arg);
    }

    public static <T> Predicate<T> id (Predicate<T> pred) {
        return pred;
    }

    public static final Predicate<Integer> FALSE = new Predicate<Integer>() {
        public boolean apply (Integer arg) {
            return false;
        }
    };

    public static void testId () {
        assert id(FALSE).apply(0) == false;
    }

If you were to put on your magical x-ray glasses and watch the compiler as it processed this code, you’d see at some point a call to Infer.instantiateMethod where it attempted to determine the type of the expression id(FALSE). This call would be told that it has a method of type <T>(Predicate<T>)Predicate<T> (which means we have a method forall T that takes a Predicate<T> and returns a Predicate<T>), and it has a list of type arguments to be inferred, just T, and that the type of the single argument to the method is Predicate<Integer>.

Infer.instantiateMethod is very smart, and it will tell us that this particular expression has type (Predicate<Integer>)Predicate<Integer>. This is just what I would expect.

Now suppose we decide to get fancy, and make our id() method contravariant in its type parameter:

    public static <T> Predicate<T> id (final Predicate<? super T> pred) {
        return new Predicate<T>() {
            public boolean apply (T arg) {
                return pred.apply(arg);
            }
        };
    }

Don’t mind the fact that the body of the method got more complex. That’s invisible to Infer.instantiateMethod, it only sees the type signature. So as far as it’s concerned, the only change is that the Predicate<T> formal parameter type is now Predicate<? super T>.

However, when we put on our x-ray specs, we see some funny behavior from Infer.instantiateMethod. Now instead of typing id(FALSE) as (Predicate<? super Integer>)Predicate<Integer>, which is what I would expect, it types it as (Predicate<? super T>)Predicate<T>. Note that this is not the same as the uninstantiated type of the method which is <T>(Predicate<? super T>)Predicate<T>. It has thrown away the universal quantification and given us back a type that contains free type variables.

So clearly there must be some other magic somewhere in the compiler that can somehow differentiate between a sanely typed method and a bizarrely typed method, and knows what to do with those free type variables. I’m not especially looking forward to tracking that code down and figuring out how it works.

Posted to code by mdb at 6:03 pm | Comments (0)       

May 16, 2010

Life on the edge (case)

My current research project has given me myriad opportunities to break the Java compiler. Usually, this is because I’m tinkering with its internal data structures in a way that it neither anticipated nor feels especially good about. In such circumstances, I usually apologize profusely and look for a less intrusive way to accomplish my goals. Today however, I managed to cause javac to take umbrage without having first gone rummaging around in its underwear drawer.

Consider the following (admittedly useless) code:

public class Test {
    public class Inner {}
    public static void main (String[] args) {
        Test test = new Test();
        test.new Inner() {
        };
    }
}

It compiles and runs without incident. The following equally useless variation also compiles and runs without problem:

    public static void main (String[] args) {
        Object test = new Test();
        ((Test)test).new Inner() {
        };
    }

However, the following variation successfully compiles into bytecode which generates a verifier error:

    public static void main (String[] args) {
        Object test = new Test();
        Test.class.cast(test).new Inner() {
        };
    }

Exception in thread "main" java.lang.VerifyError: (class: Test, method: main signature: ([Ljava/lang/String;)V) Incompatible argument to function" to be precise.

Having perused the internals of javac, I know that there is some hackery magic that takes place in relation to some of java.lang.Class's methods. So to be sure, I tried doing the casting myself:

    public static void main (String[] args) {
        Object test = new Test();
        cast(Test.class, test).new Inner() {
        };
    }
    public static  T cast (Class clazz, Object o) {
        return (T)o;
    }

Same verifier error. Interestingly, this variation works just fine:

    public static void main (String[] args) {
        Object test = new Test();
        Test.class.cast(test).new Inner();
    }

So it's the combination of using a universally quantified method with an anonymous inner class that bakes javac's noodle. Looking at the decompiled bytecode we can see where it goes astray. The non-anonymous example immediately above generates the following bytecode:

   ...
   12:  ldc_w   #2; //class Test
   15:  aload_1
   16:  invokevirtual   #5; //Method java/lang/Class.cast:(Ljava/lang/Object;)Ljava/lang/Object;
   19:  checkcast       #2; //class Test
   22:  dup
   23:  invokevirtual   #6; //Method java/lang/Object.getClass:()Ljava/lang/Class;
   26:  pop
   27:  invokespecial   #7; //Method Test$Inner."":(LTest;)V
   ...

And the equivalent example with the anonymous inner-class generates:

   ...
   12:  ldc_w   #2; //class Test
   15:  aload_1
   16:  invokevirtual   #5; //Method java/lang/Class.cast:(Ljava/lang/Object;)Ljava/lang/Object;
   19:  dup
   20:  invokevirtual   #6; //Method java/lang/Object.getClass:()Ljava/lang/Class;
   23:  pop
   24:  invokespecial   #7; //Method Test$1."":(LTest;)V
   ...

We have a case of a mysteriously disappearing checkcast. If I wasn't so deathly tired of mucking around with the internals of javac, I might try to figure out what the problem was and submit a patch along with the bug report. As it is, I'm just hoping that the highlight of my future Sunday evenings will be more exciting than finding bugs in javac.

Posted to code by mdb at 7:50 pm | Comments (0)       

January 24, 2010

“Duplicate field expected”

In my recent programming adventures, I separately encountered two perplexing error messages:

  • RuntimeException: Variable is used without definition!
  • IllegalArgumentException: Duplicate field expected

Inspection of the underlying code brought clarity:

  • throw new RuntimeException("Variable " + m + " used without definition!");
  • throw new IllegalArgumentException("Duplicate field " + fieldName);

The former was reporting a problem with a variable declared as InputStream is, the latter a problem with JUnit’s @Test annotation which defines a parameter java.lang.Class<? extends java.lang.Throwable> expected.

Tip from the pros: always wrap identifiers in quotes when including them in error messages.

Posted to code by mdb at 4:33 pm | Comments (1)       

November 4, 2009

Thank Gosling

The following is not legal Java:

    public static void main (String[] args) {
        int[] foo = new int[1];
        int[] bar = new int[1];
        ((args.length == 0) ? foo[0] : bar[0]) = 1;
    }

That makes my life easier.

Posted to code by mdb at 5:25 pm | Comments (2)       

October 9, 2009

GWT AsyncGen

The back-breaking straw landed on this camel earlier today when adding some methods to a GWT RemoteService interface and then remembering that, “Oh yeah, I have to go add those manually to the async mirror interface that goes with my service.”

First I poked around on the web. I found some sort of Maven plugin that in theory automatically generates these extra services, but I don’t use Maven and I’m not about to start now. Plus it had some bugs filed against it that made me think it was written by a barrel full of monkeys. There was some sort of IDE plugin, but I’m definitely not going to start using some random IDE. I also found some crazy old command line tool that harkened back to the @gwt.typeArgs days. No good.

So I did what I should have done a couple of years ago and wrote an ant task to DWIFN and to do it properly. So if you find yourself using GWT presently or in the future, run, don’t walk, to the following URL and spend the two minutes it will take to save yourself a bunch of annoying typing:

http://code.google.com/p/gwt-asyncgen/

Posted to code by mdb at 6:57 pm | Comments (0)       

September 20, 2009

Why am I surprised?

I go to set up my My Qwest account page for my glorious new phone service and when I click “Create account” after typing in a bunch of annoying crap, it gives me a page that says “Oh, sorry, our shit is down right now. Try again later.” (I’m paraphrasing.) I try again a few minutes later and it claims that my username already exits. Awesome.

My account must already be created, so I try logging in and get:

Sorry, there was an Error
Found in: /profile/pageFlows/profileValidation/ProfileValidationController.jpf

The source of this error is:
Developer: Please check tag attributes and closing tags. Be careful of reserved words

[View Error]  [Close]

com.bea.portlet.adapter.scopedcontent.ActionLookupFailedException: javax.servlet.ServletException
 at com.bea.portlet.adapter.scopedcontent.ScopedContentCommonSupport.executeAction(ScopedContentCommonSupport.java:699)
 at com.bea.portlet.adapter.scopedcontent.ScopedContentCommonSupport.renderInternal(ScopedContentCommonSupport.java:268)
 at com.bea.portlet.adapter.scopedcontent.PageFlowStubImpl.render(PageFlowStubImpl.java:136)
 at com.bea.netuix.servlets.controls.content.NetuiContent.preRender(NetuiContent.java:288)
 at com.bea.netuix.nf.ControlLifecycle$6.visit(ControlLifecycle.java:427)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:708)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:720)
 at com.bea.netuix.nf.ControlTreeWalker.walk(ControlTreeWalker.java:183)
 at com.bea.netuix.nf.Lifecycle.processLifecycles(Lifecycle.java:361)
 at com.bea.netuix.nf.Lifecycle.processLifecycles(Lifecycle.java:339)
 at com.bea.netuix.nf.Lifecycle.runOutbound(Lifecycle.java:186)
 at com.bea.netuix.nf.Lifecycle.run(Lifecycle.java:140)
 at com.bea.netuix.servlets.manager.UIServlet.runLifecycle(UIServlet.java:370)
 at com.bea.netuix.servlets.manager.UIServlet.processControlTree(UIServlet.java:256)
 at com.bea.netuix.servlets.manager.PortalServlet.service(PortalServlet.java:779)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:856)
 at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
 at weblogic.servlet.internal.StubSecurityHelper.invokeServlet(StubSecurityHelper.java:125)
 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:283)
 at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:26)
 at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
 at master.com.qwest.filters.PrimaryServletFilter.doFilter(PrimaryServletFilter.java:149)
 at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
 at master.com.qwest.filters.log.TeaLeafServletFilter.doFilter(TeaLeafServletFilter.java:78)
 at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
 at com.bea.p13n.servlets.PortalServletFilter.doFilter(PortalServletFilter.java:251)
 at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:42)
 at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3229)
 at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
 at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:121)
 at weblogic.servlet.internal.WebAppServletContext.securedExecute(WebAppServletContext.java:2002)
 at weblogic.servlet.internal.WebAppServletContext.execute(WebAppServletContext.java:1908)
 at weblogic.servlet.internal.ServletRequestImpl.run(ServletRequestImpl.java:1362)
 at weblogic.work.ExecuteThread.execute(ExecuteThread.java:209)
 at weblogic.work.ExecuteThread.run(ExecuteThread.java:181) Caused by: javax.servlet.ServletException
 at org.apache.beehive.netui.util.internal.ServletUtils.throwServletException(ServletUtils.java:164)
 at org.apache.beehive.netui.pageflow.PageFlowRequestProcessor.processInternal(PageFlowRequestProcessor.java:580)
 at org.apache.beehive.netui.pageflow.PageFlowRequestProcessor.process(PageFlowRequestProcessor.java:851)
 at org.apache.beehive.netui.pageflow.AutoRegisterActionServlet.process(AutoRegisterActionServlet.java:630)
 at org.apache.beehive.netui.pageflow.PageFlowActionServlet.process(PageFlowActionServlet.java:157)
 at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:414)
 at org.apache.beehive.netui.pageflow.PageFlowUtils.strutsLookup(PageFlowUtils.java:1169)
 at org.apache.beehive.netui.pageflow.PageFlowUtils.strutsLookup(PageFlowUtils.java:1200)
 at com.bea.portlet.adapter.scopedcontent.ScopedContentCommonSupport.executeAction(ScopedContentCommonSupport.java:688)
... 55 more
Caused by: java.lang.NullPointerException
 at com.qwest.commons.utilities.AccountInfoHelper.createAccountInfo(AccountInfoHelper.java:72)
 at com.qwest.commons.controls.ProfileLoginManagerImpl.getAccountInfo(ProfileLoginManagerImpl.java:254)
 at com.qwest.commons.controls.ProfileLoginManagerImpl.getNextFailure(ProfileLoginManagerImpl.java:372)
 at com.qwest.commons.controls.ProfileLoginManagerImpl.validateProfile(ProfileLoginManagerImpl.java:190)
 at com.qwest.commons.controls.ProfileLoginManagerBean.validateProfile(ProfileLoginManagerBean.java:164)
 at profile.pageFlows.profileValidation.ProfileValidationController.validateProfile(ProfileValidationController.java:225)
 at sun.reflect.GeneratedMethodAccessor1294.invoke(Unknown Source)
 at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:585)
 at org.apache.beehive.netui.pageflow.FlowController.invokeActionMethod(FlowController.java:869)
 at org.apache.beehive.netui.pageflow.PageFlowController.internalExecute(PageFlowController.java:305)
 at org.apache.beehive.netui.pageflow.internal.FlowControllerAction.execute(FlowControllerAction.java:51)
 at org.apache.beehive.netui.pageflow.PageFlowRequestProcessor$ActionRunner.execute(PageFlowRequestProcessor.java:2042)
 at org.apache.beehive.netui.pageflow.interceptor.action.internal.ActionInterceptors[snip](ActionInterceptors.java:57)
 at org.apache.beehive.netui.pageflow.PageFlowRequestProcessor.processActionPerform(PageFlowRequestProcessor.java:2114)
 at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:229)
 at org.apache.beehive.netui.pageflow.PageFlowRequestProcessor.processInternal(PageFlowRequestProcessor.java:554)
 at org.apache.beehive.netui.pageflow.PageFlowRequestProcessor.process(PageFlowRequestProcessor.java:851)
 at org.apache.beehive.netui.pageflow.PageFlowActionServlet.process(PageFlowActionServlet.java:157)
 at org.apache.beehive.netui.pageflow.PageFlowUtils.strutsLookup(PageFlowUtils.java:1169)
 at org.apache.beehive.netui.pageflow.PageFlowUtils.strutsLookup(PageFlowUtils.java:1200)
 at com.bea.portlet.adapter.scopedcontent.ScopedContentCommonSupport.executeAction(ScopedContentCommonSupport.java:688)
 at com.bea.netuix.servlets.controls.content.NetuiContent.preRender(NetuiContent.java:288)
 at com.bea.netuix.nf.ControlLifecycle$6.visit(ControlLifecycle.java:428)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:708)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.ControlTreeWalker.walkRecursivePreRender(ControlTreeWalker.java:717)
 at com.bea.netuix.nf.Lifecycle.run(Lifecycle.java:140)
 at com.bea.netuix.servlets.manager.PortalServlet.service(PortalServlet.java:780)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:857)
 at weblogic.servlet.internal.StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227)
 at weblogic.servlet.internal.ServletStubImpl.execute(ServletStubImpl.java:283)
 at weblogic.servlet.internal.TailFilter.doFilter(TailFilter.java:27)
 at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:43)
 at master.com.qwest.filters.PrimaryServletFilter.doFilter(PrimaryServletFilter.java:150)
 at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:43)
 at master.com.qwest.filters.log.TeaLeafServletFilter.doFilter(TeaLeafServletFilter.java:79)
 at weblogic.servlet.internal.FilterChainImpl.doFilter(FilterChainImpl.java:43)
 at com.bea.p13n.servlets.PortalServletFilter.doFilter(PortalServletFilter.java:252)
 at weblogic.servlet.internal.WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3229)
 at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
 ... 4 more

We have noted the error and will be working on the issue.

I’m sure it’s going to be fun to get my half-created account fixed. Joy.

I especially like the built-in warning to their obviously awesome developers: “Please check tag attributes and closing tags. Be careful of reserved words.”

Posted to code by mdb at 8:54 am | Comments (2)       

June 17, 2009

A technique for type-safe, concise actors in Java

Actors are great. They provide a concurrency model that is not too taxing to reason about and an organizing principle that tends to jive with how you would naturally structure your code. I recently put actors to use in a real project (implemented in Java), and in the process had to overcome some aesthetic difficulties with a technique that seemed worth sharing.

The Itch

The requirements I was trying to satisfy are as follows (note: I realize that most people use the term messages to describe what actors process but I prefer to say that actors process actions, so hopefully you can bear with my rogue nomenclature):

  1. I want the declaration of my actor’s public interface to be like declaring a set of public methods. The declaration should be concise and should be easily documented to communicate to the caller what each action does.
  2. I don’t want the act of posting an action to an actor to be dramatically syntactically more verbose than just calling a method on the actor.
  3. Conversely, I want posting an action to an actor to be visibly distinct in the code. The programmer needs to know that the operation will be performed asynchronously, so it should not look exactly like calling a normal method.
  4. When implementing my actors, I want to simply define methods that are invoked with actor semantics (these methods are only ever executed by a single thread at any particular time). I don’t want an explosion of anonymous inner classes.
  5. My actions may take arguments.
  6. I want as much help from the compiler as possilble.

I am using the Functional Java actor framework, which provides a QueueActor that implements the actor semantics I desire given an ExecutorService. Using the QueueActor is pretty straightforward:

class Action { /* some action class */ }
Strategy<Unit> strategy = Strategy.executorStrategy(someExecutor);
QueueActor<Action> actor = QueueActor.queueActor(strategy, new Effect<Action>() {
    public void e (Action action) {
        // handle your action, blissfully ignore concurrency
    }
});
actor.act(new Action(...));

However, you can imagine that using this building block to meet my requirements was going to require some effort. After trying and discarding a number of unsatisfactory approaches involving large quantities of boilerplate and/or a lack of type safety, I eventually arrived at a technique that avoids both. The key is to use generics and Java’s wee bit of type inference to our advantage.

The Scratch

To see how this all hangs together, let’s start by looking at an implementation of a basic actor using the technique. As you can see below, we have created an AbstractActor class and our actor implementations derive from that.

public class Thingy extends AbstractActor<Thingy>
{
    /** Instructs the thingy to initialize itself.
      * Param: File - the directory in which the thingy operates. */
    public static final Action1<Thingy, File> INIT =
        newAction(Thingy.class, "init", File.class);

    /** Instructs the thingy to shut itself down. */
    public static final Action<Thingy> SHUTDOWN =
        newAction(Thingy.class, "shutdown");

    public Thingy (Strategy<Unit> executor) {
        super(executor);
    }

    /** Handles the {@link #INIT} action. */
    protected void init (File dir) { ... }

    /** Handles the {@link #SHUTDOWN} action. */
    protected void shutdown () { ... }
}

To send this actor an action, we do the following:

thingy.act(Thingy.INIT, new File(somedir));
thingy.act(Thingy.SHUTDOWN);

Going back to my requirements, we can see that a few of them are already in good shape:

  1. We define the actor’s actions with fairly concise constant objects which also serve as a convenient place to provide documentation for the actions. The actions are essentially the public interface of the actor.
  2. Calling a method on an actor via the act() method is pretty concise. We do have to repeat the actor class name when supplying the action constant but that’s not a horrible burden. You could even use static imports to lessen that burden, but I don’t advocate such craziness.
  3. act() makes it pretty clear, without being obtrusive, that we’re sending an action to an actor and not just calling a normal method.
  4. Actions are implemented directly by protected (or private if you swing that way) methods in the actor class. There is no manual wiring and it’s pretty easy to see what’s going on in the actor code.
  5. It’s very easy to indicate that actions take arguments and to pass those arguments to a call to act(). No special classes are needed to wrap up the arguments to an action, and the arguments show up just where you want them as parameters on the method that implements the action.
  6. It’s not entirely obvious without seeing the code for AbstractActor, but all of this is completely type-safe. If you try to pass a Thingy action to a Whatsit actor’s act() method you get a compile error. If you try to pass a String to an action that takes a File, you get a compile error. If you neglect to pass an argument to an action that takes an argument you get a compile error.

The Gory Details

Let’s take a look behind the curtain and see how it works. First let’s look at what newAction() does:

public abstract class AbstractActor<T extends AbstractActor>
{
    // ...
    protected static <A extends AbstractActor> Action<A> newAction (Class<A> clazz, String name) {
        return new Action<A>(resolveMethod(clazz, name));
    }

    protected static <A extends AbstractActor, A1> Action1<A, A1> newAction (
        Class<A> clazz, String name, Class<A1> arg1) {
        return new Action1<A, A1>(resolveMethod(clazz, name, arg1));
    }
    // ...
}

Here’s where we start to get into some type gymnastics. newAction() returns an Action class (or subclass) parameterized by the types of the actor class and the arguments to that action. So when we call newAction(Thingy.class, “init”, File.class) above, we get back an Action1<Thingy, File> object. This is critical to propagating the type information to the act() method and enforcing the correct types and count for the action arguments. Let’s take a look at act() now:

public abstract class AbstractActor<T extends AbstractActor>
{
    // ...
    public void act (Action<T> action) {
        enact(action);
    }

    public <A1> void act (Action1<T, A1> action, A1 arg1) {
        enact(action, arg1);
    }
    // ...
}

There are a couple of things going on here. First, you can see that AbstractActor is parameterized in the type of itself, or rather of the type of the class that extends abstract actor. So we have Thingy extends AbstractActor<Thingy>. This then requires that all act() methods take Action objects that are typed Action<Thingy> and makes it a compile error to supply an Action typed for some other AbstractActor derived class.

To enforce the correct number of arguments, we simply have unique Action derivations for every arity. Action for zero-argument methods, Action1 for one-argument methods, and so on. In the examples I’ve only provided guts for zero and one argument actions, but you can trivially extend this to as many arguments as you desire.

Finally to enforce the type of the arguments, we use Java’s limited, but useful, generic type inference. By declaring our second act() method with type <A1>, which is inferred from the type of the Action1<T, A1> that is supplied when you call the method, we define A1 via the first parameter and the compiler requires that the subsequent “A1 arg” parameter match. So the type that was provided back in our newAction() call is properly preserved and enforced any time that Action is passed to an act() method.

Note that I’m glossing over argument covariance here for clarity. What you really want is:

    public <A1, B1 extends A1> void act (Action1<T, A1> action, B1 arg1) {

This allows you to declare a method that takes Number, for example, and pass an Integer to it. Java methods are covariant in their argument’s primary types, so we probably want actor actions to be the same.

Now let’s look at how we wire up and dispatch the action implementation methods internally. As you may suspect, we use reflection to do this. Normally one gives up compile time type checking when making use of reflection, but we’ve fortunately already handled that. One downside is that reflective method invocation is slower than a non-reflective method invocation. However, later I’ll show you how you can do away with the reflective dispatch for specific situations where you’ve discovered that performance is actually critical. This comes at the cost of some boilerplate which is why we prefer the reflective dispatch by default.

Now then, back to the implementation. You’ll notice that newAction() above calls a method “resolveMethod()” and passes its results into the Action constructor. This method is pretty simple. newAction() takes the name of the method and the types of its parameters, and Class provides a mechanism to look up a method given that exact information. We’d just call it directly except that unfortunately NoSuchMethodException is a checked exception.

public abstract class AbstractActor<T extends AbstractActor>
{
    // ...
    protected static Method resolveMethod (Class<?> clazz, String name, Class<?>... ptypes) {
        try {
            return clazz.getDeclaredMethod(name, ptypes);
        } catch (NoSuchMethodException nsme) {
            throw new IllegalArgumentException("Unable to resolve actor method: " + name, nsme);
        }
    }
    // ...
}

There’s another downside to using reflection. We specify the name of the reflected method to be called as a String passed to newAction(). What if you misspell that? What if you pass the argument classes in the wrong order? Since newAction() is called to initialize static fields, it will be called as soon as the Thingy class is resolved. This means that as soon as the Thingy class is referenced (a class that invokes actions on Thingy is resolved, or an instance of Thingy is created, etc.), the programmer finds out that they made a mistake in an action declaration. This is not delayed until the program actually sends the action in question to the actor. So while the class still has to be loaded, which could conceivably not happen immediately upon program startup, we’re still very likely to discover any problems as soon as possible during the development process.

The remainder is pretty simple. The Action class simply reflectively invokes the method when it is acted upon and AbstractActor’s enact() method simply queues up a Runnable to invoke the Action using appropriate actor semantics. I’ll include the entirety of AbstractActor here so that you can see how it all hangs together.

import java.lang.reflect.Method;

import fj.Effect;
import fj.Unit;
import fj.control.parallel.QueueActor;
import fj.control.parallel.Strategy;

public abstract class AbstractActor<T extends AbstractActor>
{
    public void act (Action<T> action) {
        enact(action);
    }

    public <A1> void act (Action1<T, A1> action, A1 arg1) {
        enact(action, arg1);
    }

    protected void enact (final Action<T> action, final Object... args) {
        _actor.act(new Runnable() {
            public void run () {
                @SuppressWarnings("unchecked") T self = (T)AbstractActor.this;
                action.invoke(self, args);
            }
        });
    }

    protected AbstractActor (Strategy<Unit> execor) {
        _actor = QueueActor.queueActor(execor, new Effect<Runnable>() {
            public void e (Runnable action) {
                action.run();
            }
        });
    }

    protected static <A extends AbstractActor> Action<A> newAction (Class<A> clazz, String name) {
        return new Action<A>(resolveMethod(clazz, name));
    }

    protected static <A extends AbstractActor, A1> Action1<A, A1> newAction (
        Class<A> clazz, String name, Class<A1> arg1) {
        return new Action1<A, A1>(resolveMethod(clazz, name, arg1));
    }

    protected static Method resolveMethod (Class<?> clazz, String name, Class<?>... ptypes) {
        try {
            return clazz.getDeclaredMethod(name, ptypes);
        } catch (NoSuchMethodException nsme) {
            throw new IllegalArgumentException("Unable to resolve actor method: " + name, nsme);
        }
    }

    protected static class Action<A extends AbstractActor> {
        public Action (Method method) {
            _method = method;
        }

        public void invoke (A actor, Object... args) {
            try {
                _method.invoke(actor, args);
            } catch (Exception e) {
                throw new RuntimeException(e);
            }
        }

        protected Method _method;
    }

    protected static class Action1<A extends AbstractActor, A1> extends Action<A> {
        public Action1 (Method method) {
            super(method);
        }
    }

    protected QueueActor<Runnable> _actor;
}

The Fine Print

I mentioned above that I’d indicate how to avoid the reflective call. You can probably figure this out yourself now that you’ve seen all the code, but just in case you’re conserving brain power for more important projects, I’ll show you an example of how to do that here:

public class Thingy extends AbstractActor<Thingy>
{
    public static final Action1<Thingy, String> ZIP = new Action1<Thingy, String>(null) {
        public void invoke (Thingy actor, Object... args) {
            actor.zip((String)args[0]);
        }
    };

    /** Implements the {@link #ZIP} action. */
    protected void zip (String zoom) { ... }
}

Note that you don’t have to worry about checking that args has one argument or that the argument is a String, because the compiler is ensuring that you can only call act(Thingy.ZIP, …) with a single string argument. It is worth noting that if you want primitive arguments you have to worry about both boxing and unboxing and that someone could do something stupid like: actor.act(Thingy.FOO, (Integer)null), but fortunately that brand of stupidity usually doesn’t happen accidentally.

You also might be wondering about actors sending responses to other actors and callers magically blocking until they receive their response. Functional Java’s actor library doesn’t support blocking on a response from an act() call, so that’s out. I don’t think it would be incompatible with this technique were it to be supported. In terms of sending responses at all: you could pretty easily pass a concrete actor to another actor and have that actor send an action in response. The trickier thing would be if you wanted to support sending a response to, say, any actor that supported the appropriate response action. Without having actually tried it, I think it would probably be possible using some clever definition of interfaces and more type gymnastics. I’ll leave that as an exercise for the reader or for myself and a future blog post.

Posted to code by mdb at 1:19 pm | Comments (0)       

December 1, 2008

Git out of my head

Last night I dreamt about GIT and implementing complex revision control policies atop its basic tools. In my dream I danced around the room as changes were fetched and pushed.

Posted to code by mdb at 8:30 am | Comments (0)       

November 5, 2006

Java Puzzlers (3)

An excellent compendium of misleading and non-intuitive constructions in the Java language. A small few are contrived, but most ring true with my ten years of experience with the language. I have discovered more than a few of them “the hard way”. Highly recommended for anyone who writes a lot of Java code.

Posted to books|code by mdb at 5:05 pm | Comments (1)       

August 6, 2006

Scala

Scala has piqued my interest more than any language to come down the pike in a long time. Type inference makes me cry out with joy, Java-compatibility, sensible mix-in support (made all the more useful by their very sophisticated type system), first-class functions, continuations (mostly), very SML-like type pattern matching, list comprehensions. One could get into some serious trouble with this language.

I was then even more excited when I saw a paper by the language designer describing how he was able to easily implement an elegant stackless threads system as a library using only the functionality provided by the language (and the small compromise of requiring a continuation passing style, but the elegance of the language construct being used makes that hardly noticable).

I wish I wasn’t mired in a bastard mashup of Java, JavaScript and ActionScript at the moment. I’ll have to wait a bit before giving Scala a thorough run around the block.

Posted to code by mdb at 7:15 pm | Comments (0)       

July 7, 2004

Random bitching

What kind of error message is this?

[11:20pm] baltic:~ > java -jar foo.jar
Exception in thread "main" java.util.zip.ZipException: No such file or directory
        at java.util.zip.ZipFile.open(Native Method)
        at java.util.zip.ZipFile.(ZipFile.java:112)
        at java.util.jar.JarFile.(JarFile.java:127)
        at java.util.jar.JarFile.(JarFile.java:65)

It doesn’t even tell me what file or directory is missing. You might think it’s obvious because I just ran the command, but frequently java is run from a shell script and all I see is this baffling crap about zip files.

I was about to suggest that it be more like the error message one gets when specifying a class that doesn’t exist:

[11:24pm] baltic:~ > java nonexistent.Class
Exception in thread "main" java.lang.NoClassDefFoundError: nonexistent/Class

However, that largely sucks as well. At least its not six calls deep into classloader code, which would be on par with the zip business, and it does tell me which class it couldn’t find, but it’s still a sorry excuse for an error message. Is it that difficult to catch the most common exceptions and report something sensible?

Posted to code by mdb at 11:31 pm | Comments (0)       

MDB FriendFeed Twitter Flickr Facebook
Me
Blog I/O Bits Ludography Archives:

©2001 - 2010 Michael Bayne <mdb@samskivert.com>