I have an ongoing interest in getting functional languages to be happy fun times on mobile devices, the first target of which is android and clojure. I mentioned there that there are some ideas around getting the DEX stage faster, and now I’ve spent some time chasing them up.

The problem:

The DEX stage takes somewhat over one minute for the clojure runtime. The current arrangement of clojure stuffs takes about 90 seconds to build up and install a complete artifact including that; if we don’t dex the clojure runtime, then the time to build up and install drops down to more like 10 seconds.

DEX wha?

Android is java, almost. The VM on the device is a Dalvik VM, not a Java VM, and it understands Dalvik bytecode. I believe this is a strategy for doing space-efficient VMs without treading on Oracle’s (nee Sun) patent toes, but whatever the motivation, you do need to send Dalvik bytecode to the device. The DEX stage takes Java bytecode as input and produces Dalvik bytecode.

There’s an obvious solution, right?

Yep! instead of converting the clojure runtime to dalvik every time we build, let’s just convert it once, then include the converted stuff every time we build up a package.

So what’s in that github repo linked by the mailing list thread?

It makes Clojure android-aware. If Clojure is running on android, then with the patches:

• the writeClassFile method will also dex the resulting class file and pack it into a .apk
• when it needs to look up a class file for an ObjExpr, it will do so by loading up the relevant .apk file with the DexClassLoader
• when trying to load a file, it looks – if on android – for a .apk file and throws it to the DexClassLoader

So you just use that, right?

Um, no. It checks if it’s running on android, and acts differently if so. I want to compile stuff up on the desktop and send it down to the device.

So you just use some part of that, right?

Well, in theory. I hacked it up so that on the desktop it will dex up all of the android classes, and included them into my app along with clojure’s RT.class so that the magical loading function is available to get clojure libs out of dex files. Then I used the same compiler to build up my app. I haven’t been able to get it to work, though – it turns out that getting the magical loading function to work is rather tricky.

I thought that sounded like overkill, though: it should be enough to build up clojure, dex clojure.jar, shove the resulting .dex file into a .apk, and stick that .apk file into your applications’ .apk. Then all you need to do is stick a class loader in so that the VM loads up your clojure apk before it tries to load up your activities and such. Turns out that doesn’t work either, because the VM will use the ClassLoader that it used to load your class, to load any classes that your class references. My activity class is defined in my applications’ apk, but being written in clojure needs some stuff from the clojure apk. By the time android has started up my activity, it’s too late for me to hook the classloaders up the way I want.

So what do you do, then?

There are three options I am still exploring, one of which is suggested by MHOO’s approach. Clojure has an internal hook for loading the .class file corresponding to a method, and it’s possible to hack that hook up so that it looks to a .apk instead (this is the third point above). As long as everything in the Activity’s class goes through that hook to get its clojure machinery, that should work.

But that’s clojure-specific..

There are two more general strategies I’m considering.

1. Translate the language’s .jar to .dex. Keep that file around. Instead of re-dexing it each time, augment the language’s jar with the application’s dexed stuffity stuff. I don’t know enough about the dex format to say if this will actually be any faster than dexing the language and application every time.
2. Compile your language to a jar. Dex that jar. Compile your application to a jar. Dex that jar. Then have a handful of bootstrap classes at the top level which load the language .apk, use that classloader as the parent classloader to load your application .apk, and implements proxies to all of the Activities, Services &c in your application.

And there’s a third option: live with the slow build time.

Hopefully I will – at some point – follow this up with a post containing an actual solution :)

• Uncategorized