Java in Firefox extension hosed again

A full-featured browser, an XUL front end, and the wealth of libraries available in Java makes for a powerful and flexible combination. The browser extension capability of Firefox, along with LiveConnect has been used by at least three extensions:

• Simile's Piggy Bank, a semantic web browser
• XQuseme, embeds a XQuery engine into the browser
• Firegoose, exchanges data between bioinformatics web sites

Most of what I've figured out about using Java from an extension came from Simile's David Huynh. Sadly, development of Piggy Bank has now "quiesced".

I don't know about the others, but Firegoose is hosed by the latest Java 6 update 10. Apparently, Java 6.10 introduces some significant changes into LiveConnect and the Java browser plugin. It's certainly good that Java in the browser is getting some attention, but I wish Java in a Firefox extension was a supported and regression tested use case (see whining here). The fact that it's such an arcane, unsupported and brittle hack is holding back what could otherwise be a nice technique.

Interest in Java in Firefox extensions appears to exist according to these posts in the MozillaZine Extension Development forum:

• getting java.security.PrivilegedActionException from JAR
• Java Based Extension Development
• Java
• Using java in an Extension, how ?
• Calling JAVA method. Chrome URL

First Problem: The error I get appears to happen when reflectively instantiating a Java array and looks like this:

Error calling method on NPObject!
[plugin exception: java.lang.IllegalArgumentException: No method
found matching name newInstance and arguments
[com.sun.java.browser.plugin2.liveconnect.v1.JavaNameSpace,
java.lang.Integer]].

Instantiating the array through reflection was, itself, a work-around for another LiveConnect issue with type conversion between Javascript arrays and Java arrays. It's barfing on line 03 below:

// from http://simile.mit.edu/repository/java-firefox-extension/firefox/chrome/content/scripts/browser-overlay.js
_toJavaUrlArray: function(a) {
 var urlArray = java.lang.reflect.Array.newInstance(java.net.URL, a.length);
 for (var i = 0; i < a.length; i++) {
  var url = a[i];
  java.lang.reflect.Array.set(
   urlArray,
   i,
   (typeof url == "string") ? new java.net.URL(url) : url
  );
 }
 return urlArray;
}

Update 1: First problem solved easily enough.

var dummyUrl = new java.net.URL("http://gaggle.systemsbiology.org");
var urlArray = java.lang.reflect.Array.newInstance(dummyUrl.getClass(), a.length);

Now, on to more hideousness:

Error calling method on NPObject!
[plugin exception: java.security.AccessControlException: access denied
(java.lang.RuntimePermission createClassLoader)].

...caused by trying to instantiate a URLClassLoader. The next-generation Java Plug-in, including in update 10, makes changes to the security policy such that calls from Javascript to Java are uniformly treated as untrusted.

Update 2: A Work-around!

A post on the java.net forum has a work-around. You can disable the "next-generation plug-in" through the Java control panel. Under the Advanced tab, open Java Plug-in, deselect Enable the next-generation Java Plug-in, then, restart Firefox. There is a bug filed whose comments seem to suggest that it will be addressed in a future release of the Java Plug-in.

Update 3: According to this thread on java.net, a fix is on the way in Java SE 6 update 12. Thanks, Sun!

More references:

• Java Firefox Extension by ancaluca
• mozilla.dev.tech.java
• java security policy reference
• Mozilla on Java in a Firefox Extension
• Simile mailing lists

SQLiteJDBC from Xerial

As noted earlier, I haven't been able to get native mode JDBC drivers for SQLite to work on Java 6 on OS X. Xerial has made what appears to be a fork of the Zentus SQLiteJDBC driver. They even have a discussion group, which apparently Zentus used to have but doesn't anymore.

Maybe ch-werner's javasqlite is another possibility.

Clean code, properties, and patterns

Robert C. Martin has a new book out called Clean Code that looks worthwhile, as if I'm not far enough behind in reading. I also came across his Principles and Patterns article. Completely unrelated to that, The Universal Design Pattern by Steve Yegge describes object system of Javascript as an instance of what he calls the Properties Pattern. Java's static types and even classes in Python and Ruby feel constrained after working with Javascript's bag-of-properties approach to objects. There's also some interesting stuff at a blog called Red and Sensual Java, if you can get past the name.

Clay Shirky on Econ-Talk

Clay Shirky appeared on the excellent podcast Econ-Talk, to which I've become hopelessly addicted. (My geekosity knows few bounds.) He was hawking his book Here Comes Everybody.

Shirky is a great thinker about what makes the social aspects of technology tick. His current theory goes something like this: There is a cognitive surplus brought about the increasing amount of leisure time in modern industrialized societies. For a generation, that surplus has been dissipated through the artifice of television. Now, that surplus is being put to use. Open source software, tagging, Wikipedia, and other social web phenomena are the early results.

The question now is, what happens when someone figures out how to move this phenomena out to the physical world. Someone posted this quote in the comments section to the podcast:

The effect of a newspaper is not only to suggest the same purpose to a great number of persons, but to furnish means for executing in common the designs which they may have singly conceived.

-Alexis de Tocqueville

Notes on SqliteJDBC and Java Web Start

Java has a long standing limitation about loading native code from a jar, stemming from the unfortunate belief that native code was somehow impure. In the practical world, there was occasional need to do this, so people came up with the work-around of copying the native library out of the jar and into the local filesystem where it could then be loaded with System.load().

Another way do do this, within web start is to platform specific resource elements like these to your jnlp:

  
<resources os="Windows" arch="x86">
  <j2se href="http://java.sun.com/products/autodl/j2se" version="1.5+"/>
  <nativelib href="sqlitejdbc-v052-native-win.jar"/>
</resources>
<resources os="Mac OS">
  <j2se href="http://java.sun.com/products/autodl/j2se" version="1.5+"/>
  <nativelib href="sqlitejdbc-v052-native-mac.jar"/>
</resources>
<resources os="Linux">
  <j2se href="http://java.sun.com/products/autodl/j2se" version="1.5+"/>
  <nativelib href="sqlitejdbc-v052-native-lin.jar"/>
</resources>

This way, we get the native library for the right platform, if all goes according to plan. One wacky detail is that System.loadLibrary("foo") munges the library name in some platform specific and poorly documented way. So, you need a libfoo.jnilib for OS X, libfoo.so for linux, and a foo.dll for windoze. It's kind-of a hassle to set up all these separate jars. And System.load(...) just tries to do what it's told without munging the name, so people still frequently use the trick described above.

SqliteJDBC does just that to load the native Sqlite library. SqliteJDBC also includes "pure Java" drivers, which are considerably slower for some operations, but act as a nice fallback when the proper native libraries aren't handy.

This all works well enough. But, on OS X 10.5, Java 6, it ends up falling back on the non-native drivers. After digging a bit, I found that the call to System.load(..) was failing with this exception:

/private/tmp/libsqlitejdbc-19214.lib: 
java.lang.UnsatisfiedLinkError: /private/tmp/libsqlitejdbc-19214.lib: 
 at java.lang.ClassLoader$NativeLibrary.load(Native Method)
 at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1822)
 at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1702)
 at java.lang.Runtime.load0(Runtime.java:770)
 at java.lang.System.load(System.java:1005)
        ...

The same thing works find when using Java 5 on the same machine and on Java 6 on Windoze, so I'm guessing the native library needs to be recompiled to work correctly with OS X's 64-bit Java 6.

A little spelunking into SqliteJDBC's code reveals that the author has kindly provided hooks where you can insert the path and name of your own driver using these system properties:

• org.sqlite.lib.path
• org.sqlite.lib.name

The driver will then attempt this:

System.load(new File(libpath, libname).getAbsolutePath());

More Resources:

• Unofficial JNLP FAQ
• How to package a native library in a jar

WTF NCBI?

A previous post, Hacking NCBI Entrez, dealt with how to retrieve sequence information from NCBI's databases. That method seems to work for prokaryotes and for yeast, but fails for most other eukaryotes.

• NCBI's data formats (as ASN.1)

For mammals, efetch for genome XML gives back crap like this (for rat):

<gbseq_contig>join(NW_001084776.1:1..691014,gap(182895),NW_001084777.1:1..1914699,gap(182895),NW_001084778.1:1..26673,gap(182895),NW_001084779.1:1..2730,gap(182895),NW_001084780.1:1..61755,gap(182895),NW_001084781.1:1..20466,gap(182895),NW_001084782.1:1..657670,gap(182895),NW_001084783.1:1..55883,gap(182895),NW_001084784.1:1..9292,gap(182895),NW_001084785.1:1..10599,gap(182895),NW_001084786.1:1..14198,gap(182895),NW_001084787.1:1..3561,gap(182895),NW_001084788.1:1..106511,gap(182895),NW_001084789.1:1..21205827,gap(182895),NW_001084790.1:1..11152534,gap(182895),NW_001084791.1:1..6015389,gap(182895),NW_001084792.1:1..686425,gap(182895),NW_001084793.1:1..9344793)</gbseq_contig>

Or this for XML for human Y chromosome. Totally useless. I take it I'm supposed to request each of the referenced sequences and parse out the regions for each? What a colossal pain in the ass!

By the way, BioJava has a parser called GenbankXmlFormat, but it's docs say, "Deprecated. Use org.biojavax.bio.seq.io.INSDseqFormat". What INSDseqFormat is or how that is supposed to replace GenbankXML is totally unclear.

eScience

Ed Lazowska's elements of eScience:

• Sensors 
• Networking 
• Visualization 
• Databases 
• Data mining 
• Machine learning

eScience is what happens when scalable mass produced computing infrastructure is applied to scientific problems. It's not about the computation, it's about the data.

...which reminds me of a Peter Karp paper.