JCP is dead - long live Java

Finally - JCP is dead. Not because of the bad APIs it produced to help Sun/Oracle sell products, but because of licencing restrictions and broken promises. But at least it's dead, and Java Language can finally move forward and be open.

And the way forward is really open APIs, with implementations that support not only in latest J2SE, but also GCJ, Mono and other VMs that support Java programming language. Without 'rewrite everything in pure Java' - but playing nice with other languages and VMs.

My intention is to pick the one or 2 of the Java APIs  I had the most pain with - JSSE and NIO - and try to improve tomcat-native and APR bindings to the point they are a better APIs. They are already faster and provide far more features - but need more docs and friendlier APIs.

Hopefully people will pick the other Java API that went beyond complexity craziness - it's time for an alternative to the Servlet API which supports the new reality of the web and is not just an extension of CGIs with J2EE mega-complexity added in.

Now is the time to promote the other APIs that were shadowed by the JCP false claims of 'open' and 'standard', time for people to re-learn about better alternatives to java.sql, java logging, swing, RMI or URLConnection.

SSL optimizations

ImperialViolet - Overclocking SSL

Strophe client and Openfire XMPP server

It took me 2 hours: I was trying to play a bit with Javascript, long-lived connections/BOSH and xmpp.

I installed Openfire - a java XMPP server, it includes a BOSH servlet, all is very easy to setup. Build from source is also easy and the result works with now extra work. Used the internal database, created user, tested with an XMPP client.

Next I wanted to test from browser - got strophe, changed the examples. The easiest way to set it up is to install it in openfire/resources/spank. The 'bosh' servlet on port 7070 runs with the embedded Jetty, and will serve anything in that dir. This avoids the cross-domain problems. In a prod env you'll probably mix resources from different sources on port 80 - there are some good instructions on how to set apache or nginx, or how to use flash and crossdomain.xml if you really need the bosh server to be on a different domain.

The main trick - the resource name is not /xmpp-bind, not /http-bind as some docs on the web suggest, but /http-bind/. Without the '/' it gets a redirect, and after redirect a GET is done instead of a POST.

Access gvfs mounts from the command line

Access gvfs mounts from the command line

Finding an entity in appengine console

Appengine has a nice "datastore viewer" - you can browse your data, but if you want to find a specific row you need to run a query.  The syntax is not obvious from the documentation if you're selecting by primary key:

SELECT * FROM DeviceInfo WHERE __key__ = Key('DeviceInfo', 'my_key')

I have a DeviceInfo table, with a primary key called 'name'. You can find about __key__ and Key in the docs, but I had to try many combinations to find that the first param is actually the type and the key name doesn't matter.

Of course, this doesn't work from java JDO - which is smart and uses the actual key name. And it doesn't use AND but &&. But it has as useless error messages as GQL, so at least there is some similarity.

Why use the primary key ? According to the docs and some presentations, the actual data is stored in the 'primary' entity table, using the primary key. All other indexes involve 2 lookups ( sorf of a JOIN ). And some use cases - like simple store by key - fit better in the 'bigtable' model than in the rich SQL.

Server to Android messages

Also check: http://code.google.com/p/chrometophone

Froyo required.

Backup windows from linux

ntfsclone - to copy/backup to, included on ubuntu live/install CD. Tar doesn't seem to work very well.

This is for moving windows to a larger disk, or making a snapshot to restore later.

I got a new netbook, removed all the garbage, installed my stuff ( chrome, firefox, etc ), saved the 'restore point' and installed ubuntu. I only use windows for netflix and occasional testing - so I don't want to deal with any windows-specific backup program or to have to reinstall.

ZLib memory overhead

I was testing some compression code - and the load tests were running out of memory. Looks like ZLib allocates about  256K per compressor. I tried this with both  BEST_COMPRESSION and BEST_SPEED - looking at Shallow Heap in eclipse heap analyzer, 2x32k short[] and 2x64k byte[].

With SPDY you need to keep the header compressor around for the entire kept-alive connection. HTTP connections only use gzip for individual requests - the context and memory can be reclaimed, so a kept-alive connection has a very small cost - can be as low as just a socket in a selector, sometimes few extra buffers.In tomcat-lite there is one 8k buffer associated with the connection - not hard to get rid of it, but low priority.

Heap dumps - looking at all objects and fields in a live system, without a debugger

Do you have a java server - maybe in production - and you want to look at the value of some field ? You can't attach a debugger, this would require starting the server with various flags - but you can get a heap dump, which includes all objects, including the value of their fields. 

The other use is to generate a heap dump before and after running a load test - and look at garbage to evaluate how much work you put on the garbage collector and find leaks. There are profilers that can get better results - but this is pretty fast and free way to get the same result, and you can use it against running systems. 

Getting heap dumps:

 jmap -dump:format=b,file=heap3.bin PID_OF_JAVA_PROCESS

Using code ( in a servlet, etc ):

server = ManagementFactory.getPlatformMBeanServer();
server.invoke(new ObjectName("com.sun.management:type=HotSpotDiagnostic"),
              "dumpHeap",
              new Object[] {fileName, Boolean.TRUE}, 
              new String[] {String.class.getName(), "boolean"});

Looking at the data:

  Eclipse MemoryAnalyzer 

or 

  jhat heap3.bin 

  open http://localhost:7000

Flush() in a compressed stream

I finally understood the difference between 'sync' and partial flush: http://www.bolet.org/~pornin/deflate-flush.html

Since last byte may not end on a 8-bit boundary - you need to pad, i.e. to end the current compression block and start a new one.  The interesting part is that a flush with Z_SYNC_FLUSH inserts 0x00 0x00 0xFF 0xFF - but protocols like PPP can strip it.

Another good link: http://www.zlib.net/zlib_tech.html