Chrome, SSL and Tomcat - or other JSSE-based engines

If you try to use Chromium with a java server running https - be aware of  this bug:
http://bugs.sun.com/view_bug.do?bug_id=6728126

Chrome will use a nice TLS extension - RFC4507, SessionTicket - to allow servers to reuse the session without a second roundtrip and without having to cache the session ( which is quite tricky if you use load balancing - there is no easy way in java to do so ). Of course - JSSE doesn't support the extension, the bug is that it can't even ignore it. Besides saving the roundtrip this also saves some CPU time on both ends - good thing chrome added this.

I spent some time today trying to figure out what was happening - the good news seems to be that it's fixed in recent jdk builds. Unfortunately ubuntu doesn't have the fix yet (not even Karmic), and probably Mac is also behind.

Filed a chrome bug as well - I think reversing the order of SessionTicket and hostname extension would solve this.

The server name extension is another nice thing - it allows chrome to specify which host it's looking for in the first ClientHelo packet - so server can send the right certificate. The "Host" header will be sent after the encryption is started - this is why many https servers require one IP address per host and can't do virtual hosting with SSL.

Sipdroid, gizmo, android, NAT fun

Few notes on my setup - in hope others will not have to waste so much time. I have a couple of VOIP devices behind a router without SIP support, and SIP is very sensitive to UDP ports.

Network looks like this:

1. pretty dumb DSL router ( not modem !) - I'm setting it for my parents, that's what they have. 
2. a linux host running siproxd. Next I'll try with an even smaller box ( an openWRT router ) - to save power. 
3. One Grandstream box connected to a regular phone
4. Android device, with SipDroid. 

The firewall is set to forward all UDP and TCP to the linux box - I added a 'custom' rule, I don't want to deal with the dsl router, all real filtering in linux. You can specify the SIP range - 5060 + whatever RTP. 

I'm using Gizmo - I like that it's integrated with google voice, relatively easy to set software for most OSes, etc. Well - not so easy to get sound to get gizmo to work on ubuntu with pulseaudio - lot of time wasted there too - I set gizmo to OSS and used "aoss gizmo", this seems to work for me. 

One big problem: siproxd can't deal with the 2 identities, i.e. the ascii username and the phone number. 

As a rule, I ignore all the gizmo usernames and use only phone numbers. That means user@proxy01.sipphone.com will not work, neither "call gizmo" in the UI. To dial you must use 1234@proxy01.sipphone.com, or "call SIP" in their UI.

On Grandstream - normal registration, with outbound proxy set, no STUN.

On Sipdroid - there is no direct support for proxy, but you can set the "Server" to the IP address of the linux box running siproxd, and than set Domain to proxy01.sipphone.com. Note that older versions of sipdroid don't have the option - just upgrade with latest from market. 

So far no problems with the setup - getting my parents to figure out how to dial and where to connect the wires is another story.

Handbrake 0.9.4 on Hardy

Handbrake is a very nice tool - 'just works' in most cases, has good presets so you don't need to know arcane compression options. They just released a new version - which drops support for .AVI and few other things. I think it's great to see this happen - it's not that common for projects to cut features instead of half-supporting them. On the other hand some devices only work with .avi - so I'll keep the old version around.

Another remarcable thing about Handbrake - they link statically, the binary has only very basic dependencies.

The official download is for Ubuntu 9.10 - I had to compile from source, the CLI version is easy to compile - for GUI you need webkit and few other things, so I'll wait for an official backport.

The only trick for compiling CLI is also compiling current yasm from source, the version in hardy is too old. Simple configure/make install works, just make sure you remove the package with "apt-get remove yasm" if you have the old version installed.

Required packages (besides libtool/autoconf/build-essentials): zlib1g-dev libbz2-dev intltool libglib2.0-dev libdbus-glib-1-dev libgtk2.0-dev libhal-dev libhal-storage-dev  libnotify-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev

Compiled binary, if you want to skip the compilation -
 "gzip -d   HandBrakeCLI-0.9.4.gz ; chmod +x HandBrakeCLI-0.9.4"

Trying Maven

I have never used maven in a project I'm working on - mostly because I disliked the rigidity and 'we know better   than you what you need and want'.

Today I wanted to make it a bit easier to build tomcat-lite - and decided to add a maven option. Tomcat-lite is far simpler than tomcat - it may work. I even accepted moving few files around - I couldn't get myself to create the ugly, deep hierarchy - but it seems you can specify 'sourceDirectory' and include resources from the java tree.

Results: I could build the connector and servlets just fine, downloading the deps was nicer than the custom ant tasks that tomcat uses. The main engine - which depends on some files from main tomcat -  it looks like no luck.

I found no way to specify exclude patterns on sources, and using a pre-generated jar file is a mess - you must import it in the repository, far from easy or automated. And all the syntax I had to use took a lot of effort to find and tweak - documentation is certainly not targeted to how to make maven work the way you need, but to make you work in whatever way is easier for maven.

Conclusion: Apache-Ivy does the download better - and has a nice task to copy the jars in a lib, and integrates quite nicely with ant. It is even easy to download Ivy and get it to download the rest. Far easier than Maven - and I just used the same pom.xml I was using for maven.

I'll let the pom files around, in case someone wants to use them - but I will continue to avoid maven and projects that only build with maven.

Fixing the annoying Java 2-line logging format

java.util.logging is far from perfect, but for me the main reason to not use it is the hard-to-read default formatter which shows the date on a whole row. There are more compact formatters - but you need to write code to install them, or edit the command line.

So every time I install a JVM, I change the logging formatter:

1. in $JAVA_HOME/jre/lib/logging.properties, replace

java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
with

java.util.logging.ConsoleHandler.formatter = org.apache.juli.JdkLoggerFormatter

2. copy TOMCAT_HOME/bin/tomcat-juli.jar to JAVA_HOME/jre/lib/ext.

Java flush() - not what you would expect

What I expect: flush() on the output stream to send the data. After flush() is completed, I expect the receiver to be able to read the data. Operating system and network may have their own delays - flush() is less than commit, so there is no guarantee that data is completely received, or it will ever be received when flush() is done, but it shouldn't be stuck in an intermediate java buffer with no way out until close().

OutputStream.flush() documentation seems to match: 'forces any output bytes to be written out' and clarifies that if any bytes have been buffered, they should be written to their "intended destination". And it clarifies that the OS may further buffer "the bytes are passed to the OS for writing", and not that "they are actually written".

Let's take few examples: SocketOutputStream and FileOutputStream do nothing in flush() because they don't buffer anything, so write() will pass the bytes to the OS.

BufferedOutputStream is the best example of what to do when buffering: it does send all the bytes to the next stream in the chain AND calls out.flush(), so the data will go to the 'intended destination' - the buffer is just an intermediary. Well known that if you want any decent performance you should use BufferedOutputStream. Not so well known - you can use better buffers, for example one that doesn't reallocate and copy the byte[], but instead keeps a list of buffers.

The biggest offender: DeflaterOutputStream. Flush will just call out.flush()., but keeps recent bytes in its buffer. Worse - the actual Deflater doesn't even support zlib flush.

For Servlets, flush() does the right thing - the bytes go all the way to the net. Unless you're using compression, and the serlvet engine uses java Deflater - in which case flush() will not push bytes to the net, they get stuck in deflater. GZIPOutputStream extends Deflater and has the same problem - tomcat's coyote GzipOutputFilter and the example compression filter won't actually flush to the net.

The alternative for compressed output that flush() - http://www.jcraft.com/jzlib/, BSD licence, pure java - has some good info about the flush() problem ). You can also use JNI and libz - Harmony's Deflater is a good starting point - but Harmony has same bug as Sun VM.

More on FutureTask

Adding to my previous FutureTask post: Future is a very nice interface, but the main problems with FutureTask are the lack of a callback and the clumsy constructor that requires a Callable. I guess their main use case was someone creating a FutureTask that performs a slow operation - as in my example, it is relatively easy to wrap the method in a Callable and then execute the FutureTask in a thread pool.

It is possible to add a callback by overriding FutureTask.done(), which is called after FutureTask.set().

There are few tricks to better use FutureTask. Let's say you have an operation that happens in background already, and you're notified with a callback. You want to wrap it in a Future to make it easier to use:

  interface Callback {
    void done(Object1);
  }

  void doSomethingAsync(Callback callWhenDone) {
  }

  class MyFuture extends FutureTask {
    public MyFuture() { 
      // FutureTask _requires_ a callable - even if it's not using it. Let's give him a dummy one.
     super(new Callable() { public Object1 call() { return null; }});
    }
    // set() is protected - need a visible method to let future know the result 
    public void gotTheResult(Object1 result) {
      this.set(result);
    } 
  }	

  Future<div class="youtube-video"><object1> getSomethingFuture() {
    final MyFuture res = new MyFuture();
    // We _don't_ submit it to an executor - so run() will not be called
    doSomethingAsync(new Calback() {void done(Object1 result) {
      // do what innerRun() does - i.e. call innerSet with the result. 
      // innerSet() will set state to RAN , even if the callable was never called.
      res.gotTheResult(result);});
    return res;
  }