Sipdroid, gizmo, android, NAT fun

Few notes on my setup - in hope others will not have to waste so much time. I have a couple of VOIP devices behind a router without SIP support, and SIP is very sensitive to UDP ports.

Network looks like this:

1. pretty dumb DSL router ( not modem !) - I'm setting it for my parents, that's what they have. 
2. a linux host running siproxd. Next I'll try with an even smaller box ( an openWRT router ) - to save power. 
3. One Grandstream box connected to a regular phone
4. Android device, with SipDroid. 

The firewall is set to forward all UDP and TCP to the linux box - I added a 'custom' rule, I don't want to deal with the dsl router, all real filtering in linux. You can specify the SIP range - 5060 + whatever RTP. 

I'm using Gizmo - I like that it's integrated with google voice, relatively easy to set software for most OSes, etc. Well - not so easy to get sound to get gizmo to work on ubuntu with pulseaudio - lot of time wasted there too - I set gizmo to OSS and used "aoss gizmo", this seems to work for me. 

One big problem: siproxd can't deal with the 2 identities, i.e. the ascii username and the phone number. 

As a rule, I ignore all the gizmo usernames and use only phone numbers. That means user@proxy01.sipphone.com will not work, neither "call gizmo" in the UI. To dial you must use 1234@proxy01.sipphone.com, or "call SIP" in their UI.

On Grandstream - normal registration, with outbound proxy set, no STUN.

On Sipdroid - there is no direct support for proxy, but you can set the "Server" to the IP address of the linux box running siproxd, and than set Domain to proxy01.sipphone.com. Note that older versions of sipdroid don't have the option - just upgrade with latest from market. 

So far no problems with the setup - getting my parents to figure out how to dial and where to connect the wires is another story.

Handbrake 0.9.4 on Hardy

Handbrake is a very nice tool - 'just works' in most cases, has good presets so you don't need to know arcane compression options. They just released a new version - which drops support for .AVI and few other things. I think it's great to see this happen - it's not that common for projects to cut features instead of half-supporting them. On the other hand some devices only work with .avi - so I'll keep the old version around.

Another remarcable thing about Handbrake - they link statically, the binary has only very basic dependencies.

The official download is for Ubuntu 9.10 - I had to compile from source, the CLI version is easy to compile - for GUI you need webkit and few other things, so I'll wait for an official backport.

The only trick for compiling CLI is also compiling current yasm from source, the version in hardy is too old. Simple configure/make install works, just make sure you remove the package with "apt-get remove yasm" if you have the old version installed.

Required packages (besides libtool/autoconf/build-essentials): zlib1g-dev libbz2-dev intltool libglib2.0-dev libdbus-glib-1-dev libgtk2.0-dev libhal-dev libhal-storage-dev  libnotify-dev libgstreamer0.10-dev libgstreamer-plugins-base0.10-dev

Compiled binary, if you want to skip the compilation -
 "gzip -d   HandBrakeCLI-0.9.4.gz ; chmod +x HandBrakeCLI-0.9.4"

Trying Maven

I have never used maven in a project I'm working on - mostly because I disliked the rigidity and 'we know better   than you what you need and want'.

Today I wanted to make it a bit easier to build tomcat-lite - and decided to add a maven option. Tomcat-lite is far simpler than tomcat - it may work. I even accepted moving few files around - I couldn't get myself to create the ugly, deep hierarchy - but it seems you can specify 'sourceDirectory' and include resources from the java tree.

Results: I could build the connector and servlets just fine, downloading the deps was nicer than the custom ant tasks that tomcat uses. The main engine - which depends on some files from main tomcat -  it looks like no luck.

I found no way to specify exclude patterns on sources, and using a pre-generated jar file is a mess - you must import it in the repository, far from easy or automated. And all the syntax I had to use took a lot of effort to find and tweak - documentation is certainly not targeted to how to make maven work the way you need, but to make you work in whatever way is easier for maven.

Conclusion: Apache-Ivy does the download better - and has a nice task to copy the jars in a lib, and integrates quite nicely with ant. It is even easy to download Ivy and get it to download the rest. Far easier than Maven - and I just used the same pom.xml I was using for maven.

I'll let the pom files around, in case someone wants to use them - but I will continue to avoid maven and projects that only build with maven.

Fixing the annoying Java 2-line logging format

java.util.logging is far from perfect, but for me the main reason to not use it is the hard-to-read default formatter which shows the date on a whole row. There are more compact formatters - but you need to write code to install them, or edit the command line.

So every time I install a JVM, I change the logging formatter:

1. in $JAVA_HOME/jre/lib/logging.properties, replace

java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
with

java.util.logging.ConsoleHandler.formatter = org.apache.juli.JdkLoggerFormatter

2. copy TOMCAT_HOME/bin/tomcat-juli.jar to JAVA_HOME/jre/lib/ext.

Java flush() - not what you would expect

What I expect: flush() on the output stream to send the data. After flush() is completed, I expect the receiver to be able to read the data. Operating system and network may have their own delays - flush() is less than commit, so there is no guarantee that data is completely received, or it will ever be received when flush() is done, but it shouldn't be stuck in an intermediate java buffer with no way out until close().

OutputStream.flush() documentation seems to match: 'forces any output bytes to be written out' and clarifies that if any bytes have been buffered, they should be written to their "intended destination". And it clarifies that the OS may further buffer "the bytes are passed to the OS for writing", and not that "they are actually written".

Let's take few examples: SocketOutputStream and FileOutputStream do nothing in flush() because they don't buffer anything, so write() will pass the bytes to the OS.

BufferedOutputStream is the best example of what to do when buffering: it does send all the bytes to the next stream in the chain AND calls out.flush(), so the data will go to the 'intended destination' - the buffer is just an intermediary. Well known that if you want any decent performance you should use BufferedOutputStream. Not so well known - you can use better buffers, for example one that doesn't reallocate and copy the byte[], but instead keeps a list of buffers.

The biggest offender: DeflaterOutputStream. Flush will just call out.flush()., but keeps recent bytes in its buffer. Worse - the actual Deflater doesn't even support zlib flush.

For Servlets, flush() does the right thing - the bytes go all the way to the net. Unless you're using compression, and the serlvet engine uses java Deflater - in which case flush() will not push bytes to the net, they get stuck in deflater. GZIPOutputStream extends Deflater and has the same problem - tomcat's coyote GzipOutputFilter and the example compression filter won't actually flush to the net.

The alternative for compressed output that flush() - http://www.jcraft.com/jzlib/, BSD licence, pure java - has some good info about the flush() problem ). You can also use JNI and libz - Harmony's Deflater is a good starting point - but Harmony has same bug as Sun VM.

More on FutureTask

Adding to my previous FutureTask post: Future is a very nice interface, but the main problems with FutureTask are the lack of a callback and the clumsy constructor that requires a Callable. I guess their main use case was someone creating a FutureTask that performs a slow operation - as in my example, it is relatively easy to wrap the method in a Callable and then execute the FutureTask in a thread pool.

It is possible to add a callback by overriding FutureTask.done(), which is called after FutureTask.set().

There are few tricks to better use FutureTask. Let's say you have an operation that happens in background already, and you're notified with a callback. You want to wrap it in a Future to make it easier to use:

  interface Callback {
    void done(Object1);
  }

  void doSomethingAsync(Callback callWhenDone) {
  }

  class MyFuture extends FutureTask {
    public MyFuture() { 
      // FutureTask _requires_ a callable - even if it's not using it. Let's give him a dummy one.
     super(new Callable() { public Object1 call() { return null; }});
    }
    // set() is protected - need a visible method to let future know the result 
    public void gotTheResult(Object1 result) {
      this.set(result);
    } 
  }	

  Future<div class="youtube-video"><object1> getSomethingFuture() {
    final MyFuture res = new MyFuture();
    // We _don't_ submit it to an executor - so run() will not be called
    doSomethingAsync(new Calback() {void done(Object1 result) {
      // do what innerRun() does - i.e. call innerSet with the result. 
      // innerSet() will set state to RAN , even if the callable was never called.
      res.gotTheResult(result);});
    return res;
  } 

My synergy settings

I have multiple laptops and computers, and sometimes use them at the same time - it is convenient to use a single keyboard
and mouse. The problem is that the config is not fixed - and synergy is quite hard to reconfigure dynamically.

I start a synergys server on each computer whose keyboard I may use, and use a couple of scripts to create tunnels and
restart the clients.

The trick is to not use specific hostnames - but dummy ones, and explicitly set the "-c clientname" param. Then all real
config is done using ssh tunnels - with the extra benefit of

Server:
  synergys -n localhost -a 127.0.0.1:24800

  # optional: ssh to shared server ( if the keyboard host is behind another firewall ):
  ssh -f -N -R 24800:127.0.0.1:24800  public_computer_with_sshd

Server config: ~/.synergy.conf

section: screens<br />      localhost:<br />      up:<br />      down:<br />      left:<br />      right:<br />end<br /><br />section: links<br />      localhost:<br />        down = down<br />        up = up<br />        left = left<br />        right = right<br />     up:<br />        down = localhost<br />     down:<br />       up = localhost<br />     left:<br />       right = localhost<br />     right:<br />       left = localhost<br />end<br /><br /><br />

On client:
  
   ssh -N 24800:127.0.0.1:24800 server &
   echo $! > ~/.synergyssh.pid  

   killall -9 synergyc
   synergyc -n down localhost

I have few client scripts, pointing to different keyboard servers and layouts.

Intercepting SIGTERM

The 'proper' way to terminate a java process on unix is by sending SIGTERM, waiting a bit, and then sending SIGKILL. This gives java a chance to clean - using Runtime.getRuntime().getShutdownHook().

If you want to intercept this and do some early cleanup, before letting the others do theirs you need to use sun.misc.Signal - which can also be used to intercept other signals.

Things to do in a shutdownHook ? Flush the logs, finish important background processes and much more.

Code to intercept the signal:

oldSigTERM = Signal.handle(new Signal("TERM"),
                new SignalHandler() {
    public void handle(Signal signal) {
      System.err.println("Quit using SIGTERM hook");
      notificationHandler.prepareToQuit();
      if (oldSigTERM != null) {
        oldSigTERM.handle(signal);
      }
    }
  });

Common-sense on unit testing

The Cafes » Harold’s Corollary to Knuth’s Law