Monday, January 11, 2010

ZLib memory overhead

I was testing some compression code - and the load tests were running out of memory. Looks like ZLib allocates about  256K per compressor. I tried this with both  BEST_COMPRESSION and BEST_SPEED - looking at Shallow Heap in eclipse heap analyzer, 2x32k short[] and 2x64k byte[].

With SPDY you need to keep the header compressor around for the entire kept-alive connection. HTTP connections only use gzip for individual requests - the context and memory can be reclaimed, so a kept-alive connection has a very small cost - can be as low as just a socket in a selector, sometimes few extra buffers.In tomcat-lite there is one 8k buffer associated with the connection - not hard to get rid of it, but low priority.

No comments: