Skip to main content

HttpClient 4.x Timeout



HttpClient is one of the most versatile Java libraries. Unfortunately, it comes with a lot of configuration options that may be way too cryptic or difficult. While the API for 4.x series has been significantly improved there are still some sharp edges.

The deprecated or 3.x way of setting the timeout.

This is done using params. Note that this is still 4.x code but a deprecated one.

DefaultHttpClient httpClient = ...;
   
HttpParams httpParams = httpClient.getParams();
httpParams.setParameter(CoreConnectionPNames.CONNECTION_TIMEOUT, 1000L);
httpParams.setParameter(CoreConnectionPNames.SO_TIMEOUT, 1000L);
httpParams.setParameter(ClientPNames.CONN_MANAGER_TIMEOUT, 1000L);


Now the part httpClient.getParams() is deprecated since obviously this is a nasty API. You need to keep track of parameters with Enums/constants and their type as well.

The right 4.x way aka the builder way


HttpClient 4 is full of builder for everything. While I often love the Builder patterns in some cases the testing part is really difficult but I guess it is always a tradeoff. This would be the "right way".


public RequestConfig requestConfigWithTimeout(int timeoutInMilliseconds) {
        return RequestConfig.copy(RequestConfig.DEFAULT)
                .setSocketTimeout(timeoutInMilliseconds)
                .setConnectTimeout(timeoutInMilliseconds)
                .setConnectionRequestTimeout(timeoutInMilliseconds)
                .build();
}

The meaning of the parameters

Notice that we are actually setting 3 different timeouts.
  • Socket Timeout - this is the time of inactivity to wait for packets to arrive
  • Connection Timeout - the time to establish a connection with the remote host
  • Connection Manager Timeout - the time to fetch a connection from the connection pool
The third one costed me dearly, it wasn't until the client was under a high load that issues started happening. The connection pool I was using had 10 connections per route limitation set and the scenario was quite common. As you can see these settings do not provide a mechanism for making an N millisecond hard timeout.

If we were to setup the timeout to 10 seconds on each of this we could end up with a request that lasts 9(sec to get a connection) + 9( sec to open connection ) + 9( sec of inactivity ) = 27 sec.

More info

A bug when setting the ConnectionRequestTimeout
The RequestConfig Javadoc
Apache HttpClient examples

Popular posts from this blog

HTML 5 data-* attributes, how to use them and why

It is always tempting to add custom attributes in HTML so that you can use the data stored there to do X. But if you do that there is no way of knowing if your HTML attribute will not be overridden in the future and used for something else and additionally you will not be writing valid HTML markup that can pass HTML 5 validator and with that you can create some very bad side effects. That is why there is a spec in HTML 5 called custom data attributes that enable number of useful features.

You may go around and read the specs, but the basic idea is very simple, you can add any attribute that starts with "data-" and that attribute will be treated as non-visible data for that attribute. By non-visible I mean that it is not something that gets rendered to the client so it does not affect the layout or style of the page, but it is there in the HTML so in no way this is private.
So let's get right into it, the following snippet is a valid HTML5 markup

<div id="aweso…

Temporary files and directories in Java 7 and before

Sometimes we want to create a temporary file, whether to save some data that gets written by some other application or just to temporary store stuff. Well, usually applications have their own temporary folder where they do this and it gets somehow configured. But why not use the underlying OS specific file like "/tmp/" in Linux so there must be some system property that has this info and there is. The key is "java.io.tmpdir" resulting in "/tmp" in my case or by code:
String tempDir = System.getProperty("java.io.tmpdir"); We can use tempDir  folder as a temporary place to store files, but there are a lot nicer ways to work with files like this even in JDK6 not just in JDK7:
import java.io.File; import java.io.IOException; import java.nio.file.Files; import java.nio.file.Path; public class TempFile { public static void main(String[] args) { try { // create a temp file File tempFile = File.createTempFile("old-file",…

Basic Authentication with RestTemplate

Spring Rest Templates are very good way of writing REST clients. By default they work with basic HTTP so if we need to use Basic Authorization we would need to init the rest template with custom HttpClient. This way the Rest Template will automatically use Basic Auth and append to the HTTP headers "Authorization: Basic BASE64ENCODED_USER_PASS".

HttpClient client = new HttpClient(); UsernamePasswordCredentials credentials = new UsernamePasswordCredentials("USERNAME","PASS"); client.getState().setCredentials( new AuthScope("www.example.com", 9090, AuthScope.ANY_REALM), credentials); CommonsClientHttpRequestFactory commons = new CommonsClientHttpRequestFactory(client); RestTemplate template = new RestTemplate(commons); SomeObject result = template.getForObject( "http://www.example.com:9090/",SomeObject.class );

In EE application this would probably be managed by DI framework like Spring Core and only initialized once sin…