By BalusC


2010-05-08 06:16:11 8 Comments

Use of java.net.URLConnection is asked about pretty often here, and the Oracle tutorial is too concise about it.

That tutorial basically only shows how to fire a GET request and read the response. It doesn't explain anywhere how to use it to among others perform a POST request, set request headers, read response headers, deal with cookies, submit a HTML form, upload a file, etc.

So, how can I use java.net.URLConnection to fire and handle "advanced" HTTP requests?

11 comments

@BalusC 2010-05-08 06:16:37

First a disclaimer beforehand: the posted code snippets are all basic examples. You'll need to handle trivial IOExceptions and RuntimeExceptions like NullPointerException, ArrayIndexOutOfBoundsException and consorts yourself.


Preparing

We first need to know at least the URL and the charset. The parameters are optional and depend on the functional requirements.

String url = "http://example.com";
String charset = "UTF-8";  // Or in Java 7 and later, use the constant: java.nio.charset.StandardCharsets.UTF_8.name()
String param1 = "value1";
String param2 = "value2";
// ...

String query = String.format("param1=%s&param2=%s", 
     URLEncoder.encode(param1, charset), 
     URLEncoder.encode(param2, charset));

The query parameters must be in name=value format and be concatenated by &. You would normally also URL-encode the query parameters with the specified charset using URLEncoder#encode().

The String#format() is just for convenience. I prefer it when I would need the String concatenation operator + more than twice.


Firing a HTTP GET request with (optionally) query parameters

It's a trivial task. It's the default request method.

URLConnection connection = new URL(url + "?" + query).openConnection();
connection.setRequestProperty("Accept-Charset", charset);
InputStream response = connection.getInputStream();
// ...

Any query string should be concatenated to the URL using ?. The Accept-Charset header may hint the server what encoding the parameters are in. If you don't send any query string, then you can leave the Accept-Charset header away. If you don't need to set any headers, then you can even use the URL#openStream() shortcut method.

InputStream response = new URL(url).openStream();
// ...

Either way, if the other side is a HttpServlet, then its doGet() method will be called and the parameters will be available by HttpServletRequest#getParameter().

For testing purposes, you can print the response body to stdout as below:

try (Scanner scanner = new Scanner(response)) {
    String responseBody = scanner.useDelimiter("\\A").next();
    System.out.println(responseBody);
}

Firing a HTTP POST request with query parameters

Setting the URLConnection#setDoOutput() to true implicitly sets the request method to POST. The standard HTTP POST as web forms do is of type application/x-www-form-urlencoded wherein the query string is written to the request body.

URLConnection connection = new URL(url).openConnection();
connection.setDoOutput(true); // Triggers POST.
connection.setRequestProperty("Accept-Charset", charset);
connection.setRequestProperty("Content-Type", "application/x-www-form-urlencoded;charset=" + charset);

try (OutputStream output = connection.getOutputStream()) {
    output.write(query.getBytes(charset));
}

InputStream response = connection.getInputStream();
// ...

Note: whenever you'd like to submit a HTML form programmatically, don't forget to take the name=value pairs of any <input type="hidden"> elements into the query string and of course also the name=value pair of the <input type="submit"> element which you'd like to "press" programmatically (because that's usually been used in the server side to distinguish if a button was pressed and if so, which one).

You can also cast the obtained URLConnection to HttpURLConnection and use its HttpURLConnection#setRequestMethod() instead. But if you're trying to use the connection for output you still need to set URLConnection#setDoOutput() to true.

HttpURLConnection httpConnection = (HttpURLConnection) new URL(url).openConnection();
httpConnection.setRequestMethod("POST");
// ...

Either way, if the other side is a HttpServlet, then its doPost() method will be called and the parameters will be available by HttpServletRequest#getParameter().


Actually firing the HTTP request

You can fire the HTTP request explicitly with URLConnection#connect(), but the request will automatically be fired on demand when you want to get any information about the HTTP response, such as the response body using URLConnection#getInputStream() and so on. The above examples does exactly that, so the connect() call is in fact superfluous.


Gathering HTTP response information

  1. HTTP response status:

    You need a HttpURLConnection here. Cast it first if necessary.

    int status = httpConnection.getResponseCode();
    
  2. HTTP response headers:

    for (Entry<String, List<String>> header : connection.getHeaderFields().entrySet()) {
        System.out.println(header.getKey() + "=" + header.getValue());
    }
    
  3. HTTP response encoding:

    When the Content-Type contains a charset parameter, then the response body is likely text based and we'd like to process the response body with the server-side specified character encoding then.

    String contentType = connection.getHeaderField("Content-Type");
    String charset = null;
    
    for (String param : contentType.replace(" ", "").split(";")) {
        if (param.startsWith("charset=")) {
            charset = param.split("=", 2)[1];
            break;
        }
    }
    
    if (charset != null) {
        try (BufferedReader reader = new BufferedReader(new InputStreamReader(response, charset))) {
            for (String line; (line = reader.readLine()) != null;) {
                // ... System.out.println(line) ?
            }
        }
    } else {
        // It's likely binary content, use InputStream/OutputStream.
    }
    

Maintaining the session

The server side session is usually backed by a cookie. Some web forms require that you're logged in and/or are tracked by a session. You can use the CookieHandler API to maintain cookies. You need to prepare a CookieManager with a CookiePolicy of ACCEPT_ALL before sending all HTTP requests.

// First set the default cookie manager.
CookieHandler.setDefault(new CookieManager(null, CookiePolicy.ACCEPT_ALL));

// All the following subsequent URLConnections will use the same cookie manager.
URLConnection connection = new URL(url).openConnection();
// ...

connection = new URL(url).openConnection();
// ...

connection = new URL(url).openConnection();
// ...

Note that this is known to not always work properly in all circumstances. If it fails for you, then best is to manually gather and set the cookie headers. You basically need to grab all Set-Cookie headers from the response of the login or the first GET request and then pass this through the subsequent requests.

// Gather all cookies on the first request.
URLConnection connection = new URL(url).openConnection();
List<String> cookies = connection.getHeaderFields().get("Set-Cookie");
// ...

// Then use the same cookies on all subsequent requests.
connection = new URL(url).openConnection();
for (String cookie : cookies) {
    connection.addRequestProperty("Cookie", cookie.split(";", 2)[0]);
}
// ...

The split(";", 2)[0] is there to get rid of cookie attributes which are irrelevant for the server side like expires, path, etc. Alternatively, you could also use cookie.substring(0, cookie.indexOf(';')) instead of split().


Streaming mode

The HttpURLConnection will by default buffer the entire request body before actually sending it, regardless of whether you've set a fixed content length yourself using connection.setRequestProperty("Content-Length", contentLength);. This may cause OutOfMemoryExceptions whenever you concurrently send large POST requests (e.g. uploading files). To avoid this, you would like to set the HttpURLConnection#setFixedLengthStreamingMode().

httpConnection.setFixedLengthStreamingMode(contentLength);

But if the content length is really not known beforehand, then you can make use of chunked streaming mode by setting the HttpURLConnection#setChunkedStreamingMode() accordingly. This will set the HTTP Transfer-Encoding header to chunked which will force the request body being sent in chunks. The below example will send the body in chunks of 1KB.

httpConnection.setChunkedStreamingMode(1024);

User-Agent

It can happen that a request returns an unexpected response, while it works fine with a real web browser. The server side is probably blocking requests based on the User-Agent request header. The URLConnection will by default set it to Java/1.6.0_19 where the last part is obviously the JRE version. You can override this as follows:

connection.setRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36"); // Do as if you're using Chrome 41 on Windows 7.

Use the User-Agent string from a recent browser.


Error handling

If the HTTP response code is 4nn (Client Error) or 5nn (Server Error), then you may want to read the HttpURLConnection#getErrorStream() to see if the server has sent any useful error information.

InputStream error = ((HttpURLConnection) connection).getErrorStream();

If the HTTP response code is -1, then something went wrong with connection and response handling. The HttpURLConnection implementation is in older JREs somewhat buggy with keeping connections alive. You may want to turn it off by setting the http.keepAlive system property to false. You can do this programmatically in the beginning of your application by:

System.setProperty("http.keepAlive", "false");

Uploading files

You'd normally use multipart/form-data encoding for mixed POST content (binary and character data). The encoding is in more detail described in RFC2388.

String param = "value";
File textFile = new File("/path/to/file.txt");
File binaryFile = new File("/path/to/file.bin");
String boundary = Long.toHexString(System.currentTimeMillis()); // Just generate some unique random value.
String CRLF = "\r\n"; // Line separator required by multipart/form-data.
URLConnection connection = new URL(url).openConnection();
connection.setDoOutput(true);
connection.setRequestProperty("Content-Type", "multipart/form-data; boundary=" + boundary);

try (
    OutputStream output = connection.getOutputStream();
    PrintWriter writer = new PrintWriter(new OutputStreamWriter(output, charset), true);
) {
    // Send normal param.
    writer.append("--" + boundary).append(CRLF);
    writer.append("Content-Disposition: form-data; name=\"param\"").append(CRLF);
    writer.append("Content-Type: text/plain; charset=" + charset).append(CRLF);
    writer.append(CRLF).append(param).append(CRLF).flush();

    // Send text file.
    writer.append("--" + boundary).append(CRLF);
    writer.append("Content-Disposition: form-data; name=\"textFile\"; filename=\"" + textFile.getName() + "\"").append(CRLF);
    writer.append("Content-Type: text/plain; charset=" + charset).append(CRLF); // Text file itself must be saved in this charset!
    writer.append(CRLF).flush();
    Files.copy(textFile.toPath(), output);
    output.flush(); // Important before continuing with writer!
    writer.append(CRLF).flush(); // CRLF is important! It indicates end of boundary.

    // Send binary file.
    writer.append("--" + boundary).append(CRLF);
    writer.append("Content-Disposition: form-data; name=\"binaryFile\"; filename=\"" + binaryFile.getName() + "\"").append(CRLF);
    writer.append("Content-Type: " + URLConnection.guessContentTypeFromName(binaryFile.getName())).append(CRLF);
    writer.append("Content-Transfer-Encoding: binary").append(CRLF);
    writer.append(CRLF).flush();
    Files.copy(binaryFile.toPath(), output);
    output.flush(); // Important before continuing with writer!
    writer.append(CRLF).flush(); // CRLF is important! It indicates end of boundary.

    // End of multipart/form-data.
    writer.append("--" + boundary + "--").append(CRLF).flush();
}

If the other side is a HttpServlet, then its doPost() method will be called and the parts will be available by HttpServletRequest#getPart() (note, thus not getParameter() and so on!). The getPart() method is however relatively new, it's introduced in Servlet 3.0 (Glassfish 3, Tomcat 7, etc). Prior to Servlet 3.0, your best choice is using Apache Commons FileUpload to parse a multipart/form-data request. Also see this answer for examples of both the FileUpload and the Servelt 3.0 approaches.


Dealing with untrusted or misconfigured HTTPS sites

Sometimes you need to connect a HTTPS URL, perhaps because you're writing a web scraper. In that case, you may likely face a javax.net.ssl.SSLException: Not trusted server certificate on some HTTPS sites who doesn't keep their SSL certificates up to date, or a java.security.cert.CertificateException: No subject alternative DNS name matching [hostname] found or javax.net.ssl.SSLProtocolException: handshake alert: unrecognized_name on some misconfigured HTTPS sites.

The following one-time-run static initializer in your web scraper class should make HttpsURLConnection more lenient as to those HTTPS sites and thus not throw those exceptions anymore.

static {
    TrustManager[] trustAllCertificates = new TrustManager[] {
        new X509TrustManager() {
            @Override
            public X509Certificate[] getAcceptedIssuers() {
                return null; // Not relevant.
            }
            @Override
            public void checkClientTrusted(X509Certificate[] certs, String authType) {
                // Do nothing. Just allow them all.
            }
            @Override
            public void checkServerTrusted(X509Certificate[] certs, String authType) {
                // Do nothing. Just allow them all.
            }
        }
    };

    HostnameVerifier trustAllHostnames = new HostnameVerifier() {
        @Override
        public boolean verify(String hostname, SSLSession session) {
            return true; // Just allow them all.
        }
    };

    try {
        System.setProperty("jsse.enableSNIExtension", "false");
        SSLContext sc = SSLContext.getInstance("SSL");
        sc.init(null, trustAllCertificates, new SecureRandom());
        HttpsURLConnection.setDefaultSSLSocketFactory(sc.getSocketFactory());
        HttpsURLConnection.setDefaultHostnameVerifier(trustAllHostnames);
    }
    catch (GeneralSecurityException e) {
        throw new ExceptionInInitializerError(e);
    }
}

Last words

The Apache HttpComponents HttpClient is much more convenient in this all :)


Parsing and extracting HTML

If all you want is parsing and extracting data from HTML, then better use a HTML parser like Jsoup

@ZeissS 2010-05-28 18:54:30

You should place the apache link first, so people looking for a solution find it faster ;)

@ivanceras 2011-01-20 05:32:30

Hi BalusC, I got lost. Could you please make a much simpler example such as forwarding a request to another server while retaining the url and also no data is lost during the forwarding such as the cookies, the url string, all of the other headers such as SOAPAction if ever it is a soap request, the browser user-agent string(important not to be lost for statistics reason). -Thank you

@BalusC 2011-01-21 19:33:49

@ivanceras: If you can't boil it down based on the information in this answer, then please press Ask Question button at right top.

@oneiros 2012-02-22 20:14:41

I get a response code 500 doing a post using java.net.HTTLUrlConnection ... when doing it through jquery I don't get error even though the parameters are the same. Any idea why this is so?

@BalusC 2012-02-22 20:17:17

@imperator: It's a bug in the server's code. It threw an exception. The error page which may contain detailed exception information is available in getErrorStream(). See also "Error Handling" part in the above answer. Or if it's your own server, read its server logs.

@nikhil 2012-04-14 19:02:33

connection.getHeaderFields().entrySet() doesn't even compile. HttpURLConnection doesn't have those methods.

@Jan Algermissen 2012-04-14 19:43:20

Hey, the meaning of Accept-Charset is very different from what you write above!

@aseq 2012-05-03 07:53:01

With regards to the "System.setProperty("http.keepAlive", "false");" because "HttpURLConnection implementation is somewhat buggy", I got bitten by that. Every second connection attempt would return no results. After setting keepAlive to false it worked fine.

@BalusC 2012-10-09 13:22:58

@Brais: Please read the spec. The -- part is not part of the boundary itself. It's just a separator string. I've rolled back your invalid edit.

@user517491 2013-01-26 19:29:52

@BalusC thanks a lot for such a perfect tutorial. Please also include a heading like "Closing streams/connections". I am really confused about when and which streams/connections to close.

@Steven De Groote 2013-04-29 14:41:37

It may be good to mention the disconnect() method to actually terminate the request once you're done to prevent keeping it open for too long / eternity

@gkalpak 2013-05-09 21:53:30

@BalusC In section Firing a HTTP GET request with (optionally) query parameters it is mentioned that "The Accept-Charset header may hint the server what encoding the parameters are in.". According to the HTTP Spec "The Accept-Charset request-header field can be used to indicate what character sets are acceptable for the response." (not the request). Am I missing something here ?

@BalusC 2013-05-10 11:42:07

@Expert: That's correct. However, some servers may "guess" request parameter encoding based on this. I've also explicitly stated "may hint" and not e.g. "instructs" or so. If it doesn't work for your target server, then it's just bad luck.

@gkalpak 2013-05-10 13:30:14

@BalusC: Not that I know too much about it, but I haven't ever found a source of indication that "Accept-Charset" could hint about the request query-params encoding. Yet, if you say it does, I surely trust you more on this one than me. (Just to make clear, I have not tried it on any server, so my comment was on a pure theoreticaly speaking basis.) Thx for your answer !

@Edward Brey 2013-09-24 13:43:09

Should we add a caveat next to "HttpClient is much more convenient" that the Android SDK says "Prefer HttpURLConnection for new code"?

@Anderson 2014-01-09 03:57:07

It's necessary to set a timeout or the current thread may be blocked. See setConnectTimeout and setReadTimeout.

@yati sagade 2014-04-15 05:27:32

The sad part is that on Android it is not recommended to use the Apache HttpClient now and HttpURLConnection is cruel. android-developers.blogspot.in/2011/09/…

@To Kra 2014-06-10 19:22:48

According to my simple measures URLConn is 400-500ms faster than Apache.Httpclient

@Alston 2014-10-21 12:48:30

Why after I set httpConnection.setChunkedStreamingMode(1024);, nothing can not be uploaded? (No files appear at server.)

@Haresh Chhelana 2015-06-04 05:24:33

I can not find Files.copy() and binaryFile.toPath() such methods.

@BalusC 2015-06-04 06:49:55

@Haresh: They are part of Java 7. Like as those try-with-resources statements. Perhaps you're still on long EOL'ed Java 6 or even older? Just upgrade to at least Java 7 (we're currently already on Java 8), or replace Java 7 code by older Java equivalents. E.g. an old fashioned for loop writing a buffer from input to ouput.

@Haresh Chhelana 2015-06-04 07:12:42

@BalusC,can you please share some reference to do old fashioned for loop writing buffer from input to output or add such kind of code in ur ans ?

@scottyseus 2016-01-22 16:32:26

does anyone else find the standard lib HttpURLConnection easier to use than Apache HttpComponents?

@JonasCz 2016-02-16 15:00:19

OkHttp is even easier, works on Android. Trust me - it's so much better than plain old HttUrlConnection.

@Mike B. 2016-03-23 16:22:49

@BalusC, thanks for the detailed explanation. One more moment, JSOUP 1.8.3 contains method validateTLSCertificates(). Is there any difference/benefits between using your snippet and just call JSOUP's validateTLSCertificates() (jsoup.org/apidocs/org/jsoup/…)?

@user207421 2016-05-25 02:09:05

I'm sorry but you really should not post or recommend or even publicize these insecure TrustManagers and HostnameVerifiers. You haven't even provided a caveat.

@Luke 2018-02-12 10:37:13

How do I integrate the last part, "Dealing with untrusted or misconfigured HTTPS sites", with Jsoup.connect(httpsurl).get() method?

@BalusC 2018-02-12 11:20:43

@Luke Just copy paste the block in one of your involved classes.

@Luke 2018-02-12 13:19:02

I ran my application with -Djavax.net.debug=ssl:handshake, with and without the above code and it seems to do the same things.

@Nison Cheruvathur 2016-04-06 05:43:09

if you are using http get please remove this line

urlConnection.setDoOutput(true);

@Ali Dehghani 2016-04-29 08:07:36

Update

The new HTTP Client shipped with Java 9 but as part of an Incubator module named jdk.incubator.httpclient. Incubator modules are a means of putting non-final APIs in the hands of developers while the APIs progress towards either finalization or removal in a future release.

In Java 9, you can send a GET request like:

// GET
HttpResponse response = HttpRequest
    .create(new URI("http://www.stackoverflow.com"))
    .headers("Foo", "foovalue", "Bar", "barvalue")
    .GET()
    .response();

Then you can examine the returned HttpResponse:

int statusCode = response.statusCode();
String responseBody = response.body(HttpResponse.asString());

Since this new HTTP Client is in java.httpclient jdk.incubator.httpclient module, you should declare this dependency in your module-info.java file:

module com.foo.bar {
    requires jdk.incubator.httpclient;
}

@Utkash Bhatt 2014-12-18 12:58:46

There are 2 options you can go with HTTP URL Hits : GET / POST

GET Request :-

HttpURLConnection.setFollowRedirects(true); // defaults to true

String url = "https://name_of_the_url";
URL request_url = new URL(url);
HttpURLConnection http_conn = (HttpURLConnection)request_url.openConnection();
http_conn.setConnectTimeout(100000);
http_conn.setReadTimeout(100000);
http_conn.setInstanceFollowRedirects(true);
System.out.println(String.valueOf(http_conn.getResponseCode()));

POST request :-

HttpURLConnection.setFollowRedirects(true); // defaults to true

String url = "https://name_of_the_url"
URL request_url = new URL(url);
HttpURLConnection http_conn = (HttpURLConnection)request_url.openConnection();
http_conn.setConnectTimeout(100000);
http_conn.setReadTimeout(100000);
http_conn.setInstanceFollowRedirects(true);
http_conn.setDoOutput(true);
PrintWriter out = new PrintWriter(http_conn.getOutputStream());
if (urlparameter != null) {
   out.println(urlparameter);
}
out.close();
out = null;
System.out.println(String.valueOf(http_conn.getResponseCode()));

@user207421 2015-03-26 09:19:48

There are quite a few more: PUT, DELETE, HEAD, ...

@Zerus 2018-01-16 19:01:12

How can you view the actual JSON response?

@Ravindra babu 2015-09-25 12:07:45

Initially I was misled by this article which favours HttpClient.

Later I have been realized that HttpURLConnection is going to stay from this article

As per the Google blog:

Apache HTTP client has fewer bugs on Eclair and Froyo. It is the best choice for these releases. For Gingerbread , HttpURLConnection is the best choice. Its simple API and small size makes it great fit for Android.

Transparent compression and response caching reduce network use, improve speed and save battery. New applications should use HttpURLConnection; it is where we will be spending our energy going forward.

After reading this article and some other stack over flow questions, I am convinced that HttpURLConnection is going to stay for longer durations.

Some of the SE questions favouring HttpURLConnections:

On Android, make a POST request with URL Encoded Form data without using UrlEncodedFormEntity

HttpPost works in Java project, not in Android

@Ali Dehghani 2016-01-15 12:06:33

There is also OkHttp, which is an HTTP client that’s efficient by default:

  • HTTP/2 support allows all requests to the same host to share a socket.
  • Connection pooling reduces request latency (if HTTP/2 isn’t available).
  • Transparent GZIP shrinks download sizes.
  • Response caching avoids the network completely for repeat requests.

First create an instance of OkHttpClient:

OkHttpClient client = new OkHttpClient();

Then, prepare your GET request:

Request request = new Request.Builder()
      .url(url)
      .build();

finally, use OkHttpClient to send prepared Request:

Response response = client.newCall(request).execute();

For more details, you can consult the OkHttp's documentation

@fernandohur 2014-10-24 21:57:25

I suggest you take a look at the code on kevinsawicki/http-request, its basically a wrapper on top of HttpUrlConnection it provides a much simpler API in case you just want to make the requests right now or you can take a look at the sources (it's not too big) to take a look at how connections are handled.

Example: Make a GET request with content type application/json and some query parameters:

// GET http://google.com?q=baseball%20gloves&size=100
String response = HttpRequest.get("http://google.com", true, "q", "baseball gloves", "size", 100)
        .accept("application/json")
        .body();
System.out.println("Response was: " + response);

@yegor256 2013-11-27 10:31:52

You can also use JdkRequest from jcabi-http (I'm a developer), which does all this work for you, decorating HttpURLConnection, firing HTTP requests and parsing responses, for example:

String html = new JdkRequest("http://www.google.com").fetch().body();

Check this blog post for more info: http://www.yegor256.com/2014/04/11/jcabi-http-intro.html

@Dejell 2015-01-27 19:08:05

How do you handle cookies?

@Paal Thorstensen 2011-12-06 02:20:44

When working with HTTP it's almost always more useful to refer to HttpURLConnection rather than the base class URLConnection (since URLConnection is an abstract class when you ask for URLConnection.openConnection() on a HTTP URL that's what you'll get back anyway).

Then you can instead of relying on URLConnection#setDoOutput(true) to implicitly set the request method to POST instead do httpURLConnection.setRequestMethod("POST") which some might find more natural (and which also allows you to specify other request methods such as PUT, DELETE, ...).

It also provides useful HTTP constants so you can do:

int responseCode = httpURLConnection.getResponseCode();

if (responseCode == HttpURLConnection.HTTP_OK) {

@Patrick Kafka 2011-12-21 21:32:47

setDoOutPut true was my issue which set my GET to a POST. Thanks

@Tony Chan 2012-07-31 00:09:44

If you are trying to write data to the outputstream you must still set setDoOutput() to true otherwise an exception is thrown (even if you setRequestMethod("POST")). To be clear: setting URLConnection#setDoOutput(true) to true implicitly sets the request method to POST, but setting httpURLConnection.setRequestMethod("POST") to POST does not implicitly set setDoOutput() to true.

@RickHigh 2013-10-17 05:35:17

I was also very inspired by this response.

I am often on projects where I need to do some HTTP, and I may not want to bring in a lot of 3rd party dependencies (which bring in others and so on and so on, etc.)

I started to write my own utilities based on some of this conversation (not any where done):

package org.boon.utils;


import java.io.IOException;
import java.io.InputStream;
import java.net.HttpURLConnection;
import java.net.URL;
import java.net.URLConnection;
import java.util.Map;

import static org.boon.utils.IO.read;

public class HTTP {

Then there are just a bunch or static methods.

public static String get(
        final String url) {

    Exceptions.tryIt(() -> {
        URLConnection connection;
        connection = doGet(url, null, null, null);
        return extractResponseString(connection);
    });
    return null;
}

public static String getWithHeaders(
        final String url,
        final Map<String, ? extends Object> headers) {
    URLConnection connection;
    try {
        connection = doGet(url, headers, null, null);
        return extractResponseString(connection);
    } catch (Exception ex) {
        Exceptions.handle(ex);
        return null;
    }
}

public static String getWithContentType(
        final String url,
        final Map<String, ? extends Object> headers,
        String contentType) {
    URLConnection connection;
    try {
        connection = doGet(url, headers, contentType, null);
        return extractResponseString(connection);
    } catch (Exception ex) {
        Exceptions.handle(ex);
        return null;
    }
}
public static String getWithCharSet(
        final String url,
        final Map<String, ? extends Object> headers,
        String contentType,
        String charSet) {
    URLConnection connection;
    try {
        connection = doGet(url, headers, contentType, charSet);
        return extractResponseString(connection);
    } catch (Exception ex) {
        Exceptions.handle(ex);
        return null;
    }
}

Then post...

public static String postBody(
        final String url,
        final String body) {
    URLConnection connection;
    try {
        connection = doPost(url, null, "text/plain", null, body);
        return extractResponseString(connection);
    } catch (Exception ex) {
        Exceptions.handle(ex);
        return null;
    }
}

public static String postBodyWithHeaders(
        final String url,
        final Map<String, ? extends Object> headers,
        final String body) {
    URLConnection connection;
    try {
        connection = doPost(url, headers, "text/plain", null, body);
        return extractResponseString(connection);
    } catch (Exception ex) {
        Exceptions.handle(ex);
        return null;
    }
}



public static String postBodyWithContentType(
        final String url,
        final Map<String, ? extends Object> headers,
        final String contentType,
        final String body) {

    URLConnection connection;
    try {
        connection = doPost(url, headers, contentType, null, body);


        return extractResponseString(connection);


    } catch (Exception ex) {
        Exceptions.handle(ex);
        return null;
    }


}


public static String postBodyWithCharset(
        final String url,
        final Map<String, ? extends Object> headers,
        final String contentType,
        final String charSet,
        final String body) {

    URLConnection connection;
    try {
        connection = doPost(url, headers, contentType, charSet, body);


        return extractResponseString(connection);


    } catch (Exception ex) {
        Exceptions.handle(ex);
        return null;
    }


}

private static URLConnection doPost(String url, Map<String, ? extends Object> headers,
                                    String contentType, String charset, String body
                                    ) throws IOException {
    URLConnection connection;/* Handle output. */
    connection = new URL(url).openConnection();
    connection.setDoOutput(true);
    manageContentTypeHeaders(contentType, charset, connection);

    manageHeaders(headers, connection);


    IO.write(connection.getOutputStream(), body, IO.CHARSET);
    return connection;
}

private static void manageHeaders(Map<String, ? extends Object> headers, URLConnection connection) {
    if (headers != null) {
        for (Map.Entry<String, ? extends Object> entry : headers.entrySet()) {
            connection.setRequestProperty(entry.getKey(), entry.getValue().toString());
        }
    }
}

private static void manageContentTypeHeaders(String contentType, String charset, URLConnection connection) {
    connection.setRequestProperty("Accept-Charset", charset == null ? IO.CHARSET : charset);
    if (contentType!=null && !contentType.isEmpty()) {
        connection.setRequestProperty("Content-Type", contentType);
    }
}

private static URLConnection doGet(String url, Map<String, ? extends Object> headers,
                                    String contentType, String charset) throws IOException {
    URLConnection connection;/* Handle output. */
    connection = new URL(url).openConnection();
    manageContentTypeHeaders(contentType, charset, connection);

    manageHeaders(headers, connection);

    return connection;
}

private static String extractResponseString(URLConnection connection) throws IOException {
/* Handle input. */
    HttpURLConnection http = (HttpURLConnection)connection;
    int status = http.getResponseCode();
    String charset = getCharset(connection.getHeaderField("Content-Type"));

    if (status==200) {
        return readResponseBody(http, charset);
    } else {
        return readErrorResponseBody(http, status, charset);
    }
}

private static String readErrorResponseBody(HttpURLConnection http, int status, String charset) {
    InputStream errorStream = http.getErrorStream();
    if ( errorStream!=null ) {
        String error = charset== null ? read( errorStream ) :
            read( errorStream, charset );
        throw new RuntimeException("STATUS CODE =" + status + "\n\n" + error);
    } else {
        throw new RuntimeException("STATUS CODE =" + status);
    }
}

private static String readResponseBody(HttpURLConnection http, String charset) throws IOException {
    if (charset != null) {
        return read(http.getInputStream(), charset);
    } else {
        return read(http.getInputStream());
    }
}

private static String getCharset(String contentType) {
    if (contentType==null)  {
        return null;
    }
    String charset = null;
    for (String param : contentType.replace(" ", "").split(";")) {
        if (param.startsWith("charset=")) {
            charset = param.split("=", 2)[1];
            break;
        }
    }
    charset = charset == null ?  IO.CHARSET : charset;

    return charset;
}

Well you get the idea....

Here are the tests:

static class MyHandler implements HttpHandler {
    public void handle(HttpExchange t) throws IOException {

        InputStream requestBody = t.getRequestBody();
        String body = IO.read(requestBody);
        Headers requestHeaders = t.getRequestHeaders();
        body = body + "\n" + copy(requestHeaders).toString();
        t.sendResponseHeaders(200, body.length());
        OutputStream os = t.getResponseBody();
        os.write(body.getBytes());
        os.close();
    }
}


@Test
public void testHappy() throws Exception {

    HttpServer server = HttpServer.create(new InetSocketAddress(9212), 0);
    server.createContext("/test", new MyHandler());
    server.setExecutor(null); // creates a default executor
    server.start();

    Thread.sleep(10);


    Map<String,String> headers = map("foo", "bar", "fun", "sun");

    String response = HTTP.postBodyWithContentType("http://localhost:9212/test", headers, "text/plain", "hi mom");

    System.out.println(response);

    assertTrue(response.contains("hi mom"));
    assertTrue(response.contains("Fun=[sun], Foo=[bar]"));


    response = HTTP.postBodyWithCharset("http://localhost:9212/test", headers, "text/plain", "UTF-8", "hi mom");

    System.out.println(response);

    assertTrue(response.contains("hi mom"));
    assertTrue(response.contains("Fun=[sun], Foo=[bar]"));

    response = HTTP.postBodyWithHeaders("http://localhost:9212/test", headers, "hi mom");

    System.out.println(response);

    assertTrue(response.contains("hi mom"));
    assertTrue(response.contains("Fun=[sun], Foo=[bar]"));


    response = HTTP.get("http://localhost:9212/test");

    System.out.println(response);


    response = HTTP.getWithHeaders("http://localhost:9212/test", headers);

    System.out.println(response);

    assertTrue(response.contains("Fun=[sun], Foo=[bar]"));



    response = HTTP.getWithContentType("http://localhost:9212/test", headers, "text/plain");

    System.out.println(response);

    assertTrue(response.contains("Fun=[sun], Foo=[bar]"));



    response = HTTP.getWithCharSet("http://localhost:9212/test", headers, "text/plain", "UTF-8");

    System.out.println(response);

    assertTrue(response.contains("Fun=[sun], Foo=[bar]"));

    Thread.sleep(10);

    server.stop(0);


}

@Test
public void testPostBody() throws Exception {

    HttpServer server = HttpServer.create(new InetSocketAddress(9220), 0);
    server.createContext("/test", new MyHandler());
    server.setExecutor(null); // creates a default executor
    server.start();

    Thread.sleep(10);


    Map<String,String> headers = map("foo", "bar", "fun", "sun");

    String response = HTTP.postBody("http://localhost:9220/test", "hi mom");

    assertTrue(response.contains("hi mom"));


    Thread.sleep(10);

    server.stop(0);


}

@Test(expected = RuntimeException.class)
public void testSad() throws Exception {

    HttpServer server = HttpServer.create(new InetSocketAddress(9213), 0);
    server.createContext("/test", new MyHandler());
    server.setExecutor(null); // creates a default executor
    server.start();

    Thread.sleep(10);


    Map<String,String> headers = map("foo", "bar", "fun", "sun");

    String response = HTTP.postBodyWithContentType("http://localhost:9213/foo", headers, "text/plain", "hi mom");

    System.out.println(response);

    assertTrue(response.contains("hi mom"));
    assertTrue(response.contains("Fun=[sun], Foo=[bar]"));

    Thread.sleep(10);

    server.stop(0);


}

You can find the rest here:

https://github.com/RichardHightower/boon

My goal is to provide the common things one would want to do in a bit more easier way then....

@Vit Khudenko 2013-11-27 13:48:04

It's odd that in the doPost method there is a charset param, that is used to set request header, but then the data is written with some hard-coded charset IO.CHARSET. A bug?

@David Chandler 2012-06-13 16:37:47

Inspired by this and other questions on SO, I've created a minimal open source basic-http-client that embodies most of the techniques found here.

google-http-java-client is also a great open source resource.

@David 2013-01-23 00:24:28

I was just thinking the same. But it might also be nice to have a barebones/simple Java library that only uses URLConnection code as presented here that encapsulates the code to simpler methods to do an HTTP GET, POST, etc. The library can then be compiled & packaged as JAR and imported/used in Java code or the source class file can be included in Java project if external JARs are not desired. This could be done with other libraries like Apache, etc. but is more of a pain compared to a simple 1 file class library using URLConnection.

@Ravindra babu 2015-09-25 11:48:25

rapidvaluesolutions.com/tech_blog/… favors HttpURLConnection over HttpClient

Related Questions

Sponsored Content

60 Answered Questions

[SOLVED] How do I read / convert an InputStream into a String in Java?

64 Answered Questions

[SOLVED] How do I generate random integers within a specific range in Java?

  • 2008-12-12 18:20:57
  • user42155
  • 3787997 View
  • 3205 Score
  • 64 Answer
  • Tags:   java random integer

24 Answered Questions

[SOLVED] HTTP GET request in JavaScript?

29 Answered Questions

[SOLVED] JavaScript post request like a form submit

10 Answered Questions

[SOLVED] How to make HTTP POST web request

13 Answered Questions

[SOLVED] Are HTTPS URLs encrypted?

  • 2009-01-31 21:15:34
  • Daniel Kivatinos
  • 235026 View
  • 874 Score
  • 13 Answer
  • Tags:   ssl https httprequest

19 Answered Questions

[SOLVED] How is an HTTP POST request made in node.js?

6 Answered Questions

[SOLVED] Custom HTTP headers : naming conventions

  • 2010-08-24 21:59:38
  • Julien Genestoux
  • 315057 View
  • 969 Score
  • 6 Answer
  • Tags:   http http-headers

8 Answered Questions

[SOLVED] HTTP status code for update and delete?

8 Answered Questions

[SOLVED] How are parameters sent in an HTTP POST request?

Sponsored Content