A basic understanding of HTTP and socket programming

This blog explains the basic understanding of HTTP, client-server model, network layers, and some socket programming in Java.

author profile image
Sakchhyam G
A basic understanding of HTTP and socket programming

Table of content

  1. Roles on the Internet
  2. What is HTTP?
  3. Layers of the Internet
  4. Coding
  5. Conclusion

Roles on the Internet

Developers and designers can select different arrangements and architecture for their computers to develop an internet service. In this article, we are talking about the most commonly used client-server model. There are mainly two roles in this model:

  • client: A client computer initiates a request for the resources and the information from the server computer.
  • server: A server computer processes the client request and responds with appropriate information.
client server model {caption: Fig.1: Client-Server model}
Fig.1: Client-Server model

What is http?

To conduct communication between a client and a server a common language is necessary. We can develop and use any language we desire to make these computers communicate with each other. But to make it usable, compatible, and widely accessible we need a standardized language (aka protocol). This is where the HTTP comes in. - HTTP stand for Hypertext Transfer Protocol.

  • HTTP protocol defines the structure for request and response messages. It is like sending a parcel to your friend an address structured as street name, city name, state name, zip code, and country code.
  • HTTP defines operations that can be performed in a given resource which is called HTTP request methods. It is also referred to as HTTP verbs. Some examples are:
    • The GET method requests a representation/state of the specified resource/data. It should only retrieve data.
    • The POST method submits an entity to the specified resource, often causing a change in the server.
  • HTTP protocol also defines different sets of status codes to indicate different states that may occur during communication like successful responses (200-299), client error responses (400-499), server error responses (500-599), etc.
  • HTTP requests and responses share a similar structure and are composed of:
    • A single start-line describing the requests/response to be implemented, or it's status of whether successful or a failure.
    • An optional set of HTTP headers specifying the request/response or describing the body included in the message.
    • A blank line indicating all meta-information for the request has been sent.
    • An optional body containing data associated with the request (like the content of an HTML form) or the document associated with a response. The presence of the body and its size is specified by the start-line and HTTP headers.
HTTP Message Structure {caption: Fig.2: Structure of HTTP message}
Fig.2: Structure of HTTP message

Layers of the Internet

The software model of the Internet can be defined by two types of layered models: OSI model and IP model Internet Protocol Suite. OSI model stands for Open Systems Interconnection model. It defines the standard for the Internet and networking. Hence, it is also called the reference model. IP model stands for Internet Protocol model. It is practically implemented and used by the Internet. Without going into much depth main points are:

  • Every computer (client or server), and network devices that supports the Internet implements the IP suite model.
  • These models are structured as layers hence called layered structure
  • Each layer has its responsibility and hides its implementation (inner working) from other layers
  • Each layer provides an interface to layers immediately above or below itself

In IP suite model, Application layer combines all three layers from OSI model i.e Application + Presentation + Session
In IP suite model, Link layer combines lower two layers from OSI model i.e Physical + Datalink
The lowermost layer from each model connects with physical network devices like routers, switches, etc.

Layered Model of a network {caption: Fig.3: Layered Model of a network}
Fig.3: Layered Model of a network

HTTP comes under the Application layer which is accessible to the user process. HTTP protocol uses the services provided by the Transport layer (TCP or UDP) to establish and conduct communication with other computers on the Internet. But to glue these two layers, the operating system provides an interface called socket API. Programming that utilizes socket API is called socket programming.

A socket is like the two ends of a water pipe that handles the flow of water. But the only difference is that the information flows in both directions in a socket. A socket can be identified by a socket address which combines protocol type, IP address, and port number. Socket API makes it easy to use different protocols and develop communication in a network. The following figure shows a general flow of communication using socket API. We will understand more about these socket functions later in our coding part.

This figure shows the socket functions implemented in the C language. In this article, we will be using socket functions from Java.

Flow of communication between client-server socket using TCP (in C) {831x930} {caption: Fig.4: Flow of communication between client-server socket using TCP (in C)}
Fig.4: Flow of communication between client-server socket using TCP (in C)


Basic files and project structure

Note: Please first install java JDK for this project, setup your environment variables, and check everything is working well.

  1. Main project folder will contain a public and a resources folder. In the public folder, we will store all our .html pages, and in the resources folder, we will store our resources like images, videos, etc.
  2. Our main server program WebServer.java will be in the project's root directory.
1 <!-- ./public/index.html --> 2 <!DOCTYPE html> 3 <html> 4 <head> 5 <meta charset="UTF-8"> 6 <title>Cat Paws</title> 7 </head> 8 <body> 9 <div> 10 <h1 style="text-align: center;">Cat Paws</h1> 11 <a style="display: flex; justify-content: center;" href="cats">Serve me some cat videos</a> 12 </div> 13 </body> 14 </html>
1 <!-- ./public/cats.html --> 2 <!DOCTYPE html> 3 <html lang="en"> 4 <head> 5 <meta charset="UTF-8"> 6 <title>Cat Paws</title> 7 </head> 8 <body> 9 <h1 style="text-align: center;">Cat-videos</h1> 10 <div style="display: flex; justify-content: center;"> 11 <video width="640" height="480" controls> 12 <!-- 'watch' will be replaced later by the server --> 13 <source src="watch/cat-test-video" type="video/mp4"> 14 </video> 15 </div> 16 </body> 17 </html>
  • If you are coding along with this article please store any video inside the ./resources folder and name it cat-test-video.mp4
  • The project structure should look something like this.
Project Structure  {caption: Fig.5: Project Structure}
Fig.5: Project Structure

Creating a socket and listening for new connection

1// Inside WebServer.java file 2// Some necessary imports 3import java.net.*; 4import java.io.*; 5 6import java.nio.file.Files; 7import java.nio.file.Path; 8import java.nio.file.Paths; 9 10import java.security.NoSuchAlgorithmException; 11import java.util.Arrays; 12import java.util.Scanner; 13import java.util.regex.Matcher; 14import java.util.regex.Pattern;
1// Inside WebServer.java file 2public class WebServer{ 3 public static void main (String args[]) throws IOException NoSuchAlgorithmException { 4 int serverPort = 9090; 5 // allocating resources for server's socket and assigning a port number 6 ServerSocket listenSocket = new ServerSocket(serverPort); 7 try{ 8 System.out.println("Server program started..."); 9 while(true) { 10 // listening to the assigned port number and accepting 11 // new client connection 12 System.out.println("\nListening for new connection..."); 13 Socket clientSocket = listenSocket.accept(); 14 System.out.println("\nA client is connected....."); 15 16 // Initializing 'Connection' class with 17 // client's socket address 18 Connection c = new Connection(clientSocket); 19 } 20 } catch(IOException e) { 21 System.out.println("Listen error:" + e.getMessage()); 22 } finally{ 23 listenSocket.close(); 24 } 25 } 26}

The above code does the following things:

  1. Creates a socket and assigns a port number to listen (aka server's socket address).
  2. The server's system listens to the assigned port for any new connection request from the client-side
  3. If the connection is successful and accepted by the server, it returns the client's socket address
  4. Initializes the Connection class with the client's socket info to process the client's HTTP request and send back an HTTP response

Creating a Connection class

1// Inside WebServer.java file 2// Thread class is extended to use multi-threading facility 3class Connection extends Thread { 4 DataInputStream in; 5 DataOutputStream out; 6 Socket clientSocket; 7 8 public Connection (Socket clientSocket) { 9 try { 10 this.clientSocket = clientSocket; 11 // accessing input stream from the given socket 12 this.in = new DataInputStream( this.clientSocket.getInputStream()); 13 // accessing output stream from the given socket 14 this.out = new DataOutputStream( this.clientSocket.getOutputStream()); 15 // starts a new thread 16 this.start(); 17 } catch(IOException e) { 18 System.out.println("Connection: "+e.getMessage()); 19 } 20 } 21 22 // continue...

The above code does the following things:

  1. Accesses the input stream from the link that bridges the server and the client socket. This stream is used to read the client's HTTP request.
  2. Accesses the output stream from the link that bridges the server and the client socket. This stream is used to send an HTTP response from the server computer.

Processing client's HTTP request and sending HTTP response from server

1 // continue Connection class... 2 3 // overiding run method from Thread class 4 public void run(){ 5 // Initializing a Scanner class to read stream input from the 6 // socket connection between server and client 7 Scanner scan = new Scanner(this.in, "UTF-8"); 8 try{ 9 // separating the HTTP request header from rest of the HTTP message 10 // "\\r\\n\\r\\n" below refers to the 'empty line' from fig.1 11 String requestHeader = scan.useDelimiter("\\r\\n\\r\\n").next(); 12 13 System.out.println("*********Header Data***********"); 14 System.out.println("*******************************"); 15 System.out.println(requestHeader); 16 System.out.println("*******************************\n\n"); 17 18 // decomposing request header to understand client request message 19 String[] headerLines = requestHeader.split("\r\n"); 20 String[] startLine = headerLines[0].split(" "); 21 22 String method = startLine[0]; // GET method 23 String path = startLine[1]; // index.html 24 String version = startLine[2]; // HTTP version 25 26 // handling GET request from the client 27 if (method.equals("GET")) { 28 // calling helper method to search and get the path that 29 // leads to the resource client is looking for 30 Path filePath = getFilePath(path); 31 32 System.out.println("file path: " + filePath); 33 // checking if resource path exists 34 if (Files.exists(filePath)) { 35 // file exist 36 // calling helper method to guess the file extension 37 // type like .html, .mp4, etc 38 String contentType = guessContentType(filePath); 39 40 System.out.println("file exist"); 41 System.out.println("content type:" + contentType); 42 43 // calling helper method and setting standard HTTP 44 // status code, reponse type, and response content 45 sendResponse("200 OK", contentType, "", Files.readAllBytes(filePath)); 46 47 } else { 48 // file not found 49 System.out.println("File not found"); 50 51 byte[] notFoundContent = "<h1> File not found :( </h1>".getBytes("UTF-8"); 52 53 sendResponse("404 Not Found", "text/html", "", notFoundContent); 54 } 55 out.flush(); 56 break; 57 } 58 } catch(Exception e) { 59 System.out.println("EOF:"+e); 60 } finally { 61 scan.close(); 62 try { 63 clientSocket.close(); 64 }catch (IOException e){ 65 /*close failed*/ 66 } 67 } 68 } 69 // continue...

Some helper methods

1 // continue Connection class... 2 3 // helper method: getting the path to access the requested resource/information hosted by the server (aka routing) 4 private static Path getFilePath(String path) { 5 Matcher match = Pattern.compile("/watch/").matcher(path); 6 if(match.find()){ 7 // replacing "watch" with "resources" because we have our videos inside that directory 8 path = path.replaceAll("/watch/", "./"); 9 path = path + ".mp4"; 10 return Paths.get(path); 11 } 12 13 if ("/".equals(path)) { 14 path = "index.html"; 15 }else if("/cats".equals(path)){ 16 path = "cats.html"; 17 } 18 19 return Paths.get("./public/", path); 20 } 21 22 // helper method: guessing the resource extension eg: .html, .mp4 etc 23 private static String guessContentType(Path filePath) throws IOException { 24 return Files.probeContentType(filePath); 25 } 26 27 // helper method: creating response message for the client's request 28 private void sendResponse(String status, String contentType, String additional_header, byte[] content) throws IOException { 29 byte[] response = ( 30 "HTTP/1.1 " + status + "\r\n" 31 + "Content-Type: " + contentType + "\r\n" 32 + additional_header 33 + "\r\n").getBytes("UTF-8"); 34 out.write(response, 0, response.length); 35 out.write(content, 0, content.length); 36 out.write("\r\n\r\n".getBytes("UTF-8")); 37 } 38}

Compiling the WebServer.java file

  • Open a command line in the project directory and compile the java file with the command javac WebServer.java. Two class files WebServer.class and Connection.class will be created.
  • Then run the server with the command java WebServer
  • Open the browser, which will be our client, and you can open the inspect windows and then the network tab in the browser to see the network activity
  • Type localhost:9090 in the address bar to send a request to our server. Then our index.html page be loaded
  • Now click the blue link Serve me some cat videos to request the server for a video
  • In the network tab, we can notice that after clicking the link. Our client sends two requests to the server: one for the cats.html page and one for the video file.


What we have learned in this article is just a speck of technology used on the Internet. We should be thankful for all those brilliant minds and engineering that make the Internet possible. Most importantly stay curious, and learn.