How an HTTP Request Works
Have you ever wondered what happens behind the scenes when making an HTTP request?
Well, I've wondered about it, so I decided to dig a little bit deeper and see what I could find.
This article is what I learned, and I'm going to do my best to explain how an HTTP request travels around the Internet.
The Internet Protocols
First, let's examine what makes computers able to interact with each other. To communicate efficiently through the Internet, computers use protocols, a set of rules and standards.
These protocols are all within the Internet protocol suite, commonly known as the TCP/IP (Transmission Control Protocol/Internet Protocol) model.
This model has four different layers: the application layer, transport layer, internet layer, and link layer.
Here are the layers with some example protocols:
| Layer | Protocol |
|---|---|
| Application | HTTP(S) |
| Transport | TCP |
| Internet (or Network) | IP |
| Link (or Network Access/Interface) | IEEE 802 (WiFi) |
When a message travels from one computer to another, it goes through all the layers in both computers.
Starting at the application layer from the sender computer (a client), through the transport, internet, and link layers, and then in reverse order on the receiving computer (a server).
High-Level Flow
Let's start with a high-level overview of what happens and then go into more detail.
The application (Browser, Postman, etc.) queries a Domain Name System (DNS) server to get an IP address for the domain name ("www.example.com").
A reliable connection is made with the receiving computer with a TCP handshake.
If using HTTPS, a TLS handshake is performed to ensure the connection is secure.
The data is split into segments, and information like the senders' and receivers' IP addresses is added. The segments are then turned into packets with the size and sequence order (to help the receiver reassemble the segments) attached.
The packets travel through the internet using different paths.
When the receiver gets the packets, it error-checks them, makes sure it has all the packets, and reassembles the segments before sending them to the application, which performs the request.
The receiver sends a response back, and the connection closes.
Let's see what happens at each layer when we make an HTTP request.
Application Layer
Let's say we have clicked on an item on a website, which will be at www.example.com/items/2. The first thing that the application (Chrome, Firefox, Postman) we're using needs to do is find which computer on the Internet has this information, meaning find its IP address.
To do this, it needs to get the domain name, www.example.com, and translate it to an IP address. It does this with a DNS (Domain Name System) lookup.
DNS Lookup
The Domain Name System (DNS) is like a phonebook for the internet. Since IP addresses are hard to remember for humans, we use domain names like example.com.
However, computers need IP addresses to communicate with each other. The DNS helps applications convert domain names into IP addresses.
For example, 157.240.21.35 is one of the IP addresses for facebook.com. So, if you type that IP address in your browser, it will direct you to facebook.com.
So, the application will send a request to the DNS server and ask for the IP address for the specified domain name.
Your internet service provider will provide the IP address of the DNS server to query. Usually, the server is geographically close to you, so the query will take the shortest time possible.
For more information on DNS, have a look at this article.
Once we have the IP address, the application will add a port number to it. This port number tells the server (receiving computer) which protocol we want to communicate through.
For example, port 80 for HTTP and 443 for HTTPS are used. Port 25 is also common for the Simple Mail Transfer Protocol (SMTP) or email.
UDP Protocol
When making a DNS lookup, it's usually through the User Datagram Protocol (UDP) instead of the Transmission Control Protocol (TCP), which most requests go through.
TCP is a more reliable connection than UDP; we'll see why that is below and has error checking and recovery. But UDP has none of that, and because of that, it's faster than TCP.
Also, the DNS query is usually sent over an unencrypted connection (UDP), even if you visit a site over HTTPS, which is done to save time.
However, the DNS query can be done over an encrypted connection using the DoH (DNS over HTTPS) protocol.Learn more about UDP here
Now that the application layer has the message we want to send, the server IP address and the port number, it sends that information to the transport layer.
Transport Layer
The transport layer divides the data into segments.
It also adds a TCP header to each segment with the client and server ports, segment ordering information, and a data field known as a checksum to verify that the data has been transferred without error.
Before the transport layer can send the segments off, it needs to make sure it has a connection with the server. It does so with a TCP handshake.
TCP Handshake
A TCP handshake is a process that ensures the reliability of the connection between the client and server. It's also known as a three-way handshake because it involves three steps.
- The client sends a segment called a SYN to the server.
- The server receiving that segment sends another segment back, SYN-ACK, to acknowledge that it successfully received the SYN segment.
- In the final step, the client acknowledges the server's response by sending an ACK segment to the server.
To read more about the TCP handshake, see this article.
Once the connection is stable, the client can start sending the actual message.
TLS (SSL) Handshake
If the request uses HTTPS protocol, port 443, it must also perform a TLS (SSL) handshake to verify that the connection is secure.
It's a bit more complicated than the TCP handshake, and if you want to learn more about that, I recommend this article.
The next action of the transport layer is to transmit the segments to the internet layer.
Internet Layer
The internet layer adds an IP header to the segments and formats them into a packet.
The IP header includes the client and server IP addresses and the packets' length and sequence order.
Then, the packets are handed over to the last layer, which is the link layer.
Link Layer
The last layer, the link layer, turns the packets into a frame, attaching the third header and a footer to "frame" the packet. The header includes a field called cyclical redundancy check (CRC), which is used for error checking.
It then defines how the data should be sent physically through the network.
And voila, the data is on its way.
Data Transfer
Now, the internet is a bunch of computers and networks of computers connected together.
An example of a computer network is your home network. Your laptop, phone, printer, and Alexa are connected to your router. Your ISP then makes sure your router is linked to other routers, which in turn are linked to other routers.
Eventually, all computers on the internet are connected through multiple routers.
So, the client's data travelling to the server can take many paths. The packets will most likely take different paths from each other because it's faster that way, instead of going the same route and possibly causing a "traffic jam."
But that also means the packets can arrive at the server at different times and in a separate order, then be sent out.
Once the server has all the packets, it can put them together correctly because the ordering information is included in the headers attached.
Receiving Data
When the data arrives at the server, it will travel through the layers in reverse order.
Here's what will happen:
- Link-layer error checks the packet using the CRC field.
- The internet layer makes sure it has received all the packets
- Transport layer error checks the checksum field and reassembles the segments.
- Application layer receives the message and performs the request
Connection Closed
Once it has performed the request, it will send a response. When the response data has been transferred back to the client, the connection between the computers will close.
Thanks
Hopefully, this helped you understand better how an HTTP request travels the internet.
I had great fun digging around and learning about it.
Thank you for reading.