Building an HTTP Server From Scratch using TCP.
HTTP is something I interact with every single day, whether I am fetching data on the frontend or building backend APIs. But for a long time, if you had asked me how it actually worked under the hood, I wouldn't have had a good answer. I always treated HTTP like this intimidating, hyper-complex black box that would be miserable to understand, let alone rebuild from scratch. Once I actually dug…
HTTP is a ubiquitous protocol that we encounter daily, whether for fetching data or building APIs. Initially, I found the inner workings of HTTP intimidatingly complex. However, as I delved deeper, I discovered that its core mechanics are surprisingly simple. For this project, I utilized Go, but the concepts are language-agnostic, allowing implementation in several programming languages.
HTTP is a text-based protocol that rides atop TCP/IP. TCP/IP's Internet Protocol (IP) task is to transmit raw data between machines across the globe. To accomplish this, IP divides large files into smaller packets, typically around 1,500 bytes each, due to network hardware limitations and the need for efficient bandwidth sharing among numerous users. Each device on the internet possesses an IP address, allowing packets to reach their intended destinations.
When visiting a website, our computer initially requests the IP address of the target server via a DNS server. Following this, it encapsulates its own IP address in the packet header and transmits the packets across the network. IP is a "best-effort" protocol, meaning it launches packets into the network and ceases caring about their fate post-launch. If packets are lost or arrive out of order, it's up to other protocols to rectify these issues.
TCP (Transmission Control Protocol) bridges this gap by establishing a reliable stream atop IP. TCP employs sequence numbers to ensure data is reassembled correctly, even if packets arrive out of order. Additionally, TCP introduces ports, specifying where received data should be routed on a computer's IP address, thereby differentiating various services (e.g., web server vs. database) listening on the same IP address.
While TCP excels at reliably transmitting bytes, it remains oblivious to the content of those bytes. It treats all data as an unstructured, continuous stream. This limitation necessitated the creation of HTTP (Hypertext Transfer Protocol), a structured text format that translates TCP's raw byte stream into comprehensible, predictable messages.
HTTP employs a standardized request-response mechanism, specifying the action, target, protocol version, and data format (e.g., JSON, XML) through headers and a structured body, ultimately ensuring seamless communication between diverse applications and systems.
Written by urgent.news from Dev.to's reporting — not their text. Machine-written — may contain errors; check the original before relying on it.
