How to code a basic HTTP server using NodeJS

A good method to learn and progress in programming is to try to re-code existing projects. This is what I propose in this article to learn more about Node.js.

This article is aimed at beginners, without dwelling more than necessary on the basic notions of this environment. Feel free to visit the list of articles on Node.js and JavaScript to learn more.

When I started learning Node.js, the HTTP server was the typical example that was quoted in all articles, blogs and other courses. The goal was to show how easy it is to create a web server with Node.js.

const http = require('http')

const server = http.createServer(function (req, res) {
  res.writeHead(200)
  res.end('Hello, World!')
})

server.listen(8080)

Well, we quickly realize that it takes a little more code to have a real website, thanks to Node.js. This is why backend frameworks for Node.js have multiplied rapidly. The best known of them is called Express.

Express is easy to use, quick to set up and has a very large community.

const express = require('express')
const app = express()

app.get('/', (req, res) => {
  res.send('Hello World!')
})

app.listen(3000)

Without further complicating the code, the code above allows you to return a string for an HTTP method and a specific route. It is very simple to add routes in the same way as for /.

Express is very lightweight. It uses a middleware system that increases the capabilities of the framework.

For example, to allow Express to understand the JSON in the body of an HTTP request, you must use the bodyParser middleware.

How does Express work?

For those who already knew Express before this article – and even those who are discovering it now – have you ever wondered how Express works?

What if we code a small web framework to understand how our tools work? Don't misunderstand my intentions. The purpose of this article is not to code yet another framework in an ecosystem that is already saturated with them, and even less to produce production-ready code.

Besides, I advise you in general not to “reinvent the wheel” in your projects. Node.js has a rich universe of open-source modules for all your needs, do not hesitate to dig into it.

On the other hand, even if in everyday life it is better to use an existing backend framework, that does not prevent you from coding one for purely educational purposes!

Let's code a backend framework in Node.js

As I put the “Hello, World!” code of Node.js and Express, I will also put mine for you. This will serve as our goal.

import { createServer } from './lib/http/create-server.js'
import { url, router } from './lib/http/middlewares/index.js'
import { get } from './lib/http/methods/index.js'

const server = createServer(
  url(),
  router(
    get('/', () => `Hello, World!`),
  )
)

server.listen(3000)

This code does exactly the same thing as the one in Express. The syntax differs a lot — it is greatly inspired by RxJS's .pipe() — but it's still about displaying “Hello, World!” when the user goes to / and to return a 404 if going to an unknown route.

In the idea, we find the Express middlewares through which the request will pass to create a response to return to the client (the user's browser).

It's a very simplified diagram, but you get the idea.

You may have understood it, thanks to the syntax (and the reference to RxJS), I would like us to manage to have a rather functional approach with this project. When done well, I find it produces much more expressive code.

The HTTP server

The first function to implement is createServer. It is just a wrapper of the function of the same name in the http module of Node.js.

import http from 'http'

export function createServer() {
  return http.createServer(() => {
    serverResponse.end()
  })
}

createServer creates the server and returns it. We can then use .listen() to launch the server.

The http.createServer() callback function can take an IncommingMessage (the request made by the client) and a ServerResponse (the response that we want to return to it) as parameters.

One of our goals is to have a middleware system that will each in turn modify the request to gradually build the response to be returned to the client. For this, we need a list of middlewares to which we will pass the request and the response each time.

import http from 'http'

export function createServer(middlewares = []) {
  return http.createServer((incommingMessage, serverResponse) => {
    for (const middleware of middlewares) {
      middleware(incommingMessage, serverResponse)
    }

    serverResponse.end()
  })
}

To retrieve the middlewares and have the same syntax as the .pipe() of RxJS, I use the rest parameter that we saw in a previous article.

import http from 'http'

export function createServer(...middlewares) {
  return http.createServer((incommingMessage, serverResponse) => {
    for (const middleware of middlewares) {
      middleware(incommingMessage, serverResponse)
    }

    serverResponse.end()
  })
}

Thus middleware such as url and router can modify incommingMessage and/or serverResponse by reference. At the end of our modification pipe, we trigger the .end() event which will send the response to the client.

A few things still bother me about this code:

• incommingMessage and serverResponse are directly modified. I would prefer that they were not to stay closer to the philosophy of functional programming.
• All middleware must be called in our implementation, but Express allows you to stop the “pipe” of modifications if necessary. This is useful for an authentication middleware, for example.
• Some middleware might need to block the execution of the modification “pipe” to perform slightly longer tasks. Currently, our code would not wait and the execution of certain middleware could overlap in time.

So let's fix all that. First, the last point. It's the easiest to set up.

import http from 'http'

export function createServer(...middlewares) {
  return http.createServer(async (incommingMessage, serverResponse) => {
    for (const middleware of middlewares) {
      await middleware(incommingMessage, serverResponse)
    }

    serverResponse.end()
  })
}

Thus, if the middleware is executed asynchronously, we will wait for the end of its execution to move on.

To avoid the modification of incommingMessage and serverResponse, I think the easiest way is to use the return value of the middlewares. These are simple functions, let's use them as such: input values that must not be modified, but which are used to construct the return value. This return value is then used as the input value of the following (middleware) function. And so on.

import http from 'http'

export function createServer(...middlewares) {
  return http.createServer(async (incommingMessage, serverResponse) => {
    let requestContext = {
      statusCode: 200,
    }

    for (const middleware of middlewares) {
      requestContext = await middleware(incommingMessage, requestContext)
    }

    serverResponse.writeHead(requestContext.statusCode)

    if (requestContext.responseBody != null) {
      serverResponse.end(requestContext.responseBody)
      return
    }

    serverResponse.end()
  })
}

I created a requestContext which is a working object for middlewares. It is passed as input value to all middleware alongside the request.

The middlewares process these values and create a new context, which they then returns. We overwrite the old context with the new one which is more up to date.

When we are done modifying the context by the middlewares, we use it to emit the response from the server.

Finally, to be able to stop the pipe in the middle, I found nothing better than a small boolean. We can add it to the context. If a middleware changes it to true, the pipe is broken and the response is sent directly with the current context:

import http from 'http'

export function createServer(...middlewares) {
  return http.createServer(async (incommingMessage, serverResponse) => {
    let requestContext = {
      statusCode: 200,
      closeConnection: false
    }

    for (const middleware of middlewares) {
      if (requestContext.closeConnection === true) {
        break
      }

      requestContext = await middleware(incommingMessage, requestContext)
    }

    serverResponse.writeHead(requestContext.statusCode)

    if (requestContext.responseBody != null) {
      serverResponse.end(requestContext.responseBody)
      return
    }

    serverResponse.end()
  })
}

Parse the request URL

Let's move on to writing our first middleware: url. Its purpose is to parse the URL of the request to provide us with information that other middlewares might need next.

export function url() {
  return (incomingMessage, requestContext) => ({ ...requestContext })
}

The middleware API is simple in our framework.

This is a function that can take as parameter everything it may need for its operation or to allow the user to modify its operation. Here url does not take any parameters.

This function will then return another function which will be executed later in the for loop (const middleware of middlewares) of the createServer callback. It is this function that takes incommingMessage and requestContext as parameters and then returns a new version of requestContext.

export function url() {
  return (incomingMessage, requestContext) => ({
    ...requestContext,
    url: new URL(incomingMessage.url, `http://${incomingMessage.headers.host}`),
  })
}

The url middleware adds to the context of the request a url attribute which contains a parsed URL object.

We can therefore make requestContext.url.pathname and have access to the pathname of the request.

The router of our framework

The router of our framework will also be a middleware usable with createServer.

To make it simple, we will define a route as follows:

const route = {
  method: 'GET',
  pathname: '/',
  controller(incommingMessage, requestContext) {
    return `Hello, World!`
  }
}

This object is made up of three essential pieces of information for our router:

• the concerned pathname
• the HTTP verb used
• the function capable of generating something to return to the client

As input to our router middleware, we will take an array of routes.

export function router(routes = []) {
  return (incommingMessage, requestContext) => {
    for (const route of routes) {
      if (route.pathname !== requestContext.url.pathname) continue
      if (route.method !== requestContext.method) continue

      return {
        ...requestContext,
        responseBody: route.controller(incommingMessage, requestContext),
      }
    }
  }
}

Each time a client makes a request to the server, the router will scroll down the list of routes and check which one matches the client's request. As soon as it finds one, it stops the loop and executes the controller function of the route.

What the controller returns can then be added to the context as a responseBody. This is what the server returns to the client.

export function router(routes = []) {
  return (incommingMessage, requestContext) => {
    for (const route of routes) {
      if (route.pathname !== requestContext.url.pathname) continue
      if (route.method !== requestContext.method) continue

      return {
        requestContext,
        responseBody: route.controller(incommingMessage, requestContext),
        closeConnection: true,
      }
    }

    return {
      ...requestContext,
      statusCode: 404,
    }
  }
}

To make the client understand what happens when the router can't find the requested route, I added 404 error code handling after the loop. The return in the loop stops the function completely, so if the function did not stop in the middle of the loop, it is necessarily that we did not find a match in the declared routes. We therefore return a context with the statusCode at 404.

With this function, we can declare our routes as follows:

router([
  { 
    method: 'GET', 
    pathname: '/', 
    controller() { 
      return `Hello, World!` 
    } 
  },
])

Everything works fine and we could stop there, but we don't have exactly the same code as the one I showed you at the beginning of the article.

All that's missing is syntactical sugar: code that will only serve to make our features easier and more enjoyable to use.

export function get(pathname, controller) {
  return {
    method: 'GET',
    pathname,
    controller,
  }
}

This function is only used to return the “route” object needed by the router function. Nothing complicated here.

export function get(pathname, controller) {
  if (typeof controller !== 'function') {
    throw new Error('The get() HTTP method needs a controller to work as expected')
  }
    
  if (typeof pathname === 'string') {
  	throw new Error('The get() HTTP method needs a pathname to work as expected')
  }

  return { method: 'GET', controller, pathname }
}

Another advantage of this function is that it allows us quite simply to test our data without complicating the code of our router.

The router is now used as follows:

router([
  get('/', () => `Hello, World!`)
])

We are very close to the result we want to achieve. The last changes are to be made in the implementation of router itself.

export function router(...routes) {
  return (incommingMessage, requestContext) => {
    for (const route of routes) {
      if (route.pathname !== requestContext.url.pathname) continue
      if (route.method !== requestContext.method) continue

      return {
        requestContext,
        responseBody: route.controller(incommingMessage, requestContext),
        closeConnection: true,
      }
    }

    return {
      ...requestContext,
      statusCode: 404,
    }
  }
}

I only modified the routes parameter so that it is no longer an array, but a rest parameter as for createServer.

Thus, we have achieved our goal and ended this article. Appreciate your hard work:

import { createServer } from './lib/http/create-server.js'
import { url, router } from './lib/http/middlewares/index.js'
import { get } from './lib/http/methods/index.js'

const server = createServer(
  url(),
  router(
    get('/', () => `Hello, World!`),
  )
)

server.listen(3000)

If you want to go a little deeper into the concepts seen quickly in this article or go further, I advise you:

• destructuring objects
• rest parameter and spread operator
• arrays
• fat arrow functions