HTTP Compression

When a server delivers some messages to a client using an HTTP protocol, data compression may be performed to save bandwidth. It can make the data size becomes smaller with the cost of CPU processes. Node.js has a built-in library to handle compression, as I mentioned in another post. For instance, we will see an HTTP server built on Fastify that utilizes the zlib module to compress data returned by the server module in the following code.

import Fastify, { FastifyInstance } from 'fastify';
import { join } from 'path';
import { createReadStream } from 'fs';
import zlib from 'zlib';

const PORT = 3000;
const fastify: FastifyInstance = Fastify({ logger: true });

fastify.get('/', (request, reply) => {
  // get request header
  const acceptEncoding = request.headers['accept-encoding'] || '';
  const rawStream = createReadStream(join(process.cwd(), 'text.txt'));
  reply.header('Content-Type', 'text/plain');

  // if accepted encoding contains gzip
  if (acceptEncoding.includes('gzip')) {
    // send compressed data
    reply.header('Content-Encoding', 'gzip');
    reply.send(rawStream.pipe(zlib.createGzip()));
  } else {
    // send raw data
    reply.send(rawStream);
  }
});

fastify.listen({ port: PORT }, (err, addr) => {
  if (err) {
    fastify.log.error(err);
  } else {
    console.log(`Server is listening on ${addr}`);
  }
});

In the example above, we first read a text file containing a sentence. Then, in the router handler, we define some tasks.

1. Check the Accept-Encoding header in the request.
2. If the requester accepts the data that will be sent to be compressed, we can pass the raw data into the zlib function.
3. Add the Content-Encoding header in the response if the data is compressed.

We can test the result and count the number of bytes returned using the wc command available in the Unix shell. Firstly, we send requests to the server without any header parameter.

curl http://localhost:3000 | wc -c

Secondly, we set a header for accepting compression.

curl -H 'Accept-Encoding: gzip' http://localhost:3000 | wc -c

The result may be unlike what you are expecting where the first one returns less number of bytes than the second one. It is caused by a compression mechanism that will work effectively if there are some repetitions in the content. Let's update the content of the text file with several similar words, phrases, or sentences. Then, run again the commands with and without the encoding header.

In a real scenario, the compression process should be delegated to a web server or reverse proxy server to lighten up the main application load. Nginx has provided the GZip compression option at its core. The following example shows some parameters that can be used in the HTTP block of the Nginx configuration.

# ...
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_min_length 256;
gzip_types
  font/otf
  font/ttf
  image/svg+xml
  text/css
  text/javascript
  text/plain
# ...