Compression & encoding in nodejs

The zlib module in Node.js provides compression and decompression functionality using Gzip, Deflate/Inflate, and Brotli. It is built on top of the native Zlib library and is essential for reducing the size of data sent over networks or stored on disk. 1. Why Use Zlib? Bandwidth Efficiency: Compressing HTTP responses (like JSON or HTML) can reduce data transfer sizes by up to 80%, making your app feel faster. Storage Savings: Reducing the footprint of log files or archived data. Stream Integration: Because zlib is implemented using Node.js Transform Streams , it can compress data "on the fly" without loading the whole file into memory. 2. Main Compression Types  Algorithm   Node.js Method  Description  Gzip  createGzip()  The most common format for web assets and file compression.  Deflate  createDeflate()  A standard compression algorithm; the basis for Gzip and ZIP files.  Brotli  createBrotliCompress()  A newer algorithm (by Google) that often achieves better compression ratios than Gzip,  especially for text. 3. Using Zlib with Streams The most efficient way to use zlib is by piping a readable stream through a zlib transform stream and into a writable destination. const zlib = require('zlib'); const fs = require('fs'); const { pipeline } = require('stream'); const gzip = zlib.createGzip(); const source = fs.createReadStream('input.txt'); const destination = fs.createWriteStream('input.txt.gz'); // Pipeline handles errors and automatic cleanup pipeline(source, gzip, destination, (err) => {   if (err) {     console.error('An error occurred:', err);     process.exitCode = 1;   }   console.log('File successfully compressed!'); }); 4. HTTP Compression (The Most Common Use Case) When building a server, you can check the user's Accept-Encoding header to see if their browser supports compression (like gzip or br for Brotli). const http = require('http'); const zlib = require('zlib'); http.createServer((req, res) => {   const rawData = "This is a very long string that needs to be compressed...";   const acceptEncoding = req.headers['accept-encoding'] || '';   if (acceptEncoding.includes('gzip')) {     res.writeHead(200, { 'Content-Encoding': 'gzip' });     zlib.gzip(rawData, (err, buffer) => {       res.end(buffer);     });   } else {     res.end(rawData);   } }).listen(3000); 5. Convenience Methods (Shortcuts) If you aren't working with streams and have the entire data in a buffer, you can use these asynchronous "one-shot" methods: zlib.gzip(data, callback) / zlib.gunzip(data, callback) zlib.deflate(data, callback) / zlib.inflate(data, callback) zlib.brotliCompress(data, callback) There are also synchronous versions (e.g., zlib.gzipSync ), but as we discussed before, avoid these in server environments as they block the event loop. 6. Key Settings: Flush and Strategy You can pass an options object to the compression methods to fine-tune performance: level : Ranges from 1 (Fastest, least compression) to 9 (Slowest, best compression). Default is 6 . memLevel : Controls how much memory is allocated for the internal compression state.