Data Handling in Node.js

Both read files, but differ in how the data is delivered — and the source's claim that readFile is synchronous is wrong, so correct it:

• fs.readFile (and fs.promises.readFile) is asynchronous and non-blocking, but it reads the entire file into memory at once, returning a Buffer/string when done. Simple, but a large file can spike memory or exceed buffer limits. (The truly blocking variant is fs.readFileSync.)
• fs.createReadStream returns a readable stream that delivers the file in chunks as they're read. Memory stays roughly constant regardless of file size, and you can start processing/piping before the whole file is loaded.

const data = await fs.promises.readFile('small.json', 'utf8');   // whole file
fs.createReadStream('huge.csv').pipe(parser);                    // chunked

The rule interviewers want: use readFile for small files you need fully in memory (config, small JSON); use createReadStream for large files or when piping (serving downloads, processing CSV/logs), since it's memory-efficient and supports backpressure. Both are non-blocking — the difference is all-at-once vs streamed.

Node handles data through a few core abstractions, and a strong answer names them:

• Buffers — fixed-length raw binary data (Buffer), used for bytes from files, sockets, crypto, etc.
• Streams — process data incrementally in chunks rather than all at once (readable, writable, duplex, transform). Essential for large data and I/O efficiency, with built-in backpressure.
• JSON — JSON.parse/stringify for the most common structured-data format.
• Encodings — TextEncoder/TextDecoder and string encodings (utf8, base64) for converting between bytes and text.

import { pipeline } from 'node:stream/promises';
await pipeline(fs.createReadStream('in.csv'), transform, fs.createWriteStream('out.csv'));

The framing: Node is built for I/O-heavy data movement, so it favors streaming (constant memory, start processing early) over loading everything into memory. A current note: modern Node also implements the Web Streams API (ReadableStream/WritableStream) alongside its classic streams, plus global fetch, Blob, and structuredClone — so data handling increasingly aligns with web platform standards.

Streams are abstractions for processing data piece by piece (in chunks) as it's read or written, instead of buffering the whole payload in memory. They're central to Node because so much of what it does is move data (files, network, HTTP). The four types:

• Readable (fs.createReadStream, request bodies), Writable (fs.createWriteStream, responses), Duplex (sockets), Transform (gzip, crypto, parsers).

Why they matter:

• Memory efficiency — process a 10 GB file with a few MB of RAM.
• Time efficiency — start working on early chunks before the rest arrives (lower latency).
• Composability — chain steps with pipe/pipeline (read → transform → write).
• Backpressure — built-in flow control prevents a fast producer from overwhelming a slow consumer.

import { pipeline } from 'node:stream/promises';
await pipeline(
  fs.createReadStream('in.txt'),
  zlib.createGzip(),
  fs.createWriteStream('in.txt.gz')
);

The best-practice point: prefer pipeline() (from node:stream/promises) over manual .pipe(), because it propagates errors and cleans up all streams automatically — a raw .pipe() chain leaks resources on error. Modern Node also supports the standards-based Web Streams API.

JSON is a first-class citizen in Node (it's JavaScript), handled with the global JSON object:

• JSON.parse(text) — string → JavaScript object.
• JSON.stringify(value[, replacer, space]) — object → JSON string (with optional pretty-printing).

const obj = JSON.parse('{"id":1}');
const text = JSON.stringify(obj, null, 2);

Practical points interviewers reward:

• Always wrap JSON.parse in try/catch — malformed input throws SyntaxError.
• Reading JSON files: JSON.parse(await fs.readFile('data.json', 'utf8')); you can also import data from './data.json' with { type: 'json' } (import attributes) or require('./data.json') in CommonJS.
• In web frameworks, request bodies are parsed for you (express.json() / Fastify's built-in parser), and you send JSON with res.json(...).
• For very large JSON, JSON.parse loads everything into memory and can block the loop — use a streaming parser instead.
• stringify drops undefined/functions and throws on circular references (use a custom replacer or structuredClone-safe data).

The built-in fs ("file system") module is Node's API for interacting with files and directories: reading, writing, appending, deleting, renaming, watching, creating directories, and inspecting metadata/permissions (stat, chmod).

It comes in three flavors, and naming them signals competence:

• fs/promises — promise-based; the modern default with async/await.
• Callback API — classic error-first callbacks (fs.readFile(path, cb)).
• Sync API — fs.readFileSync, etc., which block the event loop — fine at startup, avoid in request handlers.

import { readFile, writeFile } from 'node:fs/promises';
const cfg = JSON.parse(await readFile('config.json', 'utf8'));
await writeFile('log.txt', 'done\n', { flag: 'a' });

Its role in data handling: load configuration, write logs/exports, manage uploaded files, and—crucially—provide createReadStream/createWriteStream for streaming large files with constant memory. Best practices to mention: prefer the promise API, avoid *Sync on hot paths, use the node: prefix, handle errors (missing files throw ENOENT), and use path/path.join for cross-platform paths. Note modern Node can also restrict fs access via the permission model.

A Buffer is a fixed-length chunk of raw binary data stored outside the V8 heap. It exists because core JavaScript strings can't safely represent arbitrary bytes, and Node constantly deals with binary I/O — file contents, TCP sockets, streams, crypto, image/compression data.

const buf = Buffer.from('hello', 'utf8');
console.log(buf.length, buf.toString('hex'));      // 5  68656c6c6f
const empty = Buffer.alloc(16);                    // zero-filled, safe

Points interviewers want: use Buffer.alloc(n) (zero-filled, safe) rather than the deprecated/unsafe new Buffer() or allocUnsafe (which can expose old memory) unless you immediately overwrite it; Buffers are returned by fs.readFile when you don't specify an encoding, and stream chunks are Buffers by default; and you convert with .toString(encoding) / Buffer.from(str, encoding) (utf8, base64, hex). Buffer is a subclass of Uint8Array, so it interoperates with the standard TypedArray/Web APIs — which is why modern Node code increasingly uses Uint8Array/TextEncoder alongside Buffers.