121: Filesystem, Paths, URLs, OS APIs, File Watching, and Safe File Handling

Learning objectives

You will learn to:

use node:fs/promises;
understand files, directories, metadata, permissions, and file descriptors at a practical level;
construct portable paths with node:path;
convert between filesystem paths and file URLs;
distinguish module directory and current working directory;
stream large files instead of buffering them completely;
perform safe temporary/atomic-like file workflows;
understand file watching limitations;
avoid path traversal vulnerabilities;
handle user-uploaded filenames safely.

Filesystem code looks straightforward until the surrounding assumptions change. A relative path can resolve differently when a service is started by a process manager, a file can change between a check and the operation that follows it, and an uploaded filename can be an attack payload rather than a harmless label. This lesson builds the practical model needed to use Node's filesystem APIs without overlooking those boundaries.

Promise filesystem API

For application code that already uses async and await, the Promise-based filesystem API is generally the clearest fit:

import { readFile, writeFile } from 'node:fs/promises';

const text = await readFile('./data.txt', 'utf8');

await writeFile('./copy.txt', text, 'utf8');

The first call reads the file and resolves to text because an encoding was supplied. The second writes that text to a different file. Use Promise APIs for modern async application code; they allow filesystem work to participate naturally in async functions and ordinary error handling.

Synchronous filesystem operations still have appropriate uses. Startup code that must finish before the application accepts requests, build scripts, and small command-line tools may reasonably use them. They become a poor default in request-handling code because the operation blocks the event loop while the filesystem responds.

Encoding matters

Without an encoding, readFile() returns the bytes in a Buffer:

const data = await readFile('./data.txt');

console.log(Buffer.isBuffer(data)); // true

That is the right representation when the content is binary or when another API needs the original bytes. If the file is text, ask Node to decode it:

await readFile('./data.txt', 'utf8');

The result is then a string. This distinction matters because decoding arbitrary binary data as UTF-8 can change or corrupt the data. Treat a file as text only when its format and encoding justify that choice.

Buffers are covered deeply in lesson 122.

File size

readFile() buffers the entire file in memory before it resolves. That is convenient for a small configuration file, but the same approach does not scale with file size:

For a 5 GB upload, this is unacceptable.

Use a stream when the consumer can process the file incrementally:

import { createReadStream } from 'node:fs';

const stream = createReadStream('./large.bin');

The stream keeps the application from needing one 5 GB in-memory value. The later stream lesson explains backpressure, which is the mechanism that prevents a fast producer from overwhelming a slower consumer. File size is therefore a design decision, not merely an implementation detail.

File metadata

Sometimes the application needs information about a filesystem entry in addition to its contents:

import { stat } from 'node:fs/promises';

const info = await stat('./data.txt');

console.log({
  size: info.size,
  file: info.isFile(),
  directory: info.isDirectory(),
  mtime: info.mtime,
});

This gives you the size, predicates that describe the entry type, and the modification time. It is useful for reporting, logging, display, and some application decisions.

Do not use metadata checked long before access as a security guarantee. Filesystem state can change between the check and the use. That check-then-use gap can produce a race condition, including a time-of-check/time-of-use security issue. Where correctness or security depends on the operation, prefer an operation whose flags and error behavior enforce the requirement directly.

Directories

Directory APIs let you create a hierarchy and inspect its immediate entries:

import { mkdir, readdir } from 'node:fs/promises';

await mkdir('./data/tasks', {
  recursive: true,
});

const entries = await readdir('./data/tasks', {
  withFileTypes: true,
});

for (const entry of entries) {
  console.log(entry.name, entry.isDirectory());
}

recursive: true means the parent directories are created when they do not already exist. With withFileTypes: true, each returned entry is a Dirent, so the code can ask whether it represents a directory without immediately making another metadata call. The example lists the directory's direct children; it is not a recursive walk.

Delete/rename/copy

Node exposes the common mutation operations through the Promise API as well:

import {
  rename,
  rm,
  copyFile,
} from 'node:fs/promises';

Deletion is the operation that deserves the most deliberate review. Be explicit with recursive destructive operations, and validate target paths before rm({ recursive: true }). A path derived from configuration or user input should never be allowed to turn a cleanup operation into deletion of an unrelated directory.

rename and copyFile have different behavior and failure modes, so choose based on whether the original should remain and on the filesystem guarantees the workflow needs.

`node:path`

The path module handles platform-specific separators and path rules. Use it to construct a portable path:

import path from 'node:path';

const file = path.join('data', 'tasks', '123.json');

Do not build filesystem paths by concatenating /:

'data/' + userInput

Separators and platform behavior are harder to reason about that way, and the security properties of the result are easy to misunderstand. path.join() makes composition clearer, but it is not by itself an authorization check; untrusted segments still need validation and containment checks.

Resolve

Use path.resolve() when you need an absolute path:

const absolute = path.resolve('data', 'tasks');

Resolution is relative to:

process.cwd()

unless an absolute segment is supplied. process.cwd() is the process's current working directory, which is determined by how the process was launched. It is not necessarily the directory containing the JavaScript module. This is why code that works when launched from a project root can fail when a service manager, test runner, or deployment script starts it from somewhere else.

For a path anchored to the module itself, use the module URL instead:

const file = new URL('./data/config.json', import.meta.url);

Many Node filesystem APIs accept file URLs. This can be an elegant ESM pattern when a resource belongs beside the module rather than beside the caller's current working directory.

File URLs

import.meta.url gives an ESM module's location as a URL. A relative URL can therefore identify a file next to that module:

const url = new URL('./template.html', import.meta.url);

const html = await readFile(url, 'utf8');

When a library requires a path string instead of a file URL, convert it with the URL API:

import { fileURLToPath } from 'node:url';

const filename = fileURLToPath(url);

Do not manually strip file://. URL encoding, Windows drive letters, and spaces make manual conversion incorrect. The conversion function handles those filesystem-specific details and gives the receiving library the representation it expects.

Path normalization is not authorization

A normalized path can be syntactically valid and still point outside the directory the application intended to expose. Suppose a server serves files under:

/srv/uploads

The user supplies:

../../etc/passwd

This direct join is dangerous:

const target = path.join('/srv/uploads', userPath);
return readFile(target);

The application needs to enforce containment, not merely construct a path:

const root = path.resolve('/srv/uploads');
const target = path.resolve(root, userPath);

const relative = path.relative(root, target);

if (
  relative.startsWith('..') ||
  path.isAbsolute(relative)
) {
  throw new Error('Path escapes upload root');
}

The resolved target is compared with the resolved root. A relative result beginning with .., or an absolute relative result, indicates that the target is outside the root. This protects the lexical path case shown here.

There is one further subtlety: symlinks can make the filesystem location differ from the lexical path. Also consider symlinks and the actual threat model. For security-sensitive file serving, use carefully reviewed abstractions and platform controls rather than assuming this small helper addresses every filesystem race or link scenario.

User filenames

An uploaded filename is input data, not a trusted filesystem path. Do not trust values such as:

../../report.pdf
C:\Windows\...
<script>.html
very-long-name
confusable Unicode

These values can create traversal, collision, display, or filesystem portability problems. For stored uploads:

generate server-side storage key;
retain original name as metadata if needed;
validate length/type;
avoid using original name as path authority.

For example, the storage key can be unrelated to the supplied name:

const storageName = crypto.randomUUID();

The application can retain the user-facing name as metadata while using the generated name for storage:

{
  originalName,
  storageName,
  mediaType,
  size
}

That separation prevents presentation data from deciding where the server reads or writes a file. It also gives the application a stable collision-resistant identifier while preserving the original name when it is useful to show or download later.

Extension is not content type

A filename extension is a claim made by the name. It is not proof of the bytes inside the file. A file named:

image.jpg

may not contain JPEG data.

Server-side upload validation should consider:

size;
claimed MIME;
content signature/magic bytes where relevant;
malware/scanning policy;
safe serving headers.

The claimed MIME type and extension can be compared with detected content, but neither should be treated as the only defense. The acceptable checks depend on the application and its threat model. Do not execute uploaded files. Store and serve them under controls that prevent them from being interpreted as application code.

Atomic replacement pattern

Writing directly to a configuration or data file can leave a partially written file if the process stops during the write. A common replacement pattern is:

write temporary file
fsync if durability requirement
rename temp → final

The temporary file is written separately, and the final name is replaced only after the temporary content is ready. Rename within the same filesystem is commonly atomic at filesystem level, but exact durability semantics depend on OS/filesystem. Atomic visibility and durable persistence are related but different guarantees.

A simple pattern looks like this:

const temp = `${target}.${process.pid}.tmp`;

await writeFile(temp, JSON.stringify(data));

await rename(temp, target);

This example reduces the chance that readers observe an incomplete JSON document. It is not a complete durability protocol: production durability may require stronger fsync/directory handling, and the temporary name also needs suitable uniqueness and cleanup behavior for the application's concurrency model.

For a database, do not reinvent transaction storage with JSON files. Databases provide concurrency, recovery, and transaction semantics that a small rename pattern does not replace.

Exclusive create

Some workflows need creation to succeed only when no file already exists. The wx flag requests that behavior:

await writeFile(path, data, {
  flag: 'wx',
});

The operation fails if the file exists. This is safer than first checking existsSync() and then writing, because the check and write would leave a race window.

Filesystem locking across processes is subtle. Use dedicated libraries/OS facilities where correctness matters. A flag that gives exclusive creation is useful for some lock or create-once workflows, but it does not automatically solve all coordination, stale-lock, or multi-process lifecycle problems.

File descriptors

Higher-level functions such as readFile() often open and close the underlying resource for you. Low-level code that explicitly opens a handle owns the cleanup:

const handle = await open(file, 'r');

try {
  ...
} finally {
  await handle.close();
}

Always close resources, including on error. Higher-level APIs often manage descriptors for you, but code using open() or related low-level APIs must make the lifetime explicit. Resource leaks can exhaust file descriptors and eventually prevent the process from opening files, sockets, or other resources.

File permissions

Unix-like systems represent access with permission bits. For a file containing a secret, an application can request a restrictive mode at creation time:

await writeFile(secretPath, secret, {
  mode: 0o600,
});

This mode gives the owner read and write access while withholding those permissions from group and other users, subject to platform behavior and process privileges. Platform semantics vary, so do not assume the same mode has identical effects everywhere.

Application permissions should complement—not replace—OS/container access controls. A Node option cannot compensate for an overly privileged process, a writable host mount, or an insecure deployment configuration.

Temporary directory

The operating system provides a location intended for temporary data:

import os from 'node:os';
import path from 'node:path';

const tempRoot = os.tmpdir();

Create unique temporary directories/files safely rather than predictable shared names. Predictable names create collisions and can create opportunities for another process to interfere with a temporary file. Node filesystem APIs provide temporary directory helpers; use them where the workflow needs a safely created unique location.

Clean up resources, including temporary files and directories, on both successful and failed paths where possible. Cleanup should not hide the original failure, so error handling needs to account for both operations.

OS APIs

The os module exposes runtime information that can help with diagnostics and capacity decisions:

import os from 'node:os';

console.log({
  platform: os.platform(),
  arch: os.arch(),
  cpus: os.availableParallelism?.() ?? os.cpus().length,
  totalMemory: os.totalmem(),
  freeMemory: os.freemem(),
});

This reports the platform, architecture, an estimate of available parallelism, and memory figures. It is useful context for logs and operational investigation.

Do not use machine CPU count blindly to size database connection pools or worker concurrency. Container CPU quotas and workload constraints matter. The number reported by the host may not equal the capacity your process should consume, and database limits, latency targets, and memory pressure may impose stricter limits.

Watching files

Node supports file watching, but watching is an observation mechanism rather than a durable record of every change. For modern development, Node can restart a process when its files change:

node --watch src/server.js

This is useful for process restart during development.

fs.watch() can observe filesystem changes directly:

import { watch } from 'node:fs';

const watcher = watch('./config', (eventType, filename) => {
  console.log(eventType, filename);
});

The callback reports an event type and, where the platform provides it, a filename. File watching is platform-dependent. Real applications need to account for:

duplicate events;
missing filename in cases;
rename semantics;
network filesystems;
editor atomic-save behavior.

An editor may save by writing a new file and renaming it, rather than modifying the original in place. Other environments may coalesce events or report them differently. Do not use fs.watch as guaranteed durable business event delivery. If every event matters, use a durable queue or another system designed to retain and acknowledge events.

Chokidar

For richer cross-platform development/watch behavior, Chokidar is a common package. It can smooth over some platform differences and provide more convenient higher-level behavior.

Again: filesystem watch is not a message queue. A watcher can help a development tool notice that it should rescan or restart, but it should not be the source of truth for durable business processing.

Globbing

Modern Node and ecosystem packages can match file patterns. Packages include glob, globby.

Use globbing for:

build scripts;
tooling;
file discovery.

Do not pass untrusted glob patterns into privileged filesystem scans without constraining scope. A pattern can select far more files than intended, and a privileged process may expose or modify data outside the intended project area. Apply both pattern and directory boundaries before performing sensitive operations.

File server example

This helper illustrates the same containment check in a small file-serving scenario:

async function loadPublicAsset(relativePath) {
  const root = path.resolve('./public');
  const target = path.resolve(root, relativePath);
  const rel = path.relative(root, target);

  if (rel.startsWith('..') || path.isAbsolute(rel)) {
    throw new Error('Invalid path');
  }

  return readFile(target);
}

The input is resolved beneath the public root, then compared with that root before reading. This addresses lexical traversal such as ../, but production code must still consider symlinks, error handling, content types, caching, and safe response headers. For production static hosting, a mature server/CDN is usually preferable to a hand-written implementation. Such systems have already dealt with many of the edge cases around efficient delivery and file serving.

Failure clinic

These are common symptoms of the assumptions discussed above.

`readFile` huge file

Memory spike.

The likely cause is whole-file buffering. Inspect the file size and memory profile, then switch the workflow to a stream or another incremental interface.

Relative path assumes cwd

Works locally, fails under service manager.

The process was probably launched with a different current working directory. Inspect process.cwd() at runtime and decide whether the resource should instead be anchored to import.meta.url or to an explicitly configured absolute root.

Original upload filename used as storage path

Traversal/collision risk.

Treat the original name as metadata and generate a server-side storage key. Validate the upload independently of the name.

`existsSync` then write

Race condition:

check
another process writes
you write

Use atomic operation/flags. In particular, use an operation such as exclusive creation when the requirement is "create only if absent," rather than relying on a separate existence check.

File watcher used as reliable queue

Events can coalesce/drop/duplicate.

Inspect the watcher as a hint that work may be needed, then rescan or use a durable event source as appropriate. Do not assume one callback corresponds exactly to one durable business event.

Exercises

Read/write JSON with fs/promises.
Build module-relative path using import.meta.url.
Compare cwd-relative behavior by launching from another directory.
Build a safe path-containment helper and test ../.
Implement temp-write + rename.
Store an upload with generated server filename.
Watch a directory and record editor-save event behavior.
Stream a 1 GB file without loading it into memory.

Work through the exercises in order where possible. The early tasks establish the APIs; the later tasks force you to observe launch context, race-resistant writes, untrusted input, editor behavior, and memory use in a real process.

Mastery checklist

Explain:

buffered read versus stream;
cwd versus module location;
path versus file URL;
traversal;
upload filenames;
atomic replacement;
descriptors;
permissions;
watch limitations;
OS metadata.

Being able to explain these distinctions is a useful final check because each one corresponds to a failure mode: excessive memory, deployment-dependent paths, incorrect URL conversion, unauthorized file access, unsafe storage, partial writes, resource leaks, overly broad access, unreliable notifications, or incorrect capacity assumptions.