File System Operations

File System Operations

Overview

Apply safe path handling and temporary file patterns so reads and writes are reliable, even with untrusted input or large intermediate data.

When to Use

• Use this skill only after the frontmatter triggers; otherwise start with a simple read/​write path.

Decision Tree

1. Normalize or resolve the path?
   • Normalize only: use os.path.normpath() for lexical cleanup.
   • Resolve + check existence: use os.path.realpath(path, strict=True).
2. Are you writing to a target path that may already exist?
   • Yes: write to a temp file in the same directory and replace.
3. Is the file size unknown or potentially large?
   • Yes: use tempfile.SpooledTemporaryFile to avoid unnecessary disk writes.

Workflows

1. Normalize vs Resolve Paths

1. Normalize user input with os.path.normpath() to collapse redundant segments.
2. Resolve real paths with os.path.realpath(path, strict=True) when you must confirm the file exists.
3. Reject paths that fail resolution or escape your allowed base directory.

2. Atomic Replace Write

1. Create a temporary file in the destination directory.
2. Write and flush content to the temp file.
3. Replace the destination with os.replace/os.rename after the write completes.
4. Ensure the temp file and destination are on the same filesystem.

3. Spooled Temporary Buffer

1. Create tempfile.SpooledTemporaryFile(max_size=...).
2. Stream data into the spooled file as you process it.
3. Call .rollover() to force the buffer onto disk when needed.

Non-Obvious Insights

• normpath is lexical only and can change the meaning of a path with symlinks.
• realpath(..., strict=True) raises FileNotFoundError when the path does not exist.
• os.rename is atomic on POSIX when successful; keep temp and destination on the same filesystem.
• Temporary files may not have a visible name; do not rely on name visibility.
• Spooled temp files can be forced to disk with rollover() when a file grows.

Evidence

• "Normalize a pathname by collapsing redundant separators and up-level references so that A//B, A/​B/, A/​./​B and A/​foo/​../​B all become A/​B. This string manipulation may change the meaning of a path that contains symbolic links." - Python Docs
• "FileNotFoundError is raised if path does not exist, or another OSError if it is otherwise inaccessible." - Python Docs
• "If successful, the renaming will be an atomic operation (this is a POSIX requirement)." - Python Docs
• "Rename the file or directory src to dst. If dst exists and is a file, it will be replaced silently if the user has permission. The operation may fail if src and dst are on different filesystems." - Python Docs
• "TemporaryFile, NamedTemporaryFile, TemporaryDirectory, and SpooledTemporaryFile are high-level interfaces which provide automatic cleanup and can be used as context managers." - Python Docs
• "should not rely on a temporary file created using this function having or not having a visible name in the file system." - Python Docs
• "rollover() ... causes the file to roll over to an on-disk file regardless of its size." - Python Docs

Scripts

• scripts/​file-system_tool.py: CLI for resolve, atomic write, copy, and safe reads.
• scripts/​file-system_tool.js: Node.js CLI equivalents.

Dependencies

• Python standard library (os, tempfile, pathlib, shutil) or Node standard library (fs, path, os).

References

• references/​README.md