Practice/Google/Design a Simple File System Metadata

Design a Simple File System Metadata

System DesignMust

Problem Statement

Design a metadata service that stores and manages file system metadata — the directory hierarchy, file attributes, and path-to-inode mappings — without handling actual file content. Think of it as the layer that powers operations like ls, stat, mkdir, and mv in a system like ext4 or Google Drive's backend.

The core challenge is modeling hierarchical data (directories containing files and subdirectories) in a way that supports fast path lookups, atomic renames and moves, and concurrent access from many clients. You need to handle millions of files organized in deeply nested trees while keeping metadata operations consistent and performant.

Your design should support creating, reading, updating, and deleting file and directory metadata, resolving full paths efficiently, and handling concurrent modifications without corruption or lost updates.

Key Requirements

Functional

• CRUD operations -- Create, read, update, and delete metadata for files and directories, including attributes like size, permissions, timestamps, and owner.
• Hierarchical directory structure -- Support arbitrarily nested directories with parent-child relationships and efficient traversal.
• Path resolution -- Resolve a full path like /home/user/docs/report.txt to its metadata entry quickly.
• Atomic rename and move -- Move or rename a file or directory atomically, even when the operation crosses directory boundaries.

Non-Functional

• Scalability -- Handle hundreds of millions of metadata entries across the file system.
• Latency -- Path resolution and metadata lookups complete in under 10ms at p99.
• Consistency -- All operations reflect a consistent view; no partial moves or orphaned entries.
• Durability -- Metadata must survive node failures without data loss.

What Interviewers Focus On

Based on real interview experiences, these are the areas interviewers probe most deeply:

1. Hierarchical Data Modeling

How you represent the tree structure in a relational or key-value store determines the efficiency of every operation. Hints to consider:

• Consider an inode-based model where each entry has a unique ID and a parent ID
• Think about adjacency list vs. materialized path vs. nested set tradeoffs
• How do you list all children of a directory without scanning the entire table?
• What index strategy makes lookup(parent_id, name) fast?

2. Path Resolution Performance

Walking a deep path component-by-component can be expensive if each step requires a database query. Hints to consider:

• Consider caching resolved path prefixes in Redis with appropriate TTL
• How do you invalidate cached paths when a directory is renamed or moved?
• Think about batching multiple path components into a single query
• What happens when the cache is cold — how do you prevent thundering herd?

3. Atomic Rename and Move

Moving a directory subtree is deceptively complex when metadata is distributed or when concurrent operations target the same tree. Hints to consider:

• How do you prevent cycles when moving a directory into one of its own descendants?
• Consider using advisory locks or optimistic concurrency control for cross-directory moves
• What transaction isolation level do you need to guarantee atomicity?
• How does ZooKeeper or a similar coordination service help with distributed locking?