Practice/Anthropic/Deduplicate Files

Deduplicate Files

CodingMust

Problem Overview

Given a root folder/directory, find all duplicate files within it. Return groups of file paths where each group contains files with identical content.

Example

For a directory structure like:

root/ ├── a/ │ ├── f1.mp4 │ └── f2.mp4 └── b/ ├── f3.mp4 └── tmp/ └── f4.mp4

Expected output format:

[ ["a/f1.mp4", "a/f2.mp4"], # Group 1: identical content ["b/f3.mp4", "b/tmp/f4.mp4"] # Group 2: identical content ]

Requirements

• Traverse all files in the directory tree starting from the root folder
• Group files by content — files with identical content should be in the same group
• Return only duplicates — files that appear alone (no duplicates) should not be included in the output
• Handle nested directories — recursively search through subdirectories

Important Notes

• You will need to manually create test files in the interview environment (e.g., Google Colab)
• Consider edge cases: empty files, symbolic links, file permissions
• Think about the time complexity: it depends on the total size of all files

Basic Approach

A straightforward solution:

1. Use os.walk() to traverse the directory tree
2. Compute a hash (e.g., MD5, SHA-256) of each file's full content
3. Group files by their hash value
4. Return groups with more than one file

Interface

` def find_duplicate_files(root_path: str) -> List[List[str]]: """ Find all duplicate files in the directory tree.

Args:
    root_path: Root directory to search

Returns:
    List of groups, where each group contains paths to duplicate files
"""
pass

`