README

Deduplicate files on your APFS file system.

What is this repository for?

• APFS allows cloning files instead of copying them
• This script detects duplicates and replaces them with clones
• More information (clones): https://developer.apple.com/documentation/foundation/file_system/about_apple_file_system

How do I use it?

• Requires python3
• To download wget -O deduplicate.py https://raw.githubusercontent.com/capyvara/apfs-deduplicate/master/deduplicate.py && chmod +x deduplicate.py
• Run ./deduplicate.py -h for help
• To save time, this script will first compile a list of probable duplicates by computing a hash of the first 1024 bytes of data. Of the probable matches, a hash of the full file contents are then computed, and duplicates are replaced with clones via calling cp -c
• Note: This script is considered experimental. Although it has been tested on a number of different data sets (git repositories, RDBMS storage, etc.) it should not be run on sensitive data

Credits

https://bitbucket.org/dchevell/apfs-deduplicate/