595 lines
12 KiB
Markdown
595 lines
12 KiB
Markdown
# O-PHONE
|
|
|
|
A Python command-line tool for organising and deduplicating media
|
|
imported from phones and other devices.
|
|
|
|
The program is designed primarily for creating a long-term media
|
|
backup on an external drive.
|
|
|
|
It uses SHA-256 file hashes and a SQLite database to identify files
|
|
that have already been backed up. This allows the same source folder
|
|
to be imported repeatedly without creating duplicate copies.
|
|
|
|
|
|
## WHAT IT DOES
|
|
|
|
|
|
Given a source folder and a destination folder, o-phone.py:
|
|
|
|
1. Searches the source folder recursively.
|
|
|
|
2. Identifies supported image, video and other files.
|
|
|
|
3. Determines the best available date for each file.
|
|
|
|
4. Organises media into a date-based directory structure.
|
|
|
|
5. Renames imported media using a consistent filename format.
|
|
|
|
6. Calculates a SHA-256 hash for every imported file.
|
|
|
|
7. Stores the hash and file information in a SQLite database
|
|
located in the destination folder.
|
|
|
|
8. Checks the database before importing a file.
|
|
|
|
9. Skips files whose hash is already present in the database.
|
|
|
|
10. Verifies files after they have been copied.
|
|
|
|
11. Records the imported file in the database only after the
|
|
copy and verification have succeeded.
|
|
|
|
This makes it safe to run the program again later when new photos
|
|
and videos have been copied from the phone.
|
|
|
|
|
|
## DIRECTORY STRUCTURE
|
|
|
|
The normal destination structure looks like this:
|
|
|
|
phone-media-backup/
|
|
|
|
|
+-- .phone-backup.sqlite
|
|
+-- 2024/
|
|
| +-- January/
|
|
| | +-- 03-01-2024/
|
|
| | | +-- Phone-03-01-2024-482193.jpg
|
|
| | | +-- Phone-03-01-2024-817204.mp4
|
|
| | |
|
|
| | +-- February/
|
|
| |
|
|
| +-- December/
|
|
|
|
|
+-- 2025/
|
|
| +-- January/
|
|
| |
|
|
| +-- ...
|
|
|
|
|
+-- 2026/
|
|
|
|
|
+-- screenshots/
|
|
|
|
|
+-- u-mp3/
|
|
|
|
|
+-- u-xxx/
|
|
|
|
|
|
## MEDIA ORGANISATION
|
|
|
|
Images and videos are organised by:
|
|
|
|
YEAR
|
|
MONTH
|
|
DAY-MONTH-YEAR
|
|
|
|
For example:
|
|
|
|
2026/
|
|
August/
|
|
14-08-2026/
|
|
|
|
The month is written as its full English name rather than a
|
|
numeric month.
|
|
|
|
|
|
FILENAMES
|
|
=========
|
|
|
|
Imported files are renamed to:
|
|
|
|
Phone-DD-MM-YYYY-RANDOM.ext
|
|
|
|
For example:
|
|
|
|
Phone-14-08-2026-483921.jpg
|
|
Phone-14-08-2026-182735.mp4
|
|
|
|
The six-digit number is randomly generated.
|
|
|
|
The number is not used as the duplicate identifier.
|
|
|
|
The SHA-256 hash of the actual file contents is the authoritative
|
|
duplicate identifier.
|
|
|
|
|
|
SCREENSHOTS
|
|
===========
|
|
|
|
PNG files are treated as screenshots.
|
|
|
|
They are placed into:
|
|
|
|
screenshots/
|
|
|
|
The original date information is still retained in the database.
|
|
|
|
Other supported image formats remain in the normal date-based
|
|
directory structure.
|
|
|
|
|
|
UNKNOWN FILE TYPES
|
|
==================
|
|
|
|
Files with formats that are not recognised as normal photo or
|
|
video media are placed into a directory based on their extension.
|
|
|
|
For example:
|
|
|
|
.mp3
|
|
|
|
becomes:
|
|
|
|
u-mp3/
|
|
|
|
A file with extension:
|
|
|
|
.xyz
|
|
|
|
would become:
|
|
|
|
u-xyz/
|
|
|
|
|
|
FILES WITHOUT AN EXTENSION
|
|
==========================
|
|
|
|
Files without a usable extension are handled separately rather
|
|
than being silently discarded.
|
|
|
|
The exact handling depends on the current version of o-phone.py.
|
|
|
|
|
|
DUPLICATE DETECTION
|
|
===================
|
|
|
|
Duplicate detection is based on SHA-256.
|
|
|
|
For each file being imported, o-phone.py calculates:
|
|
|
|
SHA-256(file contents)
|
|
|
|
The resulting hash is compared against hashes stored in:
|
|
|
|
.phone-backup.sqlite
|
|
|
|
If the hash already exists, the file is considered already backed
|
|
up and is skipped.
|
|
|
|
This means duplicate detection is based on the contents of the
|
|
file rather than:
|
|
|
|
- filename
|
|
- folder name
|
|
- file creation date
|
|
- file modification date
|
|
- device name
|
|
|
|
This is important because phones can rename files or give different
|
|
metadata to the same media.
|
|
|
|
|
|
WHY USE A DATABASE?
|
|
===================
|
|
|
|
The SQLite database acts as an index of the backup.
|
|
|
|
Instead of repeatedly scanning every existing file on the external
|
|
drive, o-phone.py can check the database for the SHA-256 hash.
|
|
|
|
Conceptually:
|
|
|
|
Source file
|
|
|
|
|
v
|
|
Calculate SHA-256
|
|
|
|
|
v
|
|
Search SQLite database
|
|
|
|
|
+---- hash exists ----> SKIP
|
|
|
|
|
+---- hash missing ---> COPY
|
|
|
|
|
v
|
|
Verify copy
|
|
|
|
|
v
|
|
Add to database
|
|
|
|
SQLite is built into Python, so no external database server is
|
|
required.
|
|
|
|
|
|
IMPORTANT: DATABASE UPDATE ORDER
|
|
================================
|
|
|
|
The database should only be updated after a successful copy.
|
|
|
|
The intended sequence is:
|
|
|
|
1. Calculate source hash
|
|
2. Check database
|
|
3. If already present, skip
|
|
4. Copy file
|
|
5. Verify destination file
|
|
6. Calculate/verify destination hash
|
|
7. Confirm hashes match
|
|
8. Add file to database
|
|
|
|
This prevents a failed or incomplete copy from being recorded as
|
|
successfully backed up.
|
|
|
|
|
|
RUNNING THE PROGRAM
|
|
===================
|
|
|
|
The basic command is:
|
|
|
|
python3 o-phone.py SOURCE DESTINATION
|
|
|
|
Example:
|
|
|
|
python3 o-phone.py \
|
|
"/Users/yourname/Desktop/phone-backup-moto-g13" \
|
|
"/Volumes/all/phone-media-backup"
|
|
|
|
SOURCE
|
|
------
|
|
|
|
The first argument is the folder containing the files imported from
|
|
your phone.
|
|
|
|
The source folder can contain subdirectories.
|
|
|
|
o-phone.py searches recursively.
|
|
|
|
Example:
|
|
|
|
/Users/yourname/Desktop/phone-backup-moto-g13/
|
|
|
|
|
|
DESTINATION
|
|
-----------
|
|
|
|
The second argument is the root folder of the backup.
|
|
|
|
For example:
|
|
|
|
/Volumes/all/phone-media-backup/
|
|
|
|
The destination can be located on an external hard drive or SSD.
|
|
|
|
|
|
RECOMMENDED WORKFLOW
|
|
====================
|
|
|
|
When new photos are available on the phone:
|
|
|
|
1. Copy the new phone files onto your Mac.
|
|
|
|
2. Put them into a source folder.
|
|
|
|
3. Run o-phone.py using the same destination folder.
|
|
|
|
For example:
|
|
|
|
python3 o-phone.py \
|
|
"/Users/yourname/Desktop/phone-backup-moto-g13" \
|
|
"/Volumes/all/phone-media-backup"
|
|
|
|
The program will calculate hashes for the source files.
|
|
|
|
Files already present in the database will be skipped.
|
|
|
|
New files will be copied into the appropriate destination
|
|
directories.
|
|
|
|
Therefore it is safe to use the same destination repeatedly.
|
|
|
|
|
|
DRY RUN
|
|
=======
|
|
|
|
The program supports a dry-run mode when available in the current
|
|
version.
|
|
|
|
Dry-run mode allows the proposed changes to be inspected without
|
|
actually copying files.
|
|
|
|
Example:
|
|
|
|
python3 o-phone.py SOURCE DESTINATION --dry-run
|
|
|
|
Always check the output before performing a large import.
|
|
|
|
|
|
DATABASE
|
|
========
|
|
|
|
The SQLite database is stored inside the destination directory.
|
|
|
|
Default database:
|
|
|
|
.phone-backup.sqlite
|
|
|
|
It should be treated as part of the backup.
|
|
|
|
If you move the entire backup to another drive, move the database
|
|
with it.
|
|
|
|
Do not delete or manually modify the database unless you understand
|
|
the consequences.
|
|
|
|
|
|
DATABASE INTEGRITY
|
|
==================
|
|
|
|
The database is an index of the files in the backup.
|
|
|
|
Over time it is possible for the database and the actual files to
|
|
become different.
|
|
|
|
For example, this could happen if somebody manually deletes a file
|
|
from the backup drive.
|
|
|
|
A database verification/check function is therefore intended to
|
|
compare:
|
|
|
|
SQLite records
|
|
against
|
|
actual files on disk
|
|
|
|
The check should be used periodically, especially after manually
|
|
changing files on the backup drive.
|
|
|
|
|
|
DO NOT MANUALLY ORGANISE THE BACKUP
|
|
===================================
|
|
|
|
Once files have been imported, it is recommended not to manually
|
|
rename or move them.
|
|
|
|
The database records their paths.
|
|
|
|
Moving files outside of o-phone.py can therefore cause the database
|
|
and the actual backup to become inconsistent.
|
|
|
|
If files need to be reorganised, future versions of the program may
|
|
provide dedicated maintenance commands.
|
|
|
|
|
|
SOURCE FILES ARE NOT DELETED
|
|
============================
|
|
|
|
o-phone.py is intended to be a backup/import tool.
|
|
|
|
It should not delete the original files from the source folder as
|
|
part of a normal import.
|
|
|
|
The source files should remain available until you have independently
|
|
confirmed that the backup is complete and healthy.
|
|
|
|
|
|
FILE DATES
|
|
==========
|
|
|
|
The program attempts to determine the most useful date associated
|
|
with each media file.
|
|
|
|
Depending on the file type, this may come from embedded metadata
|
|
or filesystem information.
|
|
|
|
The exact date source can vary between:
|
|
|
|
- JPEG photos
|
|
- HEIC photos
|
|
- PNG screenshots
|
|
- videos
|
|
- RAW camera files
|
|
- files copied from different phones
|
|
- files whose metadata has been removed
|
|
|
|
When metadata is unavailable, the program uses an appropriate
|
|
fallback.
|
|
|
|
|
|
RAW FILES
|
|
=========
|
|
|
|
RAW camera files are preserved rather than converted.
|
|
|
|
They are treated as source files and copied to the backup.
|
|
|
|
The original RAW data is not modified.
|
|
|
|
|
|
SUPPORTED MEDIA
|
|
===============
|
|
|
|
The exact list of supported extensions is defined in o-phone.py.
|
|
|
|
Typical photo formats include:
|
|
|
|
.jpg
|
|
.jpeg
|
|
.heic
|
|
.heif
|
|
.png
|
|
.webp
|
|
.tif
|
|
.tiff
|
|
RAW formats
|
|
|
|
Typical video formats include:
|
|
|
|
.mp4
|
|
.mov
|
|
.m4v
|
|
.avi
|
|
.mkv
|
|
.3gp
|
|
|
|
The program may recognise additional formats.
|
|
|
|
Unknown formats are not automatically discarded.
|
|
|
|
|
|
REQUIREMENTS
|
|
============
|
|
|
|
macOS or another operating system with Python 3.
|
|
|
|
Python 3.9 or newer is recommended.
|
|
|
|
The standard Python libraries used by the program do not require a
|
|
separate database server.
|
|
|
|
Some optional functionality may require additional Python packages.
|
|
|
|
|
|
INSTALLATION
|
|
============
|
|
|
|
Check Python:
|
|
|
|
python3 --version
|
|
|
|
Run the program directly:
|
|
|
|
python3 o-phone.py SOURCE DESTINATION
|
|
|
|
|
|
SAFETY
|
|
======
|
|
|
|
Before using o-phone.py with an important archive:
|
|
|
|
1. Test it with a small source folder.
|
|
|
|
2. Use --dry-run where available.
|
|
|
|
3. Confirm the destination path carefully.
|
|
|
|
4. Check the generated directory structure.
|
|
|
|
5. Open several copied files.
|
|
|
|
6. Confirm the SQLite database exists.
|
|
|
|
7. Run the database/filesystem integrity check if available.
|
|
|
|
8. Keep another independent backup of important media.
|
|
|
|
|
|
IMPORTANT PRINCIPLE
|
|
===================
|
|
|
|
The SHA-256 hash is the identity of the file.
|
|
|
|
The filename and directory are organisational information.
|
|
|
|
For example, these two files:
|
|
|
|
IMG_1234.JPG
|
|
vacation-photo.jpg
|
|
|
|
may represent exactly the same file contents.
|
|
|
|
If their SHA-256 hashes match, o-phone.py considers them duplicates.
|
|
|
|
Conversely, two files with the same filename are not considered
|
|
duplicates if their contents differ.
|
|
|
|
|
|
PROJECT PHILOSOPHY
|
|
==================
|
|
|
|
The goal of o-phone.py is to provide a simple, dependable and
|
|
repeatable way to build a personal media archive.
|
|
|
|
The important principles are:
|
|
|
|
- Do not delete source files.
|
|
- Do not overwrite existing backup files unnecessarily.
|
|
- Use file contents rather than filenames for duplicate detection.
|
|
- Verify files before recording them as successfully backed up.
|
|
- Keep a local SQLite index.
|
|
- Keep the archive organised by date.
|
|
- Make repeated imports safe.
|
|
- Prefer simple, inspectable storage over proprietary formats.
|
|
|
|
|
|
EXAMPLE
|
|
=======
|
|
|
|
Suppose the phone contains:
|
|
|
|
DCIM/Camera/IMG_1234.jpg
|
|
DCIM/Camera/IMG_1235.jpg
|
|
DCIM/Camera/VID_001.mp4
|
|
Screenshots/screenshot.png
|
|
|
|
Run:
|
|
|
|
python3 o-phone.py \
|
|
"/Users/yourname/Desktop/phone-import" \
|
|
"/Volumes/all/phone-media-backup"
|
|
|
|
The destination might become:
|
|
|
|
phone-media-backup/
|
|
|
|
|
+-- .phone-backup.sqlite
|
|
|
|
|
+-- 2026/
|
|
| |
|
|
| +-- August/
|
|
| |
|
|
| +-- 14-08-2026/
|
|
| |
|
|
| +-- Phone-14-08-2026-483921.jpg
|
|
| +-- Phone-14-08-2026-193827.mp4
|
|
|
|
|
+-- screenshots/
|
|
|
|
|
+-- Phone-14-08-2026-928374.png
|
|
|
|
If the same source files are imported again, their hashes will
|
|
already exist in the database and they will be skipped.
|
|
|
|
|
|
BACKUP REMINDER
|
|
===============
|
|
|
|
A single external drive is not a complete backup strategy.
|
|
|
|
If the media is important, maintain at least one additional,
|
|
independent copy.
|
|
|
|
A RAID array, external drive, or database does not by itself protect
|
|
against every form of data loss.
|
|
|
|
o-phone.py is an organisation and import tool, not a replacement for
|
|
a complete backup strategy.
|