o-phone/README.md

# O-PHONE

A Python command-line tool for organising and deduplicating media
imported from phones and other devices.

The program is designed primarily for creating a long-term media
backup on an external drive.

It uses SHA-256 file hashes and a SQLite database to identify files
that have already been backed up. This allows the same source folder
to be imported repeatedly without creating duplicate copies.


WHAT IT DOES
============

Given a source folder and a destination folder, o-phone.py:

1. Searches the source folder recursively.

2. Identifies supported image, video and other files.

3. Determines the best available date for each file.

4. Organises media into a date-based directory structure.

5. Renames imported media using a consistent filename format.

6. Calculates a SHA-256 hash for every imported file.

7. Stores the hash and file information in a SQLite database
   located in the destination folder.

8. Checks the database before importing a file.

9. Skips files whose hash is already present in the database.

10. Verifies files after they have been copied.

11. Records the imported file in the database only after the
    copy and verification have succeeded.

This makes it safe to run the program again later when new photos
and videos have been copied from the phone.


DIRECTORY STRUCTURE
===================

The normal destination structure looks like this:

phone-media-backup/
|
+-- .phone-backup.sqlite
+-- 2024/
|   +-- January/
|   |   +-- 03-01-2024/
|   |   |   +-- Phone-03-01-2024-482193.jpg
|   |   |   +-- Phone-03-01-2024-817204.mp4
|   |   |
|   |   +-- February/
|   |
|   +-- December/
|
+-- 2025/
|   +-- January/
|   |
|   +-- ...
|
+-- 2026/
|
+-- screenshots/
|
+-- u-mp3/
|
+-- u-xxx/


MEDIA ORGANISATION
==================

Images and videos are organised by:

    YEAR
        MONTH
            DAY-MONTH-YEAR

For example:

    2026/
        August/
            14-08-2026/

The month is written as its full English name rather than a
numeric month.


FILENAMES
=========

Imported files are renamed to:

    Phone-DD-MM-YYYY-RANDOM.ext

For example:

    Phone-14-08-2026-483921.jpg
    Phone-14-08-2026-182735.mp4

The six-digit number is randomly generated.

The number is not used as the duplicate identifier.

The SHA-256 hash of the actual file contents is the authoritative
duplicate identifier.


SCREENSHOTS
===========

PNG files are treated as screenshots.

They are placed into:

    screenshots/

The original date information is still retained in the database.

Other supported image formats remain in the normal date-based
directory structure.


UNKNOWN FILE TYPES
==================

Files with formats that are not recognised as normal photo or
video media are placed into a directory based on their extension.

For example:

    .mp3

becomes:

    u-mp3/

A file with extension:

    .xyz

would become:

    u-xyz/


FILES WITHOUT AN EXTENSION
==========================

Files without a usable extension are handled separately rather
than being silently discarded.

The exact handling depends on the current version of o-phone.py.


DUPLICATE DETECTION
===================

Duplicate detection is based on SHA-256.

For each file being imported, o-phone.py calculates:

    SHA-256(file contents)

The resulting hash is compared against hashes stored in:

    .phone-backup.sqlite

If the hash already exists, the file is considered already backed
up and is skipped.

This means duplicate detection is based on the contents of the
file rather than:

- filename
- folder name
- file creation date
- file modification date
- device name

This is important because phones can rename files or give different
metadata to the same media.


WHY USE A DATABASE?
===================

The SQLite database acts as an index of the backup.

Instead of repeatedly scanning every existing file on the external
drive, o-phone.py can check the database for the SHA-256 hash.

Conceptually:

    Source file
        |
        v
    Calculate SHA-256
        |
        v
    Search SQLite database
        |
        +---- hash exists ----> SKIP
        |
        +---- hash missing ---> COPY
                                  |
                                  v
                            Verify copy
                                  |
                                  v
                            Add to database

SQLite is built into Python, so no external database server is
required.


IMPORTANT: DATABASE UPDATE ORDER
================================

The database should only be updated after a successful copy.

The intended sequence is:

    1. Calculate source hash
    2. Check database
    3. If already present, skip
    4. Copy file
    5. Verify destination file
    6. Calculate/verify destination hash
    7. Confirm hashes match
    8. Add file to database

This prevents a failed or incomplete copy from being recorded as
successfully backed up.


RUNNING THE PROGRAM
===================

The basic command is:

    python3 o-phone.py SOURCE DESTINATION

Example:

    python3 o-phone.py \
        "/Users/yourname/Desktop/phone-backup-moto-g13" \
        "/Volumes/all/phone-media-backup"

SOURCE
------

The first argument is the folder containing the files imported from
your phone.

The source folder can contain subdirectories.

o-phone.py searches recursively.

Example:

    /Users/yourname/Desktop/phone-backup-moto-g13/


DESTINATION
-----------

The second argument is the root folder of the backup.

For example:

    /Volumes/all/phone-media-backup/

The destination can be located on an external hard drive or SSD.


RECOMMENDED WORKFLOW
====================

When new photos are available on the phone:

1. Copy the new phone files onto your Mac.

2. Put them into a source folder.

3. Run o-phone.py using the same destination folder.

For example:

    python3 o-phone.py \
        "/Users/yourname/Desktop/phone-backup-moto-g13" \
        "/Volumes/all/phone-media-backup"

The program will calculate hashes for the source files.

Files already present in the database will be skipped.

New files will be copied into the appropriate destination
directories.

Therefore it is safe to use the same destination repeatedly.


DRY RUN
=======

The program supports a dry-run mode when available in the current
version.

Dry-run mode allows the proposed changes to be inspected without
actually copying files.

Example:

    python3 o-phone.py SOURCE DESTINATION --dry-run

Always check the output before performing a large import.


DATABASE
========

The SQLite database is stored inside the destination directory.

Default database:

    .phone-backup.sqlite

It should be treated as part of the backup.

If you move the entire backup to another drive, move the database
with it.

Do not delete or manually modify the database unless you understand
the consequences.


DATABASE INTEGRITY
==================

The database is an index of the files in the backup.

Over time it is possible for the database and the actual files to
become different.

For example, this could happen if somebody manually deletes a file
from the backup drive.

A database verification/check function is therefore intended to
compare:

    SQLite records
        against
    actual files on disk

The check should be used periodically, especially after manually
changing files on the backup drive.


DO NOT MANUALLY ORGANISE THE BACKUP
===================================

Once files have been imported, it is recommended not to manually
rename or move them.

The database records their paths.

Moving files outside of o-phone.py can therefore cause the database
and the actual backup to become inconsistent.

If files need to be reorganised, future versions of the program may
provide dedicated maintenance commands.


SOURCE FILES ARE NOT DELETED
============================

o-phone.py is intended to be a backup/import tool.

It should not delete the original files from the source folder as
part of a normal import.

The source files should remain available until you have independently
confirmed that the backup is complete and healthy.


FILE DATES
==========

The program attempts to determine the most useful date associated
with each media file.

Depending on the file type, this may come from embedded metadata
or filesystem information.

The exact date source can vary between:

- JPEG photos
- HEIC photos
- PNG screenshots
- videos
- RAW camera files
- files copied from different phones
- files whose metadata has been removed

When metadata is unavailable, the program uses an appropriate
fallback.


RAW FILES
=========

RAW camera files are preserved rather than converted.

They are treated as source files and copied to the backup.

The original RAW data is not modified.


SUPPORTED MEDIA
===============

The exact list of supported extensions is defined in o-phone.py.

Typical photo formats include:

    .jpg
    .jpeg
    .heic
    .heif
    .png
    .webp
    .tif
    .tiff
    RAW formats

Typical video formats include:

    .mp4
    .mov
    .m4v
    .avi
    .mkv
    .3gp

The program may recognise additional formats.

Unknown formats are not automatically discarded.


REQUIREMENTS
============

macOS or another operating system with Python 3.

Python 3.9 or newer is recommended.

The standard Python libraries used by the program do not require a
separate database server.

Some optional functionality may require additional Python packages.


INSTALLATION
============

Check Python:

    python3 --version

Run the program directly:

    python3 o-phone.py SOURCE DESTINATION


SAFETY
======

Before using o-phone.py with an important archive:

1. Test it with a small source folder.

2. Use --dry-run where available.

3. Confirm the destination path carefully.

4. Check the generated directory structure.

5. Open several copied files.

6. Confirm the SQLite database exists.

7. Run the database/filesystem integrity check if available.

8. Keep another independent backup of important media.


IMPORTANT PRINCIPLE
===================

The SHA-256 hash is the identity of the file.

The filename and directory are organisational information.

For example, these two files:

    IMG_1234.JPG
    vacation-photo.jpg

may represent exactly the same file contents.

If their SHA-256 hashes match, o-phone.py considers them duplicates.

Conversely, two files with the same filename are not considered
duplicates if their contents differ.


PROJECT PHILOSOPHY
==================

The goal of o-phone.py is to provide a simple, dependable and
repeatable way to build a personal media archive.

The important principles are:

    - Do not delete source files.
    - Do not overwrite existing backup files unnecessarily.
    - Use file contents rather than filenames for duplicate detection.
    - Verify files before recording them as successfully backed up.
    - Keep a local SQLite index.
    - Keep the archive organised by date.
    - Make repeated imports safe.
    - Prefer simple, inspectable storage over proprietary formats.


EXAMPLE
=======

Suppose the phone contains:

    DCIM/Camera/IMG_1234.jpg
    DCIM/Camera/IMG_1235.jpg
    DCIM/Camera/VID_001.mp4
    Screenshots/screenshot.png

Run:

    python3 o-phone.py \
        "/Users/yourname/Desktop/phone-import" \
        "/Volumes/all/phone-media-backup"

The destination might become:

    phone-media-backup/
    |
    +-- .phone-backup.sqlite
    |
    +-- 2026/
    |   |
    |   +-- August/
    |       |
    |       +-- 14-08-2026/
    |           |
    |           +-- Phone-14-08-2026-483921.jpg
    |           +-- Phone-14-08-2026-193827.mp4
    |
    +-- screenshots/
        |
        +-- Phone-14-08-2026-928374.png

If the same source files are imported again, their hashes will
already exist in the database and they will be skipped.


BACKUP REMINDER
===============

A single external drive is not a complete backup strategy.

If the media is important, maintain at least one additional,
independent copy.

A RAID array, external drive, or database does not by itself protect
against every form of data loss.

o-phone.py is an organisation and import tool, not a replacement for
a complete backup strategy.