No More Manual Copying.

Tired of the time-consuming task of manually sorting and copying files?
Say goodbye to the hassle with this powerful combination of find, cp, and rsync. This approach allows you to automate the transfer of .png, .jpg, .jpeg, .mov, and .mp4 files from your local system directly to a backup location or a remote server. By leveraging these tools, you can streamline your file management, ensuring that your data is efficiently and securely transferred without manual intervention.

Case Study

Our goal is to automate the transfer of specific file types, such as .png, .jpg, .jpeg, .mov, and .mp4, either within local directories or to a remote server. This approach aims to eliminate the tedious task of manually copying files to different locations, ensuring a more efficient, streamlined, and secure transfer process.

Let's dive into this directory.

$ ls -l ~/SNUBMONKEY/

total 23760
drwx------ 351  snubmonkey  snubmonkey    11232 Aug 30 02:12 Archive
drwx------  253 snubmonkey  snubmonkey     8096 Jan 17  2024 Attachments
drwx------    2 snubmonkey  snubmonkey       64 Dec 23  2018 Cloud
drwx------    2 snubmonkey  snubmonkey       64 May 26 15:14 Sticker
-rw-r--r--    1 snubmonkey  snubmonkey  8065024 Sep  9 12:42 monkey1.db
-rw-r--r--    1 snubmonkey  snubmonkey    32768 Sep  7 14:50 monkey2.db
-rw-r--r--    1 snubmonkey  snubmonkey  2949952 Sep  9 14:24 monkey3.db

As you can see there are four (4) directories within. (drwx-)

Counting Files:

When managing directories with many files and subdirectories, it can be helpful to visualize the hierarchy and get a quick count of the total number of files. The tree command provides a graphical representation of the directory structure, while wc -l counts the lines of output, giving you an overview of the number of files and directories. 

$ tree ~/SNUBMONKEY/ | wc -l

2775

Let's it break down.

1. tree SNUBMONKEY:
• This command displays the directory structure of the SNUBMONKEY directory in a tree-like format.
• It shows all files and subdirectories, providing a visual hierarchy.

2. | (Pipe):
• This operator takes the output of the tree command and passes it as input to the next command.

3.wc -l:
• The wc (word count) command with the -l option counts the number of lines in the input.
• In this case, it counts the number of lines output by the tree command, which corresponds to the number of items listed (including both files and directories).

When combined as tree /home/SNUBMONKEY | wc -l , the command counts the total number of directories and files within the SNUBMONKEY directory. Here it resulted in 2,775 which indicates the total number of items, including all types of file extensions ranging from .txt to .mov.

Copying to Local Location

To find all .png, .jpg, .jpeg, .mov, and .mp4 files in a directory and copy them to a new location, you can use the find command with -exec to execute the cp (copy) command for each file found to another directory.

$ find ~/SNUBMONKEY/ -type f \( -iname "*.png" -o -iname "*.jpg" -o -iname "*.jpeg" -o -iname "*.mov" -o -iname "*.mp4" \) -exec cp -v {} ~/downloads/final_sorting/ \;

Let's it break down.

• find ~/SNUBMOMNKEY/ : Starts searching in the specified directory.
• -type f: Finds files only.
• \( -iname ".png" -o -iname ".jpg" -o -iname ".jpeg" -o -iname ".mov" -o -iname "*.mp4" \): Searches for files with .png, .jpg, or .jpeg extensions.
• -exec cp -v {} ~/downloads/final_sorting/: Executes the cp command to copy each file found to the ~/downloads/final_sorting/ directory -v for verbose output.
• \; : Terminates the -exec action.

Important Note:
Ensure that the destination directory ~/downloads/final_sorting/ exists before running the command.
Create it using:

$ mkdir -p ~/downloads/final_sorting/

Copying over secure SSH with rsync

Now; let's securely transfer those specific file types from your local machine to a remote server using SSH and rsync.
First; we want to cd into the source path to avoid any mismatch path result caused by the conjunction of both find and rsync this is important!

$ cd ~/SNUBMONKEY/

then

$ find . -type f \( -iname "*.png" -o -iname "*.jpg" -o -iname "*.jpeg" -o -iname "*.mov" -o -iname "*.mp4" \) -print0 | rsync -av -e ssh --no-relative --files-from=- --from0 . SNUBmonkey@192.168.1.xxx:~/downloads/rsync-final/

or chaining both with && command as such:

$ cd ~/Library/Messages && find . -type f \( -iname "*.png" -o -iname "*.jpg" -o -iname "*.jpeg" -o -iname "*.mov" -o -iname "*.mp4" \) -print0 | rsync -av --no-relative --files-from=- --from0 . xxxx@192.168.1.xxx:~/downloads/rsync-final/

Let's it break down.

• find . : Start searching in the current directory and its subdirectories.
• -type f: Look for files only.
• \( and \): Group multiple file type conditions together.
• -iname "*.png" -o -iname "*.jpg" -o -iname "*.jpeg" -o -iname "*.mov" -o -iname "*.mp4": Match files with the specified extensions. The -o operator stands for "or."
• -print0: Outputs filenames followed by a null character, handling file names with spaces or special characters.
• |: Pipes the output of find to rsync.

rsync -av -e ssh --no-relative --files-from=- --from0 . SNUBmonkey@192.168.1.xxx:~/downloads/rsync-final/

• rsync: The command used to transfer files.
• -av: -a for archive mode (preserves file attributes) and -v for verbose output.
• -e ssh: Specifies that rsync should use SSH for secure communication since rsync by itself is not secure.
• --no-relative: Prevents preserving directory structure relative to the source directory.
• --files-from=-: Reads the list of files to transfer from standard input (piped from find).
•--from0: Indicates that the file list is delimited by null characters.
• .: Source directory.
• SNUBmonkey@192.168.1.xxx:~/downloads/rsync-final/: Destination path on the remote server, with SNUBmonkey as the username and 192.168.1.xxx as the server’s IP address.

*Of course; replace the username and IP address accordingly!




As you have seen, this streamlined approach simplifies your file management, guaranteeing secure and effortless transfers to a backup location or remote server, ultimately saving you time and effort.


We hope you found this tutorial both informative and enjoyable to follow.