u/This_Is_Mo

Hi everyone,

So I finally built my Unraid server and in the process of moving 270TB from my Mac DAS to Unraid. I have faced a few hiccups but used Gemini to help with rsync copy commands. I wanted to double check that running with these flags is ok? I need to later read the files from macOS over SMB and locally through Unraid.

Disclaimer: I am good at following directions, but I am not in anyway close to being a power user.

Illegal characters SMB does not like can cause the rsync copy to break and unmount the share from macOS. In order to avoid that Gemini suggested I add some SMB extras in Unraid and to also run rsync with certain flags.

caffeinate -s /opt/homebrew/bin/rsync -rlvptDhP --no-links --append-verify --fsync /Path/To/Source/ /Path/To/Destination/

Additionally, I was running rsync without the --append-verify and --fsync, and I was only getting around 135 mb/s transfer speed on average, but with the command above I am hitting up to 240 mb/s, however mostly hovering around 175 mb/s (according to rsync not according to Unraid, with the latter being very liberal in its disk write speed reporting). So noticeable benefit to using the flags in addition to avoiding the illegal characters issue with SMB.

Quick rundown:
Unraid v 7.3
Copying to array directly
Split as required
Highwater
I don't have parity assigned (yet)
No cache

I also wanted to be notified when the transfer starts and finish or gets interrupted. Email is obviously the easiest way. So this is the entire command including the email:

EMAIL_TO="YOUR_EMAIL"
EMAIL_FROM="YOUR_EMAIL"
SMTP_SERVER="smtps://smtp.gmail.com:465"
SMTP_PASS="CREATE_APP_SPECIFIC_PASSWORD_THROUGH_PROVIDER"
LOG_FILE="$HOME/rsync_transfer_errors.txt"
rm -f "$LOG_FILE"
curl -s --url "$SMTP_SERVER" --ssl-reqd --mail-from "$EMAIL_TO" --mail-rcpt "$EMAIL_TO" --user "$EMAIL_FROM:$SMTP_PASS" -T &lt;(echo -e "Subject: Rsync v3 Media Sync Resumed\n\nThe rsync operation has started.")
caffeinate -s /opt/homebrew/bin/rsync -rlvptDhP --no-links --append-verify --fsync /PATH/TO/SOURCE/ /PATH/TO/DESTINATION/ 2&gt; "$LOG_FILE"
EXIT_CODE=$?
if [ -s "$LOG_FILE" ]; then
    EMAIL_BODY=$(cat "$LOG_FILE" | head -n 50)
    curl -s --url "$SMTP_SERVER" --ssl-reqd --mail-from "$EMAIL_TO" --mail-rcpt "$EMAIL_TO" --user "$EMAIL_FROM:$SMTP_PASS" -T &lt;(echo -e "Subject: ⚠️ Rsync Transfer Completed WITH ERRORS\n\nThe transfer finished, but errors were detected. Full log saved at $LOG_FILE.\n\nHere are the first few errors:\n\n$EMAIL_BODY")
else
    if [ $EXIT_CODE -eq 0 ]; then
        curl -s --url "$SMTP_SERVER" --ssl-reqd --mail-from "$EMAIL_TO" --mail-rcpt "$EMAIL_TO" --user "$EMAIL_FROM:$SMTP_PASS" -T &lt;(echo -e "Subject: Rsync Transfer SUCCESSFUL 🎉\n\nGreat news! The upgraded rsync process completed flawlessly with zero errors.")
    else
        curl -s --url "$SMTP_SERVER" --ssl-reqd --mail-from "$EMAIL_TO" --mail-rcpt "$EMAIL_TO" --user "$EMAIL_FROM:$SMTP_PASS" -T &lt;(echo -e "Subject: ALERT: Rsync Transfer Interrupted ❌\n\nThe rsync transfer exited prematurely with error code: $EXIT_CODE.")
    fi
fi

TLDR; Is the rsync copy command above fine to run on massive data migration from macOS to Unraid over SMB and later read it from macOS over SMB or from Unraid?

macOS Data Migration