Run MD5 check sum against all files in a directory

Couple snippets that allow us to run checksum and get unique md5 checksums.

This is two step process. First, we obtain our md5 checksum for all files

find -type f -exec md5sum "{}" + > /opt/checklist.chk

This produces file with following contents

71cc452a8ac5a27c32a83e6a0909e7ae  ./PID_190_7344_0_47710322.tif6712032974632727465.tiff
71cc452a8ac5a27c32a83e6a0909e7ae  ./PID_190_7344_0_47710322.tif174464329785828524.tiff
71cc452a8ac5a27c32a83e6a0909e7ae  ./PID_190_7344_0_47710322.tif6775939766281585264.tiff
71cc452a8ac5a27c32a83e6a0909e7ae  ./PID_190_7344_0_47710322.tif7205305688614612348.tiff
71cc452a8ac5a27c32a83e6a0909e7ae  ./PID_190_7344_0_47710322.tif3909999865608008175.tiff

Next we parse and get only unique checksums.

cat  /opt/checklist.chk | awk '{split($0, a, " "); if(!seen[a[1]]++) print a[1]}'

This produces our distinct checksums

71cc452a8ac5a27c32a83e6a0909e7ae