Adventures of a Software Engineer

Tag: awk

  • Run MD5 check sum against all files in a directory

    Couple snippets that allow us to run checksum and get unique md5 checksums.

    This is two step process. First, we obtain our md5 checksum for all files

    find -type f -exec md5sum "{}" + > /opt/checklist.chk

    This produces file with following contents

    71cc452a8ac5a27c32a83e6a0909e7ae  ./PID_190_7344_0_47710322.tif6712032974632727465.tiff
71cc452a8ac5a27c32a83e6a0909e7ae  ./PID_190_7344_0_47710322.tif174464329785828524.tiff
71cc452a8ac5a27c32a83e6a0909e7ae  ./PID_190_7344_0_47710322.tif6775939766281585264.tiff
71cc452a8ac5a27c32a83e6a0909e7ae  ./PID_190_7344_0_47710322.tif7205305688614612348.tiff
71cc452a8ac5a27c32a83e6a0909e7ae  ./PID_190_7344_0_47710322.tif3909999865608008175.tiff

    Next we parse and get only unique checksums.

    cat  /opt/checklist.chk | awk '{split($0, a, " "); if(!seen[a[1]]++) print a[1]}'

    This produces our distinct checksums

    71cc452a8ac5a27c32a83e6a0909e7ae