Next: , Previous: uniq -d, Up: Examples


4.16 Remove All Duplicated Lines

This script prints only unique lines, like ‘uniq -u’.

     #!/usr/bin/sed -f
     
     # Search for a duplicate line --- until that, print what you find.
     $b
     N
     /^\(.*\)\n\1$/ ! {
         P
         D
     }
     
     :c
     # Got two equal lines in pattern space.  At the
     # end of the file we simply exit
     $d
     
     # Else, we keep reading lines with N until we
     # find a different one
     s/.*\n//
     N
     /^\(.*\)\n\1$/ {
         bc
     }
     
     # Remove the last instance of the duplicate line
     # and go back to the top
     D