Bash
sort & uniq
sort and uniq commands for text sorting and deduplication.
31 commands
Windows
MacOS
Linux
#text-processing
#data
Basic Sorting
Sort lines alphabetically
sort file.txt
Sort in reverse order
sort -r file.txt
Sort and write to output file
sort -o sorted.txt file.txt
Case-insensitive sort
sort -f file.txt
Sort and remove duplicates
sort -u file.txt
Randomize line order
sort -R file.txt
Numeric & Field Sort
Sort numerically
sort -n numbers.txt
Sort human-readable sizes (1K, 2M)
sort -h sizes.txt
Sort CSV by second field
sort -t ',' -k2 data.csv
Sort CSV by third field numerically
sort -t ',' -k3n data.csv
Sort by field 2, then field 3 numeric
sort -k2,2 -k3,3n file.txt
Sort passwd by UID
sort -t ':' -k3 -n /etc/passwd
Unique Operations
Remove adjacent duplicate lines
uniq file.txt
Remove all duplicates
sort file.txt | uniq
Show only duplicate lines
uniq -d file.txt
Show only unique lines
uniq -u file.txt
Case-insensitive deduplication
uniq -i file.txt
Skip first field when comparing
uniq -f 1 file.txt
Frequency Counting
Count occurrences of each line
sort file.txt | uniq -c
Sort by frequency descending
sort file.txt | uniq -c | sort -rn
Top values in field 1
awk '{print $1}' log | sort | uniq -c | sort -rn
Top IPs
cut -d ' ' -f1 access.log | sort | uniq -c | sort -rn | head
Advanced
Sort IP addresses
sort -t '.' -k1,1n -k2,2n -k3,3n ips.txt
Sort by month name
sort -M dates.txt
Parallel sort for large files
sort --parallel=4 big.txt
Use null byte as line terminator
sort -z file.txt
comm -12 <(sort a.txt) <(sort b.txt) # Find common lines in two files
comm -23 <(sort a.txt) <(sort b.txt) # Lines only in first file
Quick Commands
Count and rank lines by frequency
sort file.txt | uniq -c | sort -rn
Sort CSV file by second field
sort -t ',' -k2 data.csv
Find common lines between two files
comm -12 <(sort a.txt) <(sort b.txt)