Linux Basics: Sorting and Counting Text Lines

A list becomes easier to inspect when repeated entries are grouped and counted. Debian and Ubuntu include sort, uniq, and wc for this work. They read ordinary text files without changing them unless you explicitly save their output.

Run mktemp -d, then use cd to enter the temporary directory it prints before trying these examples.

Prepare a small list

printf 'pear\napple\npear\nbanana\napple\n' > fruit.txt

Each fruit occupies one line. This makes the file suitable for line-based sorting and counting.

Sort and remove duplicates

LC_ALL=C sort fruit.txt
LC_ALL=C sort -u fruit.txt

The first command arranges the lines in byte order. The second also keeps only one copy of each identical line. LC_ALL=C makes the ordering predictable for this command without changing your shell’s saved settings.

Sorting normally depends on your locale. Uppercase letters, accented characters, and punctuation may appear in a different order under another locale.

Count repeated entries

LC_ALL=C sort fruit.txt | uniq -c
wc -l fruit.txt

      2 apple
      1 banana
      2 pear
5 fruit.txt

uniq -c counts adjacent equal lines, so sorting first brings repeated entries together. Running uniq directly on an unsorted file can miss duplicates separated by other lines.

wc -l counts newline characters. Every entry in this fixture ends with a newline. A final unterminated line does not add to that count.

For a file containing one number per line, use sort -n numbers.txt for numeric order. Plain text sorting can put 10 before 2.

References

Debian sort manual, uniq manual, and wc manual

Acknowledgements

This post was created with assistance from AI.

Linux Basics — lesson 8 of 25. Series index

Previous: Pipes and Output Redirection · Next: Selecting Text Fields with cut