3. Pipelines and filters

1. Using the pipe operator "|" (vertical bar), the standard output (stdout) of a command can be piped to the standard input (stdin) of another command. This is a powerful feature that allows us to perform complex operations on data by combining simple utilities. Let's see some examples:

   ls -l /usr/bin

   ls -l /usr/bin | less

2. We can sort the data with sort:

   ls /bin /usr/bin

   ls /bin /usr/bin | sort | less

3. uniq can omit or report repeated lines:

   ls /bin /usr/bin | sort | uniq | less

   If we want to see the list of duplicates instead we can use the option -d:

   ls /bin /usr/bin | sort | uniq -d | less

4. wc counts the lines, words, and bytes of the input:

   wc ls-output.txt

   cat ls-output.txt | wc

   If we want it to show only the lines we can use the option -l:

   ls /bin /usr/bin | sort | wc -l

   ls /bin /usr/bin | sort -u | wc -l

   ls /bin /usr/bin | sort | uniq -d | wc -l

5. grep prints the lines that match a given pattern:

   ls /bin /usr/bin | sort -u | grep zip

   ls /bin /usr/bin | sort -u | grep zip | wc -l

   The option -v shows the lines that do not match the pattern:

   ls /bin /usr/bin | sort -u | grep -v zip

   ls /bin /usr/bin | sort -u | grep -v zip | wc -l

   The option -i can be used if we want grep to ignore case when searching (case in-sensitive search).

6. head / tail print the top or the last lines of input:

   ls /usr/bin > ls-output.txt

   head ls-output.txt

   tail ls-output.txt

   tail -n 5 ls-output.txt

   tail -5 ls-output.txt

   ls /usr/bin | head -n 5

   tail /var/log/dpkg.log -n 20

   tail /var/log/bootstrap.log -f

   The option -f makes it follow the latest changes of the file in real time. Press "Ctrl-c" to terminate it.

7. tee sends its input both to stdout and to files:

   ls /usr/bin | tee ls.txt | grep zip

   ls -l ls.txt

   less ls.txt

Download lesson04/part3.cast

Loading asciinema cast...