The Linux Tail Command is often a go to tool when troubleshooting a Linux system or many other devices such as VMware ESXi hosts or Linux/Unix based appliances.
The tail command allows us to output the last lines written to a log file, which gives us a quick way to check what the last events written to the file were. This is very useful when troubleshooting an issue in real time, as it lets us see what has just happened or, as we will see, can let us watch the log in real time.
Tail Command Syntax
Running the tail command without any additional arguments will display the last ten lines of a text or log file. For example, this is using the tail command to list the last 10 lines of the dpkg.log file on an Ubuntu Linux system:
$ tail dpkg.log
2022-02-01 15:43:42 status installed mime-support:all 3.64ubuntu1
2022-02-01 15:43:42 trigproc hicolor-icon-theme:all 0.17-2 <none>
2022-02-01 15:43:42 status half-configured hicolor-icon-theme:all 0.17-2
2022-02-01 15:43:45 status installed hicolor-icon-theme:all 0.17-2
2022-02-01 15:43:45 trigproc gnome-menus:amd64 3.36.0-1ubuntu1 <none>
2022-02-01 15:43:45 status half-configured gnome-menus:amd64 3.36.0-1ubuntu1
2022-02-01 15:43:45 status installed gnome-menus:amd64 3.36.0-1ubuntu1
2022-02-01 15:43:45 trigproc man-db:amd64 2.9.1-1 <none>
2022-02-01 15:43:45 status half-configured man-db:amd64 2.9.1-1
2022-02-01 15:43:46 status installed man-db:amd64 2.9.1-1Depending on what you are doing, this may be too much information or it might be too little. Although tail defaults to displaying 10 lines, we can change the number of lines tail outputs by using the tail -n option or flag. For example, if we only wanted to see the last 4 lines of the dpkg.log file:
$ tail -n 4 dpkg.log
2022-02-01 15:43:45 status installed gnome-menus:amd64 3.36.0-1ubuntu1
2022-02-01 15:43:45 trigproc man-db:amd64 2.9.1-1 <none>
2022-02-01 15:43:45 status half-configured man-db:amd64 2.9.1-1
2022-02-01 15:43:46 status installed man-db:amd64 2.9.1-1How to use the Tail Command on Multiple Files
When you are troubleshooting an issue, often you will want to check multiple files so that you can cross reference information and timestamps from different log files. You can read multiple log files with the tail command by specifying more than one input file. In the example below, we will again output the end of the dpkg.log file, but this time will also output the end of the cloud-init.log file:
$ tail -n 4 dpkg.log cloud-init.log
==> dpkg.log <==
2022-02-07 17:19:20 status half-configured linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:20 status half-installed linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:34 status config-files linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:34 startup packages configure
==> cloud-init.log <==
2022-02-07 17:07:54,190 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
2022-02-07 17:07:54,190 - util.py[DEBUG]: Read 12 bytes from /proc/uptime
2022-02-07 17:07:54,190 - util.py[DEBUG]: cloud-init mode 'modules' took 0.637 seconds (0.64)
2022-02-07 17:07:54,190 - handlers.py[DEBUG]: finish: modules-final: SUCCESS: running modules for finalThis time tail has shown the output from the two log files in the same output. Note that it also displays the log file name in the header for each section. As a quick aside, you can display the file information during any tail command by using the -v option:
$ tail -n 4 -v dpkg.log
==> dpkg.log <==
2022-02-07 17:19:20 status half-configured linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:20 status half-installed linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:34 status config-files linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:34 startup packages configureNow, it might be that you don’t want the file names included in the output. In this case, the tail -q or quiet option can be used:
$ tail -n 4 -q dpkg.log cloud-init.log
2022-02-07 17:19:20 status half-configured linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:20 status half-installed linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:34 status config-files linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:34 startup packages configure
2022-02-07 17:07:54,190 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
2022-02-07 17:07:54,190 - util.py[DEBUG]: Read 12 bytes from /proc/uptime
2022-02-07 17:07:54,190 - util.py[DEBUG]: cloud-init mode 'modules' took 0.637 seconds (0.64)
2022-02-07 17:07:54,190 - handlers.py[DEBUG]: finish: modules-final: SUCCESS: running modules for finalThis time we still have the output from both log files, but it is listed together.
Combining Tail with the Sort Command
If we wanted we could send this output to the sort command, to sort the list of entries from both log files by the timestamp of the log entry:
$ tail -n 4 -q dpkg.log cloud-init.log | sort
2022-02-07 17:07:54,190 - handlers.py[DEBUG]: finish: modules-final: SUCCESS: running modules for final
2022-02-07 17:07:54,190 - util.py[DEBUG]: Read 12 bytes from /proc/uptime
2022-02-07 17:07:54,190 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
2022-02-07 17:07:54,190 - util.py[DEBUG]: cloud-init mode 'modules' took 0.637 seconds (0.64)
2022-02-07 17:19:20 status half-configured linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:20 status half-installed linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1
2022-02-07 17:19:34 startup packages configure
2022-02-07 17:19:34 status config-files linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1This can be very useful if you are pulling in information from different logs and want to see what happened in what order, without having to keep switching between different log files.
View Files in Real Time using the Tail Command
When troubleshooting an on-going issue, it’s often useful to view log files in real time. The tail command allows this with the tail -f option. The ‘f’ is for follow, which makes sense, as we are following the log as new entries are written to it:
$ tail -f dpkg.log
2022-02-07 17:07:54,190 - handlers.py[DEBUG]: finish: modules-final: SUCCESS: running modules for final
2022-02-07 17:07:54,190 - util.py[DEBUG]: Read 12 bytes from /proc/uptime
2022-02-07 17:07:54,190 - util.py[DEBUG]: Reading from /proc/uptime (quiet=False)
2022-02-07 17:07:54,190 - util.py[DEBUG]: cloud-init mode 'modules' took 0.637 seconds (0.64)
2022-02-07 17:19:20 status half-configured linux-image-5.11.0-1022-aws:amd64 5.11.0-1022.23~20.04.1If we use a capital F instead – e.g. tail -F, then this will include the tail –retry option as well. This has the effect of following a log file even if the file we are watching rotates, such as with many log files like syslog.
When viewing the log in real time it can be difficult to follow. To make it a little easier we can pipe the tail -f command to grep, to search for a particular word or phrase in the log file:
tail -f syslog | grep 'Timed out'
As you can see, when the log file is piped to grep in real time, words we are looking for – ‘Timed out’ – stand out in the log file as they have been colourised.
It’s also possible to view multiple files in real time using tail, for example:
$ tail -f syslog kern.logAs before, a file header will be displayed so that you can easily see which log file the entries are from.
Summary
In this article you have learned a few things about the Linux tail command. We have looked at the default behaviour of the tail command and how to change that so that tail will display a specified number of lines.
We then had a look at how to use tail to display multiple files and how to use the grep command and sort command with tail to get a more useful output – very useful when troubleshooting!
Finally, we had a look at how you can use the tail command to view a file or multiple files in real time, and even filter them in real time with help from the grep command.