lsof Command

lsof command examples

What is lsof?


lsof is a command that is used to display "open files" and the process that is responsible for opening the file. With no parameters passed, "lsof" will list all open files and associated process currently running. lsof can be used to identify which ports are currently open and by what processes, list all network connections, list users and processes who are currently accessing directories or files and identify processes that are utilising space within a filesystem. For a full list of all the options and functions that are available to the "lsof" command, please consult the man pages.

Below are a some useful examples of the "lsof" command and how to install lsof on Ubuntu/RHEL/Fedora systems. The lsof examples below were all carried out on a "CentOS 7.2" system.




Howto Install lsof on Ubuntu/Debian systems


Generally lsof is installed by default on most Ubuntu and Debian based systems (MATE, Mint etc..) If you need to install "lsof", you can issue the commands below:



sudo apt-get update

sudo apt-get install lsof

lsof -v

The "sudo apt-get update" command will update your repositories. The "sudo apt-get install lsof" command will install the lsof package. To verify the version after installation, you may issue the "lsof -v" command.


Howto Install lsof on RHEL/CentOS systems


By default on a minimal server build of RHEL, "lsof" is not installed, so you will need to install this package with the commands below. If you want to check to see if "lsof" has already been installed, you can issue the "yum info lsof" command:



yum info lsof

yum install lsof

lsof -v

Howto Install lsof on Fedora systems


Generally "lsof" comes as standard with Fedora 24, however, it can be easily installed with the commands below if it is not present:



dnf info lsof

dnf install lsof

lsof -v


lsof Command Examples



List all Open Files


Below is a small extract of the output from the command "lsof" running on a standard "CentOS 7.2" system. (Please note for clarity, many lines have been omitted).



[root@centos72m ~]# lsof
COMMAND    PID  TID    USER   FD      TYPE             DEVICE  SIZE/OFF       NODE NAME
systemd      1         root  cwd       DIR              253,0      4096        128 /
systemd      1         root  rtd       DIR              253,0      4096        128 /
systemd      1         root  txt       REG              253,1   1498184    4640738 /usr/lib/systemd/systemd
systemd      1         root  mem       REG              253,1     20032    4226787 /usr/lib64/libuuid.so.1.3.0
systemd      1         root    1u      CHR                1,3       0t0       4674 /dev/null
systemd      1         root    2u      CHR                1,3       0t0       4674 /dev/null
systemd      1         root    3u  a_inode                0,9         0       4670 [timerfd]
systemd      1         root    4u  a_inode                0,9         0       4670 [eventpoll]
lsof      1428         root    0u      CHR              136,0       0t0          3 /dev/pts/0
lsof      1428         root    1u      CHR              136,0       0t0          3 /dev/pts/0
lsof      1428         root    2u      CHR              136,0       0t0          3 /dev/pts/0
lsof      1428         root    3r      DIR                0,3         0          1 /proc
lsof      1428         root    4r      DIR                0,3         0      18136 /proc/1428/fd
lsof      1428         root    5w     FIFO                0,8       0t0      18141 pipe

By default 1 file per line is displayed. The output is organised into columns. Below are some of the frequently used fields that you will need to know about:


lsof Column Headers


COMMAND- Name of Process running
PID - Process Identification Number
USER - Process Owner

FD - Field Descriptor
   cwd - Current Working Directory
   rtd - Root Directory
   txt - Text File
   mem - Memory Mapped File
   mmap - Memory Mapped Device

Number - Represents the file Descriptor. After the number a character is used to indicate the mode to which the file was opened:

examples:

   1u - u = Read Access and Write Access
   1r - r = Read Access
   1w - w = Write Access

TYPE - Type - Specifies file type such as: REG (regular file), DIR (Directory, FIFO (First in First Out), CHR (Character Special File)


Display a Process or User using a Mount Point


Quite often as an Administrator you will need to unmount a filesystem if its no longer needed or for maintenance purposes. If you issue the standard "umount command", you may encounter the error/warning indicating that the current target or mount point can not be unmounted as its busy (currently in use by a process/user):

umount: /test: target is busy.

In the following example, we have a user called "john" who is currently accessing the mount point "/test". We will try to unmount this filesystem:



[root@centos72m /]# umount /test
umount: /test: target is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1))

As the error suggests, the mount point is busy (in use or being accessed). However, we can use the "lsof" command to identify who or what is using this mount point:



[root@centos72m /]# lsof /test
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
bash    1532 john  cwd    DIR  253,7        6  128 /test

From the above we can see that we have a user "john" who is accessing the "/test" area. We can see from the output that this is indicated by the DIR, NAME and cwd entries. We can also see that the user is using the "bash" shell to access this area. The process Identification (PID) is also indicated.

So for the administrator to unmount this file system, they will need the user to come out of that particular area or kill the identified PID/Process. Once the user has moved out of this mount point, you should now be able to unmount the area without issue.


List all processes with open files under a specified directory


In the following example, we use the lsof command to identify all process that are accessing the specified directory structure and sub-directories. The "+D causes lsof to search for all open instances of the specified directory and its sub-directories.



[root@centos72m /]# lsof +D /var/log/
COMMAND    PID   USER   FD   TYPE DEVICE SIZE/OFF    NODE NAME
systemd-j  479   root  mem    REG  253,5  8388608 6311998 /var/log/journal/1d65a007ef3549d7bd93a7030b7c50a6/system.journal
systemd-j  479   root   20u   REG  253,5  8388608 6311998 /var/log/journal/1d65a007ef3549d7bd93a7030b7c50a6/system.journal
auditd     659   root    4w   REG  253,5  2994544 4239523 /var/log/audit/audit.log
rsyslogd   690   root  mem    REG  253,5  8388608 6311998 /var/log/journal/1d65a007e

From the above output we can see which processes or users are accessing the specified path and its sub-directories.


List all files in use by a specified user


To list all files that are currently open for a specified user. We can pass the "-u parameter followed by the "userid.

In the example below, we can see all files that are open by the user "john." If you wish to exclude a particular user from the output, you may modify the command to be: lsof -u ^john.



[root@centos72m /]# lsof -u john
COMMAND  PID USER   FD   TYPE DEVICE  SIZE/OFF    NODE NAME
bash    1532 john  cwd    DIR  253,7         6     128 /test
bash    1532 john  rtd    DIR  253,0      4096     128 /
bash    1532 john  txt    REG  253,1    960392   54866 /usr/bin/bash


List all open files by a specific process ID


The "-p" parameter when passed is used to specify a particular process. In the example below, we are only outputting information that is associated with the Process (PID) "1591":



[root@centos72m /]# lsof -p 1591
COMMAND  PID USER   FD   TYPE DEVICE  SIZE/OFF    NODE NAME
bash    1591 john  cwd    DIR  253,4        79     131 /home/john
bash    1591 john  rtd    DIR  253,0      4096     128 /
bash    1591 john  txt    REG  253,1    960392   54866 /usr/bin/bash

Howto automatically repeat a lsof command


A useful option that can be used in conjunction with the "lsof" command is the "-r" parameter followed by a time duration. If no time duration is passed, then the default of 15 seconds will be used. This parameter when specified instructs lsof to repeat the command at the specified time interval. In the example below we are repeating the command every 5 seconds.

The output is divided by the following line separator "=======



[root@centos72m /]# lsof -p 1591 -r5
COMMAND  PID USER   FD   TYPE DEVICE  SIZE/OFF    NODE NAME
bash    1591 john  cwd    DIR  253,4        79     131 /home/john
bash    1591 john  rtd    DIR  253,0      4096     128 /
bash    1591 john  txt    REG  253,1    960392   54866 /usr/bin/bash
=======
COMMAND  PID USER   FD   TYPE DEVICE  SIZE/OFF    NODE NAME
bash    1591 john  cwd    DIR  253,4        79     131 /home/john
bash    1591 john  rtd    DIR  253,0      4096     128 /
bash    1591 john  txt    REG  253,1    960392   54866 /usr/bin/bash

Listing Network Connections with lsof


To list all network connections on your system you can issue the lsof command with the "-i" parameter. If you want to see only TCP connections, you can use "lsof -i tcp" and for "UDP" connection you can issue the command "lsof -i udp"



[root@centos72m /]# lsof -i
COMMAND  PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sshd    1266   root    3u  IPv4  16882      0t0  TCP *:ssh (LISTEN)
sshd    1266   root    4u  IPv6  16891      0t0  TCP *:ssh (LISTEN)
httpd   1269   root    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1345 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1346 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1347 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1348 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1349 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
master  1351   root   13u  IPv4  17362      0t0  TCP localhost:smtp (LISTEN)
master  1351   root   14u  IPv6  17363      0t0  TCP localhost:smtp (LISTEN)
sshd    1378   root    3u  IPv4  17706      0t0  TCP centos72m:ssh->192.168.0.23:37698 (ESTABLISHED)

In the above example we can see all our active connections along with their protocol and name.


Listing connections on a specific port


To display a specific port and look for connections/processes, you may issue the command "lsof -i:port_number.

In the example below, we have specified port 22. This is the default port for ssh connections. We can see from the output that the ssh daemon is listening on port 22 for connections and that a connection has been established from the remote IP address of "192.168.0.23"



[root@centos72m /]# lsof -i:22
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sshd    1266 root    3u  IPv4  16882      0t0  TCP *:ssh (LISTEN)
sshd    1266 root    4u  IPv6  16891      0t0  TCP *:ssh (LISTEN)
sshd    1378 root    3u  IPv4  17706      0t0  TCP centos72m:ssh->192.168.0.23:37698 (ESTABLISHED)

You also issue the command specifying a process to look for. In this example, we have specified the string "ssh" and "httpd":



[root@centos72m /]# lsof -i -a -c ssh
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sshd    1266 root    3u  IPv4  16882      0t0  TCP *:ssh (LISTEN)
sshd    1266 root    4u  IPv6  16891      0t0  TCP *:ssh (LISTEN)
sshd    1378 root    3u  IPv4  17706      0t0  TCP centos72m:ssh->192.168.0.23:37698 (ESTABLISHED)

[root@centos72m /]# lsof -i -a -c http
COMMAND  PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
httpd   1269   root    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1708 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1709 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)

Listing Connections on a range of ports


To specify a range of ports, we can specify the range of ports to check. In the example below, we are checking for all connections on ports 22 through to 25.



[root@centos72m /]# lsof -i:22-25
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sshd    1266 root    3u  IPv4  16882      0t0  TCP *:ssh (LISTEN)
sshd    1266 root    4u  IPv6  16891      0t0  TCP *:ssh (LISTEN)
master  1351 root   13u  IPv4  17362      0t0  TCP localhost:smtp (LISTEN)
master  1351 root   14u  IPv6  17363      0t0  TCP localhost:smtp (LISTEN)
sshd    1378 root    3u  IPv4  17706      0t0  TCP centos72m:ssh->192.168.0.23:37698 (ESTABLISHED)

Limiting output to IPv4 or IPv6


To limit the output from the lsof command to either IPv4 or IPv6, you can pass the parameter "-i4" for IPv4 and "-i6" for IPv6 only.



[root@centos72m /]# lsof -i4
COMMAND  PID USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sshd    1266 root    3u  IPv4  16882      0t0  TCP *:ssh (LISTEN)
master  1351 root   13u  IPv4  17362      0t0  TCP localhost:smtp (LISTEN)
sshd    1378 root    3u  IPv4  17706      0t0  TCP centos72m:ssh->192.168.0.23:37698 (ESTABLISHED)

[root@centos72m /]# lsof -i6
COMMAND  PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
sshd    1266   root    4u  IPv6  16891      0t0  TCP *:ssh (LISTEN)
httpd   1269   root    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1345 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1346 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1347 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1348 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
httpd   1349 apache    4u  IPv6  17198      0t0  TCP *:http (LISTEN)
master  1351   root   14u  IPv6  17363      0t0  TCP localhost:smtp (LISTEN)

Identify processes not releasing space


On occasion, you may notice that the "df" command reports that you have less space available within a filesystem. When you look for files that are using space using conventional commands like "ls" and "du", you will find that there are no listed files consuming space. You cannot visibly see any files that are consuming space. The "lsof" command can help here as it will allow you to identify unlinked files that are holding onto space.

A technique often used within programming is to create a temporary file and then unlink the file. This coding technique will leave the file and its available space available to the executing program. Sometimes programmes do not exit cleanly and can leave these unlinked files holding onto space. To determine if you have a process holding onto an unlinked file, you can issue the following command: lsof -a +L1 /your/filesystem

From the output, you should look for any processes that have a zero "0" value under the NLINK column. To release the space occupied by these processes, you will need to kill (terminate) the process that has been identified under the PID column. The amount of space can be identified under the SIZE heading column. Once the process has been killed, the space will now become usable and will be correctly reported by the "df" command.

Below is an example of output from the lsof command displaying the unlinked files:



[root@wsaps01a dev]# lsof -a +L1 /websphere/v855/

COMMAND   PID     USER   FD   TYPE DEVICE    SIZE/OFF NLINK    NODE NAME
java    20002 wassvc    229r   REG  253,4           0     0  264712 /websphere/v855/AppServer/profiles/WSS1/logs/server1/http_error.log (deleted)
java    20002 wassvc    473r   REG  253,4 15753828154     0 1062986 /websphere/v855/AppServer/profiles/WSS1/jdbc_error.log (deleted)
java    20002 wassvc    486r   REG  253,4 15753828154     0 1062986 /websphere/v855/AppServer/profiles/WSS1/jdbc_error.log (deleted)