Every time we invoke a command, one or more processes are started. Linux must keep track of lots of different programs, all running at the same time. A skilled or well trained system administrator not only needs to create processes, but also be able to keep track of them and send them different types of signals if and when required.

In this guide, we will look at job control and how to monitor processes on Linux systems.

Monitoring Active Processes

At any given time lots of active programs are running on the Linux system. Linux calls each running program a process each process is given a process ID (PID) and manages how the process uses memory and CPU time based on that PID.

When it comes to process monitoring, two invaluable tools used are top and ps commands;

Monitoring Processes with top

Launch your terminal with Ctrl+Alt+T and type top:

$ top
top - 00:28:13 up 14:53,  1 user,  load average: 0.25, 0.45, 0.55
Tasks: 232 total,   1 running, 230 sleeping,   1 stopped,   0 zombie
%Cpu(s):  3.4 us,  2.0 sy,  0.0 ni, 94.3 id,  0.3 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :   7861.2 total,   4570.1 free,   1449.7 used,   1841.4 buff/cache
MiB Swap:   2048.0 total,   2048.0 free,      0.0 used.   5912.0 avail Mem 

    PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                                                                          
   2093 frank     20   0  743228  51064  28268 S   4.0   0.6  35:55.56 Xorg                                                                                             
   2216 frank     20   0   49964  14016  10828 S   2.3   0.2  18:43.58 i3bar                                                                                            
  12586 frank     20   0 2666072 131276  79368 S   1.3   1.6   8:14.45 FoxitReader                                                                                      
   1350 mysql     20   0 2003068 358316  34404 S   0.7   4.5   3:35.22 mysqld                                                                                           
   2010 frank      9 -11  696116  18308  14088 S   0.7   0.2   4:37.87 pulseaudio                                                                                       
     11 root      rt   0       0      0      0 S   0.3   0.0   0:03.92 migration/0                                                                                      
    296 root      19  -1   82608  41384  39640 S   0.3   0.5   0:08.78 systemd-journal                                                                                  
   1155 root      20   0 1062724  10924   9512 S   0.3   0.1   1:26.12 teamviewerd                                                                                      
   2210 frank     20   0   21160   6232   5540 S   0.3   0.1   1:51.93 compton                                                                                          
   2218 frank     20   0    2504   1840   1656 S   0.3   0.0   1:48.26 i3blocks                                                                                         
   3048 frank     20   0  828396  83372  44628 S   0.3   1.0   0:30.43 terminator                                                                                       
   4312 frank     20   0  342592  89000  62580 S   0.3   1.1   2:39.50 chrome                                                                                           
   8076 frank     20   0 4827320 288404 103744 S   0.3   3.6  21:53.20 chrome                                                                                           
1759295 root      20   0       0      0      0 I   0.3   0.0   0:00.88 kworker/u16:0-events_power_efficient                                                             

top allows user interaction. By default, the output is sorted by the percentage of CPU time used by each process in descending order. This behavior can be modified by pressing the following keys from within top:

M: Sort by memory usage.

N: Sort by process ID number.

T: Sort by running time.

P: Sort by percentage of CPU usage.

Note, To switch between descending/ascending order just press R.

Other important keys to interact with top are:

? or h: Help.

k: Kill a process. top will ask for the PID of the process to be killed as well as for the signal to be sent (by default SIGTERM or 15).

r: Change the priority of a process (renice). top will ask you for the nice value. Possible values range from -20 through 19, but only the superuser (root) can set it to a value which is negative or lower than the current one.

u: List processes from a particular user (by default processes from all users are shown).

c: Show programs’ absolute paths and differentiate between userspace processes and kernelspace processes (in square brackets).

V: Forest/hierarchy view of processes.

t and m: Change the look of CPU and memory readings respectively in a four-stage cycle: the first two presses show progress bars, the third hides the bar and the fourth brings it back.

W: Save configuration settings to ~/.toprc.

Detailed output of top

The output of the top is divided into two areas: the summary area and the task area.

The Summary Area in top

It is made up of the the five top rows and gives us the following information:

1.top – 00:28:13 up 14:53, 1 user, load average: 0.25, 0.45, 0.55

  • current time (in 24-hour format): 00:28:13
  • uptime (how much time the system has been up and running):up 14:53
  • number of users logged in:1 user
  • load average of the CPU for the last 1, 5 and 15 minutes, respectively: load average: 0.25, 0.45, 0.55

2. Tasks: 232 total, 1 running, 230 sleeping, 1 stopped, 0 zombie

  • total number of processes in active mode: 232 total
  • running (those being executed): 1 running
  • sleeping (those waiting to resume execution): 230 sleeping
  • stopped (by a job control signal): 1 stopped
  • zombie (those which have completed execution but are still waiting for their parent process to remove them from the process table): 0 zombie

3. %Cpu(s): 3.4 us, 2.0 sy, 0.0 ni, 94.3 id, 0.3 wa, 0.0 hi, 0.0 si, 0.0 st

  • user processes: 3.4 us
  • system/kernel processes: 2.0 sy
  • processes set to a nice value — the nicer the value, the lower the priority: 0,0 ni
  • nothing — idle CPU time: 94.3 id
  • processes waiting for I/O operations: 0.3 wa
  • processes serving hardware interrupts — peripherals sending the processor signals that require attention: 0,0 hi
  • processes serving software interrupts: 0,0 si
  • processes serving other virtual machine’s tasks in a virtual environment, hence steal time: 0,0 st

4. MiB Mem : 7861.2 total, 4570.1 free, 1449.7 used, 1841.4 buff/cache

  • the total amount of memory: 7861.2 total
  • unused memory: 4570.1 free
  • memory in use: 1449.7 used
  • the memory which is buffered and cached to avoid excessive disk access: 1841.4 buff/cache

5. MiB Swap: 2048.0 total, 2048.0 free, 0.0 used. 5912.0 avail Mem

  • the total amount of swap space: 2048.0 total
  • unused swap space: 2048.0 free
  • swap space in use: 0.0 used
  • the amount of swap memory that can be allocated to processes without causing more swapping: 5912.0 avail Mem
The Task Area in top

Below the summary area there comes the task area, which includes a series of fields and columns reporting information about the running processes:

PID: Process identifier.

USER: User who issued the command that generated the process.

PR: Priority of process to the kernel.

NI: Nice value of process. Lower values have a higher priority than higher ones.

VIRT: Total amount of memory used by process (including Swap).

RES: RAM memory used by process.

SHR: Shared memory of the process with other processes.

S: Status of process. Values include: S (interruptible sleep — waiting for an event to finish), R (runnable — either executing or in the queue to be executed) or Z (zombie — terminated child processes whose data structures have not yet been removed from the process table).

%CPU: Percentage of CPU used by process.

%MEM: Percentage of RAM used by process, that is, the RES value expressed as a percentage.

TIME+: Total time of activity of process.

COMMAND: Name of command/program which generated the process.

Monitoring Processes with ps

ps command displays snapshot of processes that are currently running on the Linux system. By default, ps command without any arguments displays processes that are associated with the current shell.

$ ps
    PID TTY          TIME CMD
   3204 pts/0    00:00:17 zsh
   3316 pts/0    00:00:00 zsh
   3577 pts/0    00:00:00 zsh
   3578 pts/0    00:00:00 zsh
   3614 pts/0    00:00:03 gitstatusd-linu
1609949 pts/0    00:00:00 sleep
1825468 pts/0    00:00:00 top
2089640 pts/0    00:00:00 ps

From the output, we obtain the following information:

  • PID The process ID of the process
  • TTY The name of the terminal session (shell) that the process is running within
  • TIME The amount of CPU time used by the process
  • CMD The name of the command that was entered to create the process

ps Option Syntax and Output

ps can accept three different styles: BSD, UNIX and GNU. Let us see how each of these styles would work when reporting information about a particular process ID:

1. BSD

Options do not follow any leading dash:

 $ ps p 2117
    PID TTY      STAT   TIME COMMAND
   2117 tty2     S+     0:04 i3

2. UNIX

Options do follow a leading dash:

$ ps -p 2117
    PID TTY          TIME CMD
   2117 tty2     00:00:04 i3

3. GNU

Options are followed by double leading dashes:

$ ps --pid 2117
    PID TTY          TIME CMD
   2117 tty2     00:00:04 i3

In the above example, ps reports information about the process whose PID is 2117 — in this case, bash. Similarly, you can use ps to search for the processes started by a particular user:

  • ps U frank (BSD)
  • ps -u frank (UNIX)
  • ps --user frank (GNU)

Now, let us check on the processes started by frank:

 $ ps U frank
   PID  TTY      STAT  TIME  COMMAND 
   3204 pts/0    Ss    0:19   /usr/bin/zsh
1609949 pts/0    T     0:00   sleep 30
1825468 pts/0    T     0:00   top

We can get the best out of ps by combining some of its options. A very useful command (producing an output similar to that of top) is ps aux (BSD style). In this case, processes from all shells (not only the current one) are shown. The meaning of the switches are the following:

a: Show processes that are attached to a tty or terminal.

u: Display user-oriented format.

x: Show processes that are not attached to a tty or terminal.

Displaying the process with ps in BSD format;

$ ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           1  0.0  0.1 170000 12144 ?        Ss   Mar26   0:26 /sbin/init splash
root           2  0.0  0.0      0     0 ?        S    Mar26   0:00 [kthreadd]
root           3  0.0  0.0      0     0 ?        I<   Mar26   0:00 [rcu_gp]
root           4  0.0  0.0      0     0 ?        I<   Mar26   0:00 [rcu_par_gp]
root           6  0.0  0.0      0     0 ?        I<   Mar26   0:00 [kworker/0:0H-kblockd]
frank       2054  0.0  0.4 548668 36536 ?        Sl   Mar26   0:00 /usr/libexec/goa-daemon
frank       2061  0.0  0.1 317280  8888 ?        Sl   Mar26   0:00 /usr/libexec/goa-identity-service
frank       2071  0.0  0.0 319252  7548 ?        Ssl  Mar26   0:03 /usr/libexec/gvfs-afc-volume-monitor
frank       2076  0.0  0.0 238224  5688 ?        Ssl  Mar26   0:00 /usr/libexec/gvfs-mtp-volume-monitor
frank       2080  0.0  0.0 240632  6332 ?        Ssl  Mar26   0:00 /usr/libexec/gvfs-gphoto2-volume-monitor

We get the following columns:

USER: Owner of process.

PID: Process identifier.

%CPU: Percentage of CPU used.

%MEM: Percentage of physical memory used.

VSZ: Virtual memory of process in KiB.

RSS: Non-swapped physical memory used by process in KiB.

TT: Terminal (tty) controlling the process.

STAT: Code representing the state of process. Apart from SR and Z (that we saw when describing the output of top), other possible values include: D (uninterruptible sleep — usually waiting for I/O), T (stopped — normally by a control signal). Some extra modifier include: < (high-priority — not nice to other processes), N (low-priority — nice to other processes), or + (in the foreground process group).

STARTED: Time at which the process started.

TIME: Accumulated CPU time.

COMMAND: Command that started the process.

Running Jobs in the Foreground and Background

Some programs can take a long time to run, and you may not want to tie up the command line interface. Fortunately, there’s a simple solution to that problem: run the program in background mode.

Sending a Job to the Background

Running a program in background mode is very easy, it is done by placing an ampersand symbol (&) at the end of a command. We can demonstrate it using sleep command.

Sending a command to the background via the & symbol.

$ sleep 60 &
[1] 2431805

 You can find out about the active jobs (and their status) in your Linux system by running jobs:

$ jobs
[1]  + running    sleep 60

Let us explain the output:

[1]: This number is the job ID and can be used — preceded by a percentage symbol (%) — to change the job status by the fgbg and kill utilities (as you will be shown later).

+: The plus sign indicates the current, default job (that is, the last one being suspended or sent to the background). The previous job is flagged with a minus sign (-). Any other prior jobs are not flagged.

running: Description of the job status.

sleep 60: The command or job itself.

To display job’s process ID (PID) right before the status, use the -l option.

$ jobs -l
[1]  + 2461429 running    sleep 60

Other options are:

-n: Lists only processes that have changed status since the last notification. Possible status include, RunningStoppedTerminated or Done.

-p: Lists process IDs.

-r: Lists only running jobs.

-s: Lists only stopped (or suspended) jobs.

When the background process finishes, it may display a message on the terminal like:

[1]  + 2461429 done       sleep 60

This shows the job number and the status of the job (Done), along with the command that ran in the background.

Sending a Running Program to the Background

To send a program running on foreground to background, first stop or pause the process by pressing Ctrl+Z key combination and a job number is assigned . You will then use bg command to send program to the background.

Let’s pause a running job with Ctrl+Z;

$ sleep 3000
^Z
[1]  + 2524040 suspended  sleep 3000

Sending a paused job to the background with the bg command:

 $ bg %1
[1]  + 2524040 continued  sleep 3000

You can just check the process if it’s running in the background.

 $ jobs -l
[1]  + 2524040 running    sleep 3000

Bringing Jobs to the Foreground

To accomplish this, use the fg command and the background job’s number, preceded by a percent sign (%).

Bringing a background job to the foreground with the fg command.

$ jobs -l
[1]  + 3278988 suspended (signal)  top
[2]    3280570 running    sleep 3000

Bringing background job with job ID [2] to foreground with fg command

$ fg %2
[2]    3280570 running    sleep 3000

Now, when the job is brought back to the foreground mode it is tied up to the terminal until the program end.

Maintaining a Program Running after Logout

Jobs running are all attached to the session of the user who invoked them. That is, when the session is terminated the jobs are gone too.

it is possible to detach jobs from sessions and have them run even after the session is closed. This is achieved with the nohup (“no hangup”) command.

Syntax:

nohup COMMAND &

Keeping a background job running after log out with the nohup command

$ nohup sleep 30000 &
[2] 3502697
nohup: ignoring input and appending output to 'nohup.out'                                                                                                     

Modifying Process Execution Priorities

Sometimes, you may run some applications that need either a higher or lower level of priority. You can modify the priority level assigned by the system to a program using nice and renice commands which allows you to set and change a program’s niceness level.

The nice command allows you to start an application with a non-default niceness level setting.

Syntax:

nice -n VALUE COMMAND

According to the syntax;

The value parameter is a numeric value from –20 to 19. The lower the number, the higher priority the process receives. The default niceness level is zero.

The COMMAND argument indicates the program started at the specified niceness level.

Modifying an program’s niceness level with the nice command;

$ nice -n 10 sleep 3000

You can check the program niceness level using ps command.

$ ps -l 3607152
F S   UID     PID    PPID  C PRI  NI ADDR SZ WCHAN  TTY        TIME CMD
0 S  1000 3607152 3527610  0  90  10 -  2652 hrtime pts/0      0:00 sleep 3000

You notice that the value in the NI (nice) column is 10.

Changing priority of already running process using renice command.

Syntax:

renice PRIORITY [-p PIDS] [-u USERS] [-g GROUPS]

The renice command allows you to change the priority of multiple processes based on a list of PID, USERS and GROUPS.

Changing a running program’s niceness level with the renice command;

$ renice 18 -p 3607152
3607152 (process ID) old priority 10, new priority 18

Note; You can increase the priority of a running process to higher level (less than 0) with super user privileges.

$ sudo renice -15 -p 3607152
[sudo] password for frank: 
3607152 (process ID) old priority 18, new priority -15

Sending Signals to Processes

In Linux, processes communicate with each other using process signals. A process signal is a predefined message that processes recognize and may choose to ignore or act on.

Sending Signals with the kill Command

kill command allows you to send signals to processes based on their process ID (PID). By default, the kill command sends a TERM signal to all the PIDs listed on the command line. You must be the owner of the process or have super user privileges in order to send signal to a process.

Stopping a process with the kill command

$ ps 3876117
    PID TTY      STAT   TIME COMMAND
3876117 pts/1    SN     0:00 sleep 30000

So let’s terminate the above running process with kill command;

$ kill 3876117
[1]  + 3876117 terminated  sleep 30000                                             

You can see that the process is terminated, and is you check it with the ps command you will obtain no output for the successful terminated process.

$ ps 3876117
    PID TTY      STAT   TIME COMMAND

Some processes will ignore the request and you have to force the termination using -s option which allows you to specify other signals (using either their name or signal number). You can also leave off the -s switch and just precede the signal with a dash.

$ ps 3913528
    PID TTY      STAT   TIME COMMAND
3913528 pts/1    T      0:00 sleep 30000

Now,let’s terminate the process using kill command;

$ kill 3913528

let’s check if the process has been terminated;

$ ps 3913528
    PID TTY      STAT   TIME COMMAND
3913528 pts/1    T      0:00 sleep 30000

You find the process did not respond to the signal, so you have to force termination.

$ kill -9 3913528
[1]  + 3913528 killed     sleep 30000                                              

The process has been killed successfully you can confirm it with ps command.

 $ ps 3913528
    PID TTY      STAT   TIME COMMAND

kill signal number 9 (KILL) had to be employed to stop the process.

Sending Signals with the killall Command

Using kill command, you can only use the process PID instead of its command name and this makes it difficult to use sometimes. The killall command is a nice solution, because it can select a process based on the command it is executing and send it a signal.

$ ps
    PID TTY          TIME CMD
3527610 pts/0    00:00:03 zsh
3527686 pts/0    00:00:00 zsh
3527687 pts/0    00:00:00 zsh
3527690 pts/0    00:00:00 zsh
3527695 pts/0    00:00:00 gitstatusd-linu
3987480 pts/0    00:00:00 ps

Now, let’s kill zsh process with killall command,

$ killall zsh

we can confirm if zsh process has been terminated,

$ ps
    PID TTY          TIME CMD
3527695 pts/0    00:00:00 gitstatusd-linu
3996696 pts/0    00:00:00 ps

Sending Signals with the pkill Command

The pkill command is a powerful way to send processes’ signals using selection criteria other than their PID numbers or commands they are running. You can choose by username, user ID (UID), terminal with which the process is associated, and so on.

The pkill utility works hand-in-hand with the pgrep utility. With pgrep,you can test out your selection criteria prior to sending signals to the selected processes via pkill. the -t option is used on the pgrep utility to see all thePIDs attached to the terminal.

Let’s list the processes ID’s attached to pts/0 terminal;

$ pgrep -t pts/0
4058641
4058652
4058681
4058682
4058684

Stopping a group of processes with the pkill command,

$ sudo pkill -t pts/0

Now, let’s check if the processes have been killed,

$ pgrep -t pts/0
4058641

In the above output there is a process which is still running pid 4058641, let’s examine it;

$ ps 4058641
    PID TTY      STAT   TIME COMMAND
4058641 pts/0    Ss     0:01 /usr/bin/zsh

with ps, it is determined that the /usr/bin/zsh program is running on pts/0, which it is supposed to do, because it is our current terminal running.

Conclusion

That’s all about Managing processes on Linux. This guide will help you as a System Administrator to create processes and keep track of them and send them different types of signals if and when required.

You can also check the following articles, very helpful;

LPIC 101 – Managing Shared Libraries in Linux Systems

LPIC 101 – Managing Software Packages on Linux