Giovanni Tirloni

December 31, 2017 │ linux, kernel

Processes with brackets in `ps` output

Have you ever wondered why some processes have brackets around their names?

  PID TTY      STAT   TIME COMMAND
    1 ?        Ss     0:03 /usr/lib/systemd/systemd --switched-root --system
    2 ?        S      0:00 [kthreadd]
    4 ?        I<     0:00 [kworker/0:0H]
    6 ?        I<     0:00 [mm_percpu_wq]
...

The COMMAND column shows the command that was used to start that process and all its arguments in a string. By reading the ps(1) man page, you will find out that the brackets mean the arguments weren’t available. But where do they come from exactly?

First, let’s find out where ps itself comes from:

$ rpm -qf /usr/bin/ps
procps-ng-3.3.10-15.fc27.x86_64

OK, the proc filesystem.

This is an file-based interface to internal data structures in the kernel. Each process gets a directory under /proc with a bunch of files that make it easy to retrieve that information. We are interested here in the cmdlin file:

Table 1-1: Process specific entries in /proc
..................................................................
 File        Content
 clear_refs  Clears page referenced bits shown in smaps output
 cmdline     Command line arguments
 cpu         Current and last cpu in which it was executed  (2.4)(smp)
 cwd         Link to the current working directory
 environ     Values of environment variables
 exe         Link to the executable of this process
 fd          Directory, which contains all file descriptors
 maps        Memory maps to executables and library files  (2.4)
 mem         Memory held by this process
 root        Link to the root directory of this process
 stat        Process status
 statm       Process memory status information
 status      Process status in human readable form
 wchan       Present with CONFIG_KALLSYMS=y: it shows the kernel
             symbol the task is blocked in - or not blocked.
 pagemap     Page table
 stack       Report full stack trace, via CONFIG_STACKTRACE
 smaps       an extension based on maps, showing the memory consumption of
             each mapping and flags associated with it
 numa_maps   an extension based on maps, showing the memory locality and
             binding policy as well as mem usage (in pages) of each mapping.

Now that we know where this information is coming from, let’s dive into the procps-ng source code.

By using your favorite code editor (or simply grep), you will find out that the read_unvectored() function is called to read the contents of /proc/%u/cmdline and will return zero when there’s nothing in it. The fill_cmdline_cvt() function then calls escape_command() with the ESC_BRACKETS flag, which adds the brackets we see in ps’ output.

You can check for yourself that such processes really don’t have arguments (zero file size):

$ stat /proc/2/cmdline
  File: /proc/2/cmdline
  Size: 0               Blocks: 0          IO Block: 1024   regular empty file
Device: 14h/20d Inode: 28849       Links: 1
Access: (0444/-r--r--r--)  Uid: (    0/    root)   Gid: (    0/    root)
Context: system_u:system_r:kernel_t:s0
Access: 2017-08-06 23:31:28.110131480 +0000
Modify: 2017-08-06 23:31:28.110131480 +0000
Change: 2017-08-06 23:31:28.110131480 +0000
 Birth: -

But why not? Most commonly, these will be kernel threads implementing helper functions, specific subsystems, work queues, etc. These tasks will register as system time in top.

They can also be processes which were execve’ed with an empty list of arguments; or even a process that overwrote its argv[] with empty data. These occur less often.

If you’re curious to understand what these processes do in more detail, see below. This list includes links to useful information about each process.

The: Documentation directory in the Linux kernel sources is an invaluable resource. And there’s always the source code itself if you’re in doubt about something.