Linux Name spaces

Namespaces in Linux are heavily used by many applications, e.g. LXC, Docker and Openstack.
Question: How to find all existing namespaces in a Linux system?

The answer is quite difficult, because it’s easy to hide a namespace or more exactly make it difficult to find them.

Exploring the system

In the basic/default setup Ubuntu 12.04 and higher provide namespaces for

  • ipc for IPC objects and POSIX message queues
  • mnt for filesystem mountpoints
  • net for network abstraction (VRF)
  • pid to provide a separated, isolated process ID number space
  • uts to isolate two system identifiers — nodename and domainname – to be used by uname

These namespaces are shown for every process in the system. if you execute as rootls -lai /proc/1/nsShell

ls -lai /proc/1/ns
 
60073292 dr-x--x--x 2 root root 0 Dec 15 18:23 .
   10395 dr-xr-xr-x 9 root root 0 Dec  4 11:07 ..
60073293 lrwxrwxrwx 1 root root 0 Dec 15 18:23 ipc -> ipc:[4026531839]
60073294 lrwxrwxrwx 1 root root 0 Dec 15 18:23 mnt -> mnt:[4026531840]
60073295 lrwxrwxrwx 1 root root 0 Dec 15 18:23 net -> net:[4026531968]
60073296 lrwxrwxrwx 1 root root 0 Dec 15 18:23 pid -> pid:[4026531836]
60073297 lrwxrwxrwx 1 root root 0 Dec 15 18:23 uts -> uts:[4026531838]

you get the list of attached namespaces of the init process using PID=1. Even this process has attached namespaces. These are the default namespaces for ipc, mnt, net, pid and uts. For example, the default net namespace is using the ID net:[4026531968]. The number in the brackets is a inode number.

In order to find other namespaces with attached processes in the system, we use these entries of the PID=1 as a reference. Any process or thread in the system, which has not the same namespace ID as PID=1 is not belonging to the DEFAULT namespace.

Additionally, you find the namespaces created by „ip netns add <NAME>“ by default in /var/run/netns/ .

The python code

The python code below is listing all non default namespaces in a system. The program flow is

  • Get the reference namespaces from the init process (PID=1). Assumption: PID=1 is assigned to the default namespaces supported by the system
  • Loop through /var/run/netns/ and add the entries to the list
  • Loop through /proc/ over all PIDs and look for entries in /proc/<PID>/ns/ which are not the same as for PID=1 and add then to the list
  • Print the result

List all non default namespaces in a systemPython

#!/usr/bin/python
#
# List all Namespaces (works for Ubuntu 12.04 and higher)
#
# (C) Ralf Trezeciak    2013-2014
#
#
#    This program is free software: you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation, either version 3 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
#
import os
import fnmatch
 
if os.geteuid() != 0:
    print "This script must be run as root\nBye"
    exit(1)
 
def getinode( pid , type):
    link = '/proc/' + pid + '/ns/' + type
    ret = ''
    try:
        ret = os.readlink( link )
    except OSError as e:
        ret = ''
        pass
    return ret
 
#
# get the running command
def getcmd( p ):
    try:
        cmd = open(os.path.join('/proc', p, 'cmdline'), 'rb').read()
        if cmd == '':
            cmd = open(os.path.join('/proc', p, 'comm'), 'rb').read()
        cmd = cmd.replace('\x00' , ' ')
        cmd = cmd.replace('\n' , ' ')
        return cmd
    except:
        return ''
#
# look for docker parents
def getpcmd( p ):
    try:
        f = '/proc/' + p + '/stat'
        arr = open( f, 'rb').read().split()
        cmd = getcmd( arr[3] )
        if cmd.startswith( '/usr/bin/docker' ):
            return 'docker'
    except:
        pass
    return ''
#
# get the namespaces of PID=1
# assumption: these are the namespaces supported by the system
#
nslist = os.listdir('/proc/1/ns/')
if len(nslist) == 0:
    print 'No Namespaces found for PID=1'
    exit(1)
#print nslist
#
# get the inodes used for PID=1
#
baseinode = []
for x in nslist:
    baseinode.append( getinode( '1' , x ) )
#print "Default namespaces: " , baseinode
err = 0
ns = []
ipnlist = []
#
# loop over the network namespaces created using "ip"
#
try:
    netns = os.listdir('/var/run/netns/')
    for p in netns:
        fd = os.open( '/var/run/netns/' + p, os.O_RDONLY )
        info = os.fstat(fd)
        os.close( fd)
        ns.append( '-- net:[' + str(info.st_ino) + '] created by ip netns add ' + p )
        ipnlist.append( 'net:[' + str(info.st_ino) + ']' )
except:
    # might fail if no network namespaces are existing
    pass
#
# walk through all pids and list diffs
#
pidlist = fnmatch.filter(os.listdir('/proc/'), '[0123456789]*')
#print pidlist
for p in pidlist:
    try:
        pnslist = os.listdir('/proc/' + p + '/ns/')
        for x in pnslist:
            i = getinode ( p , x )
            if i != '' and i not in baseinode:
                cmd = getcmd( p )
                pcmd = getpcmd( p )
                if pcmd != '':
                    cmd = '[' + pcmd + '] ' + cmd
                tag = ''
                if i in ipnlist:
                    tag='**' 
                ns.append( p + ' ' + i + tag + ' ' + cmd)
    except:
        # might happen if a pid is destroyed during list processing
        pass
#
# print the stuff
#
print '{0:>10}  {1:20}  {2}'.format('PID','Namespace','Thread/Command')
for e in ns:
    x = e.split( ' ' , 2 )
    print '{0:>10}  {1:20}  {2}'.format(x[0],x[1],x[2][:60])
#

Copy the script to your system as listns.py , and run it as root using python listns.py

       PID  Namespace             Thread/Command
        --  net:[4026533172]      created by ip netns add qrouter-c33ffc14-dbc2-4730-b787-4747
        --  net:[4026533112]      created by ip netns add qrouter-5a691ed3-f6d3-4346-891a-3b59
        --  net:[4026533050]      created by ip netns add qdhcp-02e848cb-72d0-49df-8592-2f7a03
        --  net:[4026532992]      created by ip netns add qdhcp-47cfcdef-2b34-43b8-a504-6720e5
       297  mnt:[4026531856]      kdevtmpfs 
      3429  net:[4026533050]**    dnsmasq --no-hosts --no-resolv --strict-order --bind-interfa
      3429  mnt:[4026533108]      dnsmasq --no-hosts --no-resolv --strict-order --bind-interfa
      3446  net:[4026532992]**    dnsmasq --no-hosts --no-resolv --strict-order --bind-interfa
      3446  mnt:[4026533109]      dnsmasq --no-hosts --no-resolv --strict-order --bind-interfa
      3486  net:[4026533050]**    /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_fil
      3486  mnt:[4026533107]      /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_fil
      3499  net:[4026532992]**    /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_fil
      3499  mnt:[4026533110]      /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_fil
      4117  net:[4026533112]**    /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_fil
      4117  mnt:[4026533169]      /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_fil
     41998  net:[4026533172]**    /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_fil
     41998  mnt:[4026533229]      /usr/bin/python /usr/bin/neutron-ns-metadata-proxy --pid_fil

The example above is from an Openstack network node. The first four entries are entries created using the command ip. The entry PID=297 is a kernel thread and no user process. All other processes listed, are started by Openstack agents. These process are using network and mount namespaces. PID entries marked with ‚**‘ have a corresponding entry created with the ip command.

When a docker command is started, the output is:

PID  Namespace             Thread/Command
        --  net:[4026532676]      created by ip netns add test
        35  mnt:[4026531856]      kdevtmpfs 
      6189  net:[4026532585]      [docker] /bin/bash 
      6189  uts:[4026532581]      [docker] /bin/bash 
      6189  ipc:[4026532582]      [docker] /bin/bash 
      6189  pid:[4026532583]      [docker] /bin/bash 
      6189  mnt:[4026532580]      [docker] /bin/bash 

The docker child running in the namespaces is marked using [docker].

On a node running mininet and a simple network setup the output looks like :

 exampleShell
       PID  Namespace             Thread/Command
        14  mnt:[4026531856]      kdevtmpfs 
      1198  net:[4026532150]      bash -ms mininet:h1 
      1199  net:[4026532201]      bash -ms mininet:h2 
      1202  net:[4026532252]      bash -ms mininet:h3 
      1203  net:[4026532303]      bash -ms mininet:h4

Googles Chrome Browser

Googles Chrome Browser makes extensive use of the linux namespaces. Start Chrome and run the python script. The output looks like:Chrome’s namespaces

 PID  Namespace             Thread/Command
        63  mnt:[4026531856]      kdevtmpfs 
     30747  net:[4026532344]      /opt/google/chrome/chrome --type=zygote 
     30747  pid:[4026532337]      /opt/google/chrome/chrome --type=zygote 
     30753  net:[4026532344]      /opt/google/chrome/nacl_helper 
     30753  pid:[4026532337]      /opt/google/chrome/nacl_helper 
     30754  net:[4026532344]      /opt/google/chrome/chrome --type=zygote 
     30754  pid:[4026532337]      /opt/google/chrome/chrome --type=zygote 
     30801  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30801  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30807  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30807  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30813  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30813  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30820  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30820  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30829  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30829  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30835  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30835  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30841  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30841  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30887  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30887  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30893  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30893  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30901  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30901  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30910  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30910  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30915  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30915  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30923  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30923  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30933  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30933  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30938  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30938  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30944  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     30944  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     31271  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     31271  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     31538  net:[4026532344]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for
     31538  pid:[4026532337]      /opt/google/chrome/chrome --type=renderer --lang=en-US --for

Chrome makes use of pid and network namespaces to restrict the access of subcomponents. The network namespace does not have a link in /var/run/netns/.

Conclusion

It’s quite hard to explore the Linux namespace. There is a lot of documentation flowing around. I did not find any simple program to look for namespaces in the system. So I wrote one.

The script cannot find a network namespace, which do not have any process attached to AND which has no reference in /var/run/netns/. If root creates the reference inode somewhere else in the filesystem, you may only detect network ports (ovs port, veth port on one side), which are not attached to a known network namespace –> an unknown guest might be on your system using a „hidden“ (not so easy to find) network namespace.

And — Linux namespaces can be stacked.