switch_root...

[Rob Landley]

On Thursday 04 June 2009 09:43:27 Nickolay Chokoev wrote:
> Hi,
>
> When I use switch_root I have an error "not rootfs", which comes from the
> condition !S_ISREG(st1.st_mode). When I remove this condition, all seems to
> work fine (or at least I think so).

Sorry for the delay.  I wrote busybox's switch_root (loosely based on the one 
in klibc but mostly based on a knowledge of what the sucker needs to do).

> What is this check for?

Safety.  It does an "rm -rf /" and it gets run as root.  This is not something 
you want a program doing without as many safeguards as humanly possible.

> Can it be removed safely?

If you don't mind accidentally doing an rm -rf / of your development system's 
root filesystem the next time you accidentally run it to test something, sure.

> What's wrong with my system to break here?

If you're _not_ running out of init_ramfs (if for example you're using initrd 
instead), you probably shouldn't use switch_root because it's the wrong tool.

Basically what the sucker does is something like the following shell script:

  find / -xdev | xargs rm -rf
  cd "$1"
  shift
  mount --move . /
  exec chroot . "$@"

There are a couple reasons that won't work as a shell script:

1) If you delete the commands out of your $PATH, your shell scripts can't run 
more commands, but you can't start using dynamically linked _new_ commands 
until after you do the chroot because the path to the dynamic linker is wrong.  
So there's a step that needs to be sort of atomic but can't be as a shell 
script.  (You can work around this with static linking or very carefully laid 
out paths and sequencing, but it's brittle, ugly, and non-obvious.)

2) The "find | rm" bit will acually delete everything because the mount points 
still show up (even if their contents don't), and rm -rf will then happily zap 
that.  So the first line is an oversimplification of what you need to do _not_ 
to descend into other filesystems and delete their contents.

The reason we do this is to free up memory, by the way.  Since initramfs is a 
ramfs, deleting its contents frees up the memory it uses.  (We leave it with 
one remaining dentry for the new mount point, but that's ok.)

Note that you cannot ever umount rootfs, for approximately the same reason you 
can't kill PID 1.  The kernel tracks mount points as a doubly linked list, and 
the pointer to the start/end of that list always points to an entry that's 
known to be there (rootfs), so it never has to worry about moving that pointer 
and it never has to worry about the list being empty.  (Back around 2.6.13 
there _was_ a bug that let you umount rootfs, and the system locked hard the 
instant you did so endlessly looping to find the end of the mount list and 
never stopping.  They fixed it.)

Oh, and the reason we mount --move _and_ do the chroot is due to the way "/" 
works.  Each process has two special symlinks, ".", and "/".  Each of them 
points to the dentry of a directory, and give you a location paths can start 
from.  (Historically ".." was also special, because you could enter a 
directory via a symlink so backing out to the directory you came from doesn't 
necessarily mean the one physically above where "." points to.  These days I 
think it's just handed off to the filesystem.)

Anyway, path resolution starts with "." or "/" (although the "./" at the start 
of the path may be implicit), meaning it's relative to one of those two 
directories.  Your current directory, and your current root directory.  The 
chdir() syscall changes where "." points to, and the chroot() syscall changes 
where "/" points to.  (Again, both are per-process which is why chroot only 
affects your current process and its child processes.)

Note that chroot() does _not_ change where "." points to, and back before they 
put crazy security checks into the kernel your current directory could be 
somewhere you could no longer access after the chroot.  (The command line 
chroot does a cd as well, the chroot _syscall_ is what I'm talking about.)

The reason mounting something new over / has no obvious effect is the same 
reason mounting something over your current directory has no obvious effect: 
the . and / links aren't recalculated after a mount, so they still point to 
the same dentry they did before, even if that dentry is no longer accessible 
by other means.  Note that "cd ." is a NOP, and "chroot /" is a nop; both look 
up the cached dentry and set it right back.  They don't re-parse any paths, 
because they're what all paths your process uses would be relative to.

That's why the careful sequencing above: we cd into the new mount point before 
we do the mount --move.  Moving the mount point would otherwise make it 
totally inaccessible to is because cd-ing to the old path wouldn't give it to 
us anymore, and cd "/" just gives us the cached dentry from when the process 
was created (in this case the old initramfs one).  But the "." symlink gives 
us the dentry of the filesystem we just moved, so we can then "chroot ." to 
copy that dentry to "/" and get the new filesystem.  If we _didn't_ save that 
dentry in "." we couldn't get it back after the mount --move.

(Yes, this is all screwy and I had to email questions to Linus Torvalds to get 
it straight myself.  I keep meaning to write up a "how mount actually works" 
document someday...)

Back to your original question: the rm -rf / behavior that switch_root does is 
useless on anything that _isn't_ a ramfs, and dangerous besides.  In what 
circumstances were you encountering it?

Rob
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds