Switch User With a Fork

I like open source. Mainly because of the fact, that when I'm curious why or how something works, I can just go and check. Although my previous posts were mainly Win32 related, I'm actually a hard core Linux geek stuck in corporate windows environment :). So I decided to start a section "Dig that code".

I have many friends, who like Linux as much as I do, who write scripts and programs all the time, but who would never download and check someone else's code to find out, what's going on and why something does / doesn't work. (I'm obviously talking about people who already can program, not about users). I think there is some sort of barrier people feel when touching code for the first time, so I'm going to document my steps.

Today's example will be quite basic and simple. The question is, you're logged into a computer, where the fork bomb is currently running. You need to become root to start killing everything. However you can't start any additional programs because the resource limit is already reached.

I thought the answer is exec su, until I actually tried it out.

Theory

System call execve replaces the currently running process with the process from the specified filename. So what should happen upon calling exec su is

1. shell is replaced by su
2. su asks for root credentials
3. upon successful confirmation, su calles execve and is replaced by a new shell

Practice

Easy way how to test this is

$ echo $$
FIRST_PID
$ exec su
# echo $$
SECOND_PID

If the number FIRST_PID equals to SECOND_PID, then you would be able to avoid the fork bomb, however when trying it out, I got different numbers on couple of modern distributions I tried - Gentoo 2008.0 stable, Fedora 8 and Red Hat Enterprise Linux 5.2. I tried on couple of Unixes as well and guess what? The pid number stayed the same. I decided to investigate - dig that code.

The first thing I did, was that I downloaded coreutils and went for the su.c. However to my surprise the relevant lines were looking like this:

static void
run_shell (char const *shell, char const *command, char **additional_args,
    size_t n_additional_args)
{
  size_t n_args = 1 + fast_startup + 2 * !!command + n_additional_args + 1;
  char const **args = xnmalloc (n_args, sizeof *args);
  size_t argno = 1;

  if (simulate_login)
    {
      char *arg0;
      char *shell_basename;

      shell_basename = last_component (shell);
      arg0 = xmalloc (strlen (shell_basename) + 2);
      arg0[0] = '-';
      strcpy (arg0 + 1, shell_basename);
      args[0] = arg0;
    }
  else
    args[0] = last_component (shell);
  if (fast_startup)
    args[argno++] = "-f";
  if (command)
    {
      args[argno++] = "-c";
      args[argno++] = command;
    }
  memcpy (args + argno, additional_args, n_additional_args * sizeof *args);
  args[argno + n_additional_args] = NULL;
  execv (shell, (char **) args); // <- notice this

  {
    int exit_status = (errno == ENOENT ? EXIT_ENOENT : EXIT_CANNOT_INVOKE);
    error (0, errno, "%s", shell);
    exit (exit_status);
  }
}

Obviously, there's happening something fishy, because su is calling execv. There is simply no possibility for the final shell to have a different pid. So I used equery and rpm to find out, where the /bin/su command comes from. And to my surprise it doesn't come from coreutils, but from the shadow package.

After downloading the shadow package, unpacking and checking su.c, I found

/* This I ripped out of su.c from sh-utils after the Mandrake pam patch
 * have been applied.  Some work was needed to get it integrated into
 * su.c from shadow.
 */
static void run_shell (const char *shellstr, char *args[], int doshell,
         char *const envp[])
{
 int child;
 sigset_t ourset;
 int status;
 int ret;

 child = fork (); // <- notice this
 if (child == 0) { /* child shell */
  pam_end (pamh, PAM_SUCCESS);

  if (doshell)
   (void) shell (shellstr, (char *) args[0], envp);
  else
   (void) execve (shellstr, (char **) args, envp);
  exit (errno == ENOENT ? E_CMD_NOTFOUND : E_CMD_NOEXEC);
 } else if (child == -1) {
  (void) fprintf (stderr, "%s: Cannot fork user shell\n", Prog);
  SYSLOG ((LOG_WARN, "Cannot execute %s", shellstr));
  closelog ();
  exit (1);
 }

 ....

}

So there you have it, the forking happens in shadow's version of su, which is today quite a standard among distributions, because of the PAM integration. So next time you find yourself on a system with a fork bomb, you know who to thank to :).