Pseudoterminals ptyFork(): Creating a Child Connected via a PTY Pair

 

Chapter 64: Pseudoterminals
Part 3 of 3 β€” ptyFork(): Creating a Child Connected via a PTY Pair
ptyFork
Core Function
setsid
New Session
TLPI
Β§64.4

πŸ“š Chapter Navigation

What is ptyFork()?

ptyFork() is a higher-level function that combines opening a PTY master (via ptyMasterOpen()) with fork() to create a child process whose standard I/O is connected to the PTY slave. The parent gets the master fd; the child runs with the slave as its controlling terminal and its stdin/stdout/stderr.

This is the exact pattern used by SSH servers, terminal emulators, and the script(1) command. After ptyFork(), the child can exec() any program (like bash) and that program will think it’s running on a real terminal.

Key Terms in This Part

ptyFork() setsid() TIOCSCTTY tcsetattr() TIOCSWINSZ dup2() Controlling Terminal Session Leader openpty() forkpty()

πŸ“– ptyFork() Signature
#include "pty_fork.h"

pid_t ptyFork(int *masterFd,        /* OUT: master fd (set in parent) */
              char *slaveName,       /* OUT: slave device name (may be NULL) */
              size_t snLen,          /* IN:  size of slaveName buffer */
              const struct termios *slaveTermios, /* IN: slave terminal attrs (or NULL) */
              const struct winsize  *slaveWS);    /* IN: slave window size   (or NULL) */

/*
 * Returns:
 *   In parent : PID of child process (> 0), or -1 on error
 *   In child  : 0 (always, on success)
 *
 * After ptyFork():
 *   Parent has masterFd β€” read/write to talk to child
 *   Child has stdin/stdout/stderr all pointing to the PTY slave
 *   Child is session leader of a new session
 *   PTY slave is child's controlling terminal
 */

πŸ—οΈ Parent–Child Layout After ptyFork()

PARENT PROCESS
Has file descriptor:
masterFd β†’ /dev/ptmx
Talks to child via:
read(masterFd, …)
write(masterFd, …)
e.g. sshd, xterm, script

PTY MASTER
/dev/ptmx
β‡…
PTY SLAVE
/dev/pts/N
Line Discipline
Kernel PTY driver

CHILD PROCESS
Standard fds after dup2():
fd 0 (stdin) β†’ PTY slave
fd 1 (stdout) β†’ PTY slave
fd 2 (stderr) β†’ PTY slave
Session:
New session (setsid)
PTY slave = ctty
exec’d program: bash, sh…

πŸ”’ Step-by-Step: What ptyFork() Does

1
Open PTY master β€” calls ptyMasterOpen() to get mfd and slname.
β”‚
If slaveName != NULL: copy slname into it (for login accounting).

2
fork() β€” create child process. Both parent and child continue from here.

P
Parent path: Store master fd in *masterFd. Return child PID. Done.

C
Child path begins here (all steps below happen only in the child):
3a
setsid() β€” child creates a new session. Child becomes session leader. Loses any existing controlling terminal.
3b
close(mfd) β€” close the master fd in the child (child doesn’t need it).
3c
open(slaveName, O_RDWR) β€” open PTY slave. Because child is session leader with no controlling terminal, this automatically makes the slave the controlling terminal on Linux (System V semantics).
3d
TIOCSCTTY ioctl() (if defined) β€” on BSD, a session leader acquires a controlling terminal only via this explicit ioctl. This step makes the code portable to BSD.
3e
tcsetattr() β€” if slaveTermios != NULL, set terminal attributes (baud rate, echo, canonical mode, etc.) to match the caller’s terminal.
3f
TIOCSWINSZ ioctl() β€” if slaveWS != NULL, set window size (rows/columns) on slave to match the parent terminal.
3g
dup2(slaveFd, STDIN_FILENO) + dup2(slaveFd, STDOUT_FILENO) + dup2(slaveFd, STDERR_FILENO) β€” redirect all standard streams to the slave. Then close the original slave fd. Child returns 0.

πŸ”‘ Why setsid() is Critical
❌ Without setsid()
  • Child inherits parent’s session
  • Child inherits parent’s controlling terminal
  • Opening slave won’t make it the controlling terminal
  • Ctrl+C sent to slave would also kill the parent!
  • PTY won’t work correctly
βœ… With setsid()
  • Child is in its own new session
  • Child has NO controlling terminal initially
  • Opening PTY slave makes it the controlling terminal (System V)
  • Signals from slave (SIGINT etc.) only affect child’s process group
  • Parent and child are properly isolated
/* In child, immediately after fork(): */

pid_t sid = setsid();
/*
 * setsid() does three things:
 *   1. Creates a new session; child is its session leader
 *   2. Child becomes process group leader of new process group
 *   3. Child has no controlling terminal
 *
 * Returns: new session ID (= child's PID) on success, -1 on error
 * Fails with EPERM if child is already a process group leader
 * (prevent this by ensuring fork() is called, not the parent directly)
 */
if (sid == -1) {
    perror("setsid");
    exit(EXIT_FAILURE);
}

βš™οΈ TIOCSCTTY – Acquiring Controlling Terminal on BSD

On Linux (System V semantics), a session leader acquires a controlling terminal automatically by being the first to open a terminal device when it has none. On BSD, this does NOT happen automatically β€” you must use an explicit ioctl.

#include <sys/ioctl.h>

/* After opening slave fd in child: */

#ifdef TIOCSCTTY
    /*
     * TIOCSCTTY: "Set Controlling Terminal"
     * arg = 0: make this terminal the controlling terminal
     *           (if already have one, steal it only if arg=1 on some BSDs)
     *
     * On Linux: this is a no-op if open() already set it.
     * On BSD:   required to acquire controlling terminal.
     */
    if (ioctl(slaveFd, TIOCSCTTY, 0) == -1) {
        perror("ioctl TIOCSCTTY");
        exit(EXIT_FAILURE);
    }
#endif

/*
 * After this, the PTY slave IS the controlling terminal:
 * - Ctrl+C from master β†’ SIGINT to child's foreground process group
 * - Ctrl+Z from master β†’ SIGTSTP to child's foreground process group
 * - SIGHUP sent to child if master is closed (HUP)
 */

πŸ–₯️ Matching Terminal Attributes and Window Size

When a program like script(1) or ssh runs under your terminal, the PTY slave should have the same terminal settings and same window size as your real terminal. Otherwise programs like vim won’t know the screen width and will render incorrectly.

#include <termios.h>
#include <sys/ioctl.h>

/* --- Terminal Attributes --- */

struct termios tios;

/* Get attributes from the REAL terminal (parent's stdin) */
if (tcgetattr(STDIN_FILENO, &tios) == -1) {
    perror("tcgetattr");
    exit(EXIT_FAILURE);
}

/* Apply them to the PTY slave */
if (tcsetattr(slaveFd, TCSANOW, &tios) == -1) {
    perror("tcsetattr");
    exit(EXIT_FAILURE);
}

/*
 * struct termios contains:
 *   c_iflag  β€” input modes (ICRNL, IXON...)
 *   c_oflag  β€” output modes (OPOST, ONLCR...)
 *   c_cflag  β€” control modes (baud rate, parity...)
 *   c_lflag  β€” local modes (ECHO, ICANON, ISIG...)
 *   c_cc[]   β€” special characters (EOF=^D, INTR=^C, SUSP=^Z...)
 */

/* --- Window Size --- */

struct winsize ws;

/* Get window size from real terminal */
if (ioctl(STDIN_FILENO, TIOCGWINSZ, &ws) == -1) {
    perror("TIOCGWINSZ");
    exit(EXIT_FAILURE);
}
printf("Terminal: %d rows x %d cols\n", ws.ws_row, ws.ws_col);

/* Set the same window size on PTY slave */
if (ioctl(slaveFd, TIOCSWINSZ, &ws) == -1) {
    perror("TIOCSWINSZ");
    exit(EXIT_FAILURE);
}
/*
 * struct winsize {
 *   unsigned short ws_row;    // rows in characters
 *   unsigned short ws_col;    // columns in characters
 *   unsigned short ws_xpixel; // horizontal pixels (often 0)
 *   unsigned short ws_ypixel; // vertical pixels   (often 0)
 * };
 *
 * When window is resized, parent receives SIGWINCH.
 * Parent should then re-read ws and send TIOCSWINSZ to master fd.
 */

πŸ’» ptyFork() – Full Implementation (Listing 64-2)
/* File: pty_fork.c
 * Implements ptyFork() as described in TLPI Chapter 64, Listing 64-2
 */

#define _XOPEN_SOURCE 600
#include <fcntl.h>
#include <termios.h>
#include <sys/ioctl.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>

#define MAX_SNAME 1000

/* Assume ptyMasterOpen() is available from pty_master_open.c */
extern int ptyMasterOpen(char *slaveName, size_t snLen);

pid_t
ptyFork(int *masterFd,
        char *slaveName, size_t snLen,
        const struct termios *slaveTermios,
        const struct winsize  *slaveWS)
{
    int mfd, slaveFd, savedErrno;
    pid_t childPid;
    char slname[MAX_SNAME];

    /* Step 1: Open the PTY master */
    mfd = ptyMasterOpen(slname, sizeof(slname));
    if (mfd == -1)
        return -1;

    /* Optionally copy slave name to caller */
    if (slaveName != NULL) {
        if (strlen(slname) < snLen) {
            strncpy(slaveName, slname, snLen);
        } else {
            close(mfd);
            errno = EOVERFLOW;
            return -1;
        }
    }

    /* Step 2: Fork */
    childPid = fork();
    if (childPid == -1) {       /* fork() failed */
        savedErrno = errno;
        close(mfd);
        errno = savedErrno;
        return -1;
    }

    if (childPid != 0) {        /* ===== PARENT ===== */
        *masterFd = mfd;        /* Give master fd to parent */
        return childPid;        /* Return child PID */
    }

    /* ===== CHILD ===== (childPid == 0) */

    /* Step 3a: Create a new session */
    if (setsid() == -1) {
        perror("setsid");
        exit(EXIT_FAILURE);
    }

    /* Step 3b: Close the master fd β€” child doesn't need it */
    close(mfd);

    /* Step 3c: Open the PTY slave */
    slaveFd = open(slname, O_RDWR);
    if (slaveFd == -1) {
        perror("open slave");
        exit(EXIT_FAILURE);
    }

    /* Step 3d: BSD portability β€” acquire controlling terminal explicitly */
#ifdef TIOCSCTTY
    if (ioctl(slaveFd, TIOCSCTTY, 0) == -1) {
        perror("ioctl TIOCSCTTY");
        exit(EXIT_FAILURE);
    }
#endif

    /* Step 3e: Set terminal attributes if provided */
    if (slaveTermios != NULL) {
        if (tcsetattr(slaveFd, TCSANOW, slaveTermios) == -1) {
            perror("tcsetattr");
            exit(EXIT_FAILURE);
        }
    }

    /* Step 3f: Set window size if provided */
    if (slaveWS != NULL) {
        if (ioctl(slaveFd, TIOCSWINSZ, slaveWS) == -1) {
            perror("ioctl TIOCSWINSZ");
            exit(EXIT_FAILURE);
        }
    }

    /* Step 3g: Redirect stdin/stdout/stderr to PTY slave */
    if (dup2(slaveFd, STDIN_FILENO)  == -1 ||
        dup2(slaveFd, STDOUT_FILENO) == -1 ||
        dup2(slaveFd, STDERR_FILENO) == -1) {
        perror("dup2");
        exit(EXIT_FAILURE);
    }

    /* Close the extra slaveFd (0/1/2 are now the slave) */
    if (slaveFd > STDERR_FILENO)
        close(slaveFd);

    return 0;   /* Child returns 0 */
}

πŸ’» Complete Usage: Run a Shell via ptyFork()

This example calls ptyFork() and then execs /bin/sh in the child. The parent sends a command to the child via the master fd and reads the output.

/* File: pty_shell_demo.c
 * Demonstrates ptyFork() by running /bin/sh as child.
 * Parent sends "echo hello\n" to child and reads response.
 *
 * Compile: gcc -o pty_shell_demo pty_shell_demo.c pty_master_open.c pty_fork.c
 * Run:     ./pty_shell_demo
 */

#define _XOPEN_SOURCE 600
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <termios.h>
#include <sys/ioctl.h>
#include <sys/wait.h>

extern pid_t ptyFork(int *, char *, size_t,
                     const struct termios *, const struct winsize *);

#define SLAVE_NAME_MAX 256

int main(void)
{
    int masterFd;
    pid_t childPid;
    char slaveName[SLAVE_NAME_MAX];
    struct termios tios;
    struct winsize ws;
    char buf[256];
    ssize_t n;

    /* Get current terminal attrs and window size to pass to child */
    if (tcgetattr(STDIN_FILENO, &tios) == -1) {
        perror("tcgetattr");
        exit(EXIT_FAILURE);
    }
    if (ioctl(STDIN_FILENO, TIOCGWINSZ, &ws) == -1) {
        perror("TIOCGWINSZ");
        exit(EXIT_FAILURE);
    }

    /* Fork with PTY */
    childPid = ptyFork(&masterFd, slaveName, sizeof(slaveName),
                       &tios, &ws);
    if (childPid == -1) {
        perror("ptyFork");
        exit(EXIT_FAILURE);
    }

    if (childPid == 0) {
        /* ===== CHILD: exec a shell ===== */
        execlp("/bin/sh", "sh", (char *)NULL);
        perror("execlp");   /* Only reached on error */
        exit(EXIT_FAILURE);
    }

    /* ===== PARENT: talk to child via masterFd ===== */
    printf("Child PID: %d, Slave PTY: %s\n", childPid, slaveName);
    printf("Master fd: %d\n", masterFd);

    /* Give child a moment to start */
    sleep(1);

    /* Send a command to the child shell */
    const char *cmd = "echo 'Hello from PTY parent'\n";
    if (write(masterFd, cmd, strlen(cmd)) == -1) {
        perror("write");
        exit(EXIT_FAILURE);
    }

    /* Read child's output (may include echo of command + output) */
    sleep(1);   /* Let child process command */
    n = read(masterFd, buf, sizeof(buf) - 1);
    if (n > 0) {
        buf[n] = '\0';
        printf("Received from child:\n%s\n", buf);
    }

    /* Send exit command */
    write(masterFd, "exit\n", 5);
    sleep(1);

    /* Wait for child */
    waitpid(childPid, NULL, 0);
    close(masterFd);

    printf("Done.\n");
    return 0;
}

/*
 * Expected output (approximately):
 * Child PID: 12345, Slave PTY: /dev/pts/8
 * Master fd: 5
 * Received from child:
 * sh-5.1$ echo 'Hello from PTY parent'
 * Hello from PTY parent
 * sh-5.1$
 */

πŸ”§ BSD Helpers: openpty() and forkpty()

BSD-derived systems (and glibc on Linux) provide two convenience functions that do what ptyMasterOpen() and ptyFork() do:

#include <pty.h>        /* Linux (glibc) */
/* or <util.h> on BSD */

/*
 * openpty() β€” opens a PTY pair, returns both fds
 *
 * amaster  : OUT β€” master fd
 * aslave   : OUT β€” slave fd (already opened!)
 * name     : OUT β€” slave device name (may be NULL)
 * termp    : IN  β€” terminal attrs to set on slave (or NULL)
 * winp     : IN  β€” window size to set on slave (or NULL)
 *
 * Link with -lutil on Linux
 */
int openpty(int *amaster, int *aslave, char *name,
            const struct termios *termp,
            const struct winsize *winp);

/*
 * forkpty() β€” openpty() + fork() + dup2() all in one
 *
 * amaster  : OUT β€” master fd (set in parent)
 * name     : OUT β€” slave device name (may be NULL)
 * termp    : IN  β€” terminal attrs (or NULL)
 * winp     : IN  β€” window size (or NULL)
 *
 * Unlike ptyFork(), forkpty() does NOT have an snLen safety argument.
 * Returns: child PID in parent, 0 in child, -1 on error
 */
pid_t forkpty(int *amaster, char *name,
              const struct termios *termp,
              const struct winsize *winp);

/* Usage: */
#include <pty.h>
#include <stdio.h>

int masterFd;
pid_t pid = forkpty(&masterFd, NULL, NULL, NULL);
if (pid == 0) {
    execlp("/bin/sh", "sh", NULL);
}
/* parent: use masterFd */
⚠️ Key difference: forkpty() lacks the snLen buffer-size argument of ptyFork(). This is a potential buffer overflow risk if the name buffer is too small. TLPI’s ptyFork() is safer in this regard.

🧟 Zombie Prevention After ptyFork()

When the child created by ptyFork() terminates, it becomes a zombie unless the parent calls wait()/waitpid(). However, many PTY applications are designed so that both parent and child terminate together.

/* Pattern 1: Parent waits for child explicitly */
int status;
waitpid(childPid, &status, 0);

if (WIFEXITED(status))
    printf("Child exited with status %d\n", WEXITSTATUS(status));
else if (WIFSIGNALED(status))
    printf("Child killed by signal %d\n", WTERMSIG(status));

/* Pattern 2: Parent knows it will exit when child does
 * (e.g. xterm exits when shell exits, so no zombie problem) */

/* Pattern 3: Use SIGCHLD with SA_NOCLDWAIT */
struct sigaction sa;
sa.sa_handler = SIG_DFL;
sa.sa_flags   = SA_NOCLDWAIT;  /* auto-reap children */
sigemptyset(&sa.sa_mask);
sigaction(SIGCHLD, &sa, NULL);

/* Pattern 4: Double-fork to orphan child to init */
/* (init auto-reaps orphans) */
Note on login accounting: If slaveName is passed to ptyFork(), the caller can use it to update utmp/wtmp login records (Chapter 40). Programs that provide login services (SSH, telnet, login) do this. Programs like script(1) do not, since they don’t log users in.

🎯 Interview Questions – ptyFork()
Q1. What does ptyFork() do that a plain fork() cannot?

Plain fork() creates a child sharing the parent’s file descriptors and controlling terminal. ptyFork() additionally: opens a PTY master+slave, creates a new session in the child (setsid()), makes the PTY slave the child’s controlling terminal, optionally configures terminal attributes and window size, and redirects the child’s stdin/stdout/stderr to the slave β€” so any program the child exec()s thinks it’s running on a real terminal.

Q2. Why must the child call setsid() before opening the PTY slave?

setsid() detaches the child from the parent’s session and controlling terminal. Only a process with no controlling terminal can acquire a new one by opening a terminal device. If the child didn’t call setsid() first, opening the slave would not make it the controlling terminal (and on some systems would return ENXIO).

Q3. What is the purpose of dup2() in ptyFork()’s child path?

dup2(slaveFd, STDIN_FILENO) etc. makes file descriptors 0, 1, and 2 point to the PTY slave. This means that any program later exec()ed by the child uses the PTY slave for its standard I/O β€” it reads input from the master side and its output goes back to the master. Without this, the exec’d program would use the parent’s terminal, defeating the purpose of the PTY.

Q4. Why is the master fd closed in the child after fork()?

The child communicates via the slave, not the master. If the child keeps the master fd open, it creates a reference loop and prevents proper HUP detection (when the real master holder closes it, the kernel should see zero master references and send SIGHUP to the slave’s foreground process group, but a leaked master fd prevents this).

Q5. What is TIOCSCTTY and on which platforms is it necessary?

TIOCSCTTY is an ioctl that explicitly acquires a controlling terminal. On BSD-derived systems, opening a terminal does NOT automatically make it the controlling terminal β€” you must perform this ioctl. On Linux (System V semantics), opening a terminal as a session leader without a controlling terminal automatically acquires it, so the ioctl is redundant but harmless. TLPI includes it inside #ifdef TIOCSCTTY for portability.

Q6. Why do we pass slaveTermios and slaveWS to ptyFork()?

Interactive programs like vim and bash query the terminal for its settings and window size. If the slave has default settings instead of matching the user’s real terminal, programs will behave incorrectly β€” wrong line endings, wrong screen dimensions, wrong special characters (e.g. EOF). By copying the real terminal’s attributes and window size to the slave, the child’s programs feel like they’re running in the real terminal.

Q7. What is the difference between ptyFork() and forkpty()?

Both do the same thing (fork + PTY setup). The key difference: ptyFork() accepts snLen to safely bound the slave name buffer size (returns EOVERFLOW if too small). forkpty() does not have this safety argument. Additionally, ptyFork() is designed to have a BSD reimplementation β€” the calling code works with both PTY styles by just swapping the implementation.

Q8. When the master fd is closed by the parent, what happens to the child?

When the last fd for the master side is closed, the kernel detects that the PTY master has no more readers/writers. It sends SIGHUP followed by SIGCONT to the foreground process group of the slave. This is how terminal emulators kill their child shell when the window is closed β€” closing the master fd triggers HUP in the shell.

πŸ“Š Chapter 64 Summary: Key Functions at a Glance

Function
Purpose
Who calls it

posix_openpt()
Open PTY master via /dev/ptmx
ptyMasterOpen()

grantpt()
Set slave ownership/permissions
ptyMasterOpen()

unlockpt()
Remove slave’s kernel lock
ptyMasterOpen()

ptsname()
Get slave device name
ptyMasterOpen()

ptyMasterOpen()
All 4 steps in one call, portable
ptyFork() / application

setsid()
New session in child; lose ctty
ptyFork() child path

TIOCSCTTY
Acquire ctty explicitly (BSD)
ptyFork() child path

tcsetattr()
Set slave terminal attributes
ptyFork() child path

TIOCSWINSZ
Set slave window size
ptyFork() child path

dup2()
Redirect stdin/stdout/stderr
ptyFork() child path

Chapter 64 Complete!
You now understand PTY architecture, the 4-step master open sequence, and how ptyFork() creates a child process with a PTY as its terminal.

← Part 1: Introduction ← Part 2: Opening PTY Master

Leave a Reply

Your email address will not be published. Required fields are marked *