24.3 sockatmark Function

Whenever out-of-band data is received, there is an associated out-of-band mark. This is the position in the normal stream of data at the sender when the sending process sent the out-of-band byte. The receiving process determines whether or not it is at the out-of-band mark by calling the sockatmark function while it reads from the socket.

This function is an invention of POSIX. POSIX is replacing many ioctls with functions.

Figure 24.7 shows an implementation of this function using the commonly found SIOCATMARK ioctl.

Figure 24.7 sockatmark function implemented using ioctl.

lib/sockatmark.c

1 #include    "unp.h"

2 int
3 sockatmark(int fd)
4 {
5     int     flag;

6     if (ioctl(fd, SIOCATMARK, &flag) < 0)
7         return (-1);
8     return (flag != 0);
9 }

The out-of-band mark applies regardless of whether the receiving process is receiving the out-of-band data inline (the SO_OOBINLINE socket option) or out-of-band (the MSG_OOB flag). One common use of the out-of-band mark is for the receiver to treat all the data as special until the mark is passed.

Example

We now show a simple example to illustrate the following two features of the out-of-band mark:

1. The out-of-band mark always points one beyond the final byte of normal data. This means that, if the out-of-band data is received inline, sockatmark returns true if the next byte to be read is the byte that was sent with the MSG_OOB flag. Alternately, if the SO_OOBINLINE socket option is not enabled, then sockatmark returns true if the next byte of data is the first byte that was sent following the out-of-band data.

2. A read operation always stops at the out-of-band mark (pp. 519鈥?20 of TCPv2). That is, if there are 100 bytes in the socket receive buffer, but only 5 bytes until the out-of-band mark, and the process performs a read asking for 100 bytes, only the 5 bytes up to the mark are returned. This forced stop at the mark is to allow the process to call sockatmark to determine if the buffer pointer is at the mark.

Figure 24.8 is our sending program. It sends three bytes of normal data, one byte of out-of-band data, followed by another byte of normal data. There are no pauses between each output operation.

Figure 24.9 is the receiving program. This program does not use the SIGURG signal or select. Instead, it calls sockatmark to determine when the out-of-band byte is encountered.

Figure 24.8 Sending program.

oob/tcpsend04.c

 1 #include    "unp.h"

 2 int
 3 main(int argc, char **argv)
 4 {
 5     int     sockfd;

 6     if (argc != 3)
 7         err_quit("usage: tcpsend04 <host> <port#>");

 8     sockfd = Tcp_connect(argv[1], argv[2]);

 9     Write(sockfd, "123", 3);
10     printf("wrote 3 bytes of normal data\n");

11     Send(sockfd, "4", 1, MSG_OOB);
12     printf("wrote 1 byte of OOB data\n");

13     Write(sockfd, "5", 1);
14     printf("wrote 1 byte of normal data\n");

15     exit(0);
16 }

Figure 24.9 Receiving program that calls sockatmark.

oob/tcprecv04.c

 1 #include    "unp.h"

 2 int
 3 main(int argc, char **argv)
 4 {
 5     int     listenfd, connfd, n, on = 1;
 6     char    buff[100];

 7     if (argc == 2)
 8         listenfd = Tcp_listen(NULL, argv[1], NULL);
 9     else if (argc == 3)
10         listenfd = Tcp_listen(argv[1], argv[2], NULL);
11     else
12         err_quit("usage: tcprecv04 [ <host> ] <port#>");

13     Setsockopt(listenfd, SOL_SOCKET, SO_OOBINLINE, &on, sizeof(on));

14     connfd = Accept(listenfd, NULL, NULL);
15     sleep(5);

16     for ( ; ; ) {
17         if (Sockatmark(connfd))
18             printf("at OOB mark\n");

19         if ( (n = Read(connfd, buff, sizeof(buff) - 1)) == 0) {
20             printf("received EOF\n");
21             exit(0);
22         }
23         buff[n] = 0;            /* null terminate */
24         printf("read %d bytes: %s\n", n, buff);
25     }
26 }

Set SO_OOBINLINE socket option

13 We want to receive the out-of-band data inline, so we must set the SO_OOBINLINE socket option. But if we wait until accept returns and set the option on the connected socket, the three-way handshake is complete and out-of-band data may have already arrived. Therefore, we must set this option for the listening socket, knowing that all socket options carry over from the listening socket to the connected socket (Section 7.4).

sleep after connection accepted

14鈥?5 The receiver sleeps after the connection is accepted to let all the data from the sender be received. This allows us to demonstrate that a read stops at the out-of-band mark, even though additional data is in the socket receive buffer.

Read all data from sender

16鈥?5 The program calls read in a loop, printing the received data. But before calling read, sockatmark checks if the buffer pointer is at the out-of-band mark.

When we run this program, we get the following output:

freebsd4 % tcprecv04 6666
read 3 bytes: 123
at OOB mark
read 2 bytes: 45
received EOF

Even though all the data has been received by the receiving TCP when read is called the first time (because the receiving process calls sleep), only three bytes are returned because the mark is encountered. The next byte read is the out-of-band byte (with a value of 4), because we told the kernel to place the out-of-band data inline.

Example

We now show another simple example to illustrate two additional features of out-of-band data, both of which we mentioned earlier.

1. TCP sends notification of out-of-band data (its urgent pointer), even though it is stopped by flow control from sending data.

2. A receiving process can be notified that the sender has sent out-of-band data (with the SIGURG signal or by select) before the out-of-band data arrives. If the process then calls recv specifying MSG_OOB and the out-of-band data has not arrived, an error of EWOULDBLOCK is returned.

Figure 24.10 is the sending program.

9鈥?9 This process sets the size of its socket send buffer to 32,768, writes 16,384 bytes of normal data, and then sleeps for 5 seconds. We will see shortly that the receiver sets the size of its socket receive buffer to 4,096, so these operations by the sender guarantee that the sending TCP fills the receiver's socket receive buffer. The sender then sends 1 byte of out-of-band data, followed by 1,024 bytes of normal data, and terminates.

Figure 24.10 Sending program.

oob/tcpsend05.c

 1 #include    "unp.h"

 2 int
 3 main(int argc, char **argv)
 4 {
 5     int     sockfd, size;
 6     char    buff[16384];

 7     if (argc != 3)
 8         err_quit("usage: tcpsend05 <host> <port#>");

 9     sockfd = Tcp_connect(argv[1], argv[2]);

10     size = 32768;
11     Setsockopt(sockfd, SOL_SOCKET, SO_SNDBUF, &size, sizeof(size));

12     Write(sockfd, buff, 16384);
13     printf("wrote 16384 bytes of normal data\n");
14     sleep(5);

15     Send(sockfd, "a", 1, MSG_OOB);
16     printf("wrote 1 byte of OOB data\n");

17     Write(sockfd, buff, 1024);
18     printf("wrote 1024 bytes of normal data\n");

19     exit(0);
20 }

Figure 24.11 shows the receiving program.

14鈥?0 The receiving process sets the size of the listening socket's receive buffer to 4,096. This size will carry over to the connected socket after the connection is established. The process then accepts the connection, establishes a signal handler for SIGURG, and establishes the owner of the socket. The main loop calls pause in an infinite loop.

22鈥?1 The signal handler calls recv to read the out-of-band data.

When we start the receiver and then the sender, here is the output from the sender:

macosx % tcpsend05 freebsd4 5555
wrote 16384 bytes of normal data
wrote 1 byte of OOB data
wrote 1024 bytes of normal data

As expected, all the data fits into the sender's socket send buffer, and then it terminates. Here is the output from the receiver:

freebsd4 % tcprecv05 5555
SIGURG received
recv error: Resource temporarily unavailable

The error string printed by our err_sys function corresponds to EAGAIN, which is the same as EWOULDBLOCK in FreeBSD. TCP sends the out-of-band notification to the receiving TCP, which then generates the SIGURG signal for the receiving process. But when recv is called specifying the MSG_OOB flag, the out-of-band byte cannot be read.

Figure 24.11 Receiving program.

oob/tcprecv05.c

 1 #include    "unp.h"

 2 int     listenfd, connfd;

 3 void    sig_urg(int);

 4 int
 5 main(int argc, char **argv)
 6 {
 7     int     size;

 8     if (argc == 2)
 9         listenfd = Tcp_listen(NULL, argv[1], NULL);
10     else if (argc == 3)
11         listenfd = Tcp_listen(argv[1], argv[2], NULL);
12     else
13         err_quit("usage: tcprecv05 [ <host> ] <port#>");

14     size = 4096;
15     Setsockopt(listenfd, SOL_SOCKET, SO_RCVBUF, &size, sizeof(size));

16     connfd = Accept(listenfd, NULL, NULL);

17     Signal(SIGURG, sig_urg);
18     Fcntl(connfd, F_SETOWN, getpid());

19     for ( ;  ; )
20         pause();
21 }

22 void
23 sig_urg(int signo)
24 {
25     int     n;
26     char    buff[2048];

27     printf("SIGURG received\n");
28     n = Recv(connfd, buff, sizeof(buff) - 1, MSG_OOB);
29     buff[n] = 0;                /* null terminate */
30     printf("read %d OOB byte\n", n);
31 }

The solution is for the receiver to make room in its socket receive buffer by reading the normal data that is available. This will cause its TCP to advertise a nonzero window to the sender, which will eventually let the sender transmit the out-of-band byte.

We note two related issues in Berkeley-derived implementations (pp. 1016鈥?017 of TCPv2). First, even if the socket send buffer is full, an out-of-band byte is always accepted by the kernel from the process for sending to the peer. Second, when the process sends an out-of-band byte, a TCP segment is immediately sent that contains the urgent notification. All the normal TCP output checks (Nagle algorithm, silly-window avoidance, etc.) are bypassed.

Example

Our next example demonstrates that there is only a single out-of-band mark for a given TCP connection, and if new out-of-band data arrives before the receiving process reads some existing out-of-band data, the previous mark is lost.

Figure 24.12 is the sending program, which is similar to Figure 24.8 with the addition of another send of out-of-band data, followed by one more write of normal data.

Figure 24.12 Sending two out-of-band bytes in rapid succession.

oob/tcpsend06.c

 1 #include    "unp.h"

 2 int
 3 main(int argc, char **argv)
 4 {
 5     int     sockfd;

 6     if (argc != 3)
 7         err_quit("usage: tcpsend06 <host> <port#>");

 8     sockfd = Tcp_connect(argv[1], argv[2]);

 9     Write(sockfd, "123", 3);
10     printf("wrote 3 bytes of normal data\n");

11     Send(sockfd, "4", 1, MSG_OOB);
12     printf("wrote 1 byte of OOB data\n");

13     Write(sockfd, "5", 1);
14     printf("wrote 1 byte of normal data\n");

15     Send(sockfd, "6", 1, MSG_OOB);
16     printf("wrote 1 byte of OOB data\n");

17     Write(sockfd, "7", 1);
18     printf("wrote 1 byte of normal data\n");

19     exit(0);
20 }

There are no pauses in the sending, allowing all the data to be sent to the receiving TCP quickly.

The receiving program is identical to Figure 24.9, which sleeps for five seconds after accepting the connection to allow the data to arrive at its TCP. Here is the receiving program's output:

freebsd4 % tcprecv06 5555
read 5 bytes: 12345
at OOB mark
read 2 bytes: 67
received EOF

The arrival of the second out-of-band byte (the 6) overwrites the mark that was stored when the first out-of-band byte arrived (the 4). As we said, there is at most one out-of-band mark per TCP connection.