24.2 TCP Out-of-Band Data
TCP does not have true out-of-band data. Instead, TCP provides an
urgent mode. Assume a process has
written N bytes of data to a TCP
socket and that data is queued by TCP in the socket send buffer,
waiting to be sent to the peer. We show this in Figure 24.1 and have labeled the data bytes 1
through N.
The process now writes a single byte of
out-of-band data, containing the ASCII character a, using
the send function and the MSG_OOB flag.
send(fd, "a", 1, MSG_OOB);
TCP places the data in the next available
position in the socket send buffer and sets its urgent pointer for this connection to be the
next available location. We show this in Figure 24.2 and have labeled the out-of-band byte
"OOB."
TCP's urgent pointer has a sequence number one
greater than the byte of data that is written with the
MSG_OOB flag. As discussed on pp. 292鈥?96 of TCPv1, this
is an historical artifact that is now emulated by all
implementations. As long as the sending TCP and the receiving TCP
agree on the interpretation of TCP's urgent pointer, all is
fine.
Given the state of the TCP socket send buffer
shown in Figure 24.2, the
next segment sent by TCP will have its URG flag set in the TCP
header and the urgent offset field in the TCP header will point to
the byte following the out-of-band byte. But this segment may or
may not contain the byte that we have labeled as OOB. Whether the
OOB byte is sent depends on the number of bytes ahead of it in the
socket send buffer, the segment size that TCP is sending to the
peer, and the current window advertised by the peer.
We have used the terms urgent pointer and urgent offset. At the TCP level, the two are
different. The 16-bit value in the TCP header is called the urgent
offset and it must be added to the sequence number field in the
header to obtain the 32-bit urgent pointer. TCP looks at the urgent
offset only if another bit in the header is set, and this bit is
called the URG flag. From a
programming perspective, we need not worry about this detail and
just refer to TCP's urgent pointer.
This is an important characteristic of TCP's
urgent mode: The TCP header indicates that the sender has entered
urgent mode (i.e., the URG flag is set along with the urgent
offset), but the actual byte of data referred to by the urgent
pointer need not be sent. Indeed, if the sending TCP is stopped by
flow control (the receiver's socket receive buffer is full, so its
TCP has advertised a window of 0 to the sending TCP), the urgent
notification is sent without any data (pp. 1016鈥?017 of TCPv2), as
we will show in Figures 24.10 and
24.11. This is one
reason why applications use TCP's urgent mode (i.e., out-of-band
data): The urgent notification is always sent to the peer TCP, even if the flow
of data is stopped by TCP's flow control.
What happens if we send multiple bytes of
out-of-band data, as in
send(fd, "abc", 3, MSG_OOB);
In this example, TCP's urgent pointer points one
beyond the final byte; that is, the final byte (the c) is
considered the out-of-band byte.
Now that we have covered the sending of
out-of-band data, let's look at it from the receiver's side:
-
When TCP receives
a segment with the URG flag set, the urgent pointer is examined to
see whether it refers to new
out-of-band data, that is, whether this is the first time TCP's
urgent mode has referenced this particular byte in the stream of
data from the sender to the receiver. It is common for the sending
TCP to send multiple segments (typically over a short period of
time) containing the URG flag, but with the urgent pointer pointing
to the same byte of data. Only the first of these segments causes
the receiving process to be notified that new out-of-band data has
arrived.
-
The
receiving process is notified when a new urgent pointer arrives.
First the SIGURG signal is sent to the owner of the
socket, assuming either fcntl or ioctl has been
called to establish an owner for the socket (Figure 7.20).
Second, if the process is blocked in a call to select
waiting for this socket descriptor to have an exception condition,
select returns.
These two potential notifications to the
receiving process take place when a new urgent pointer arrives,
regardless of whether the actual byte of data pointed to by the
urgent pointer has arrived at the receiving TCP.
There is only one OOB mark; if a new OOB byte
arrives before the old OOB byte was read, the old byte is
discarded.
-
When
the actual byte of data pointed to by the urgent pointer arrives at
the receiving TCP, the data byte can be pulled out-of-band or left
inline. By default, the SO_OOBINLINE socket option is
not set for a socket, so the
single byte of data is not placed into the socket receive buffer.
Instead, the data byte is placed into a separate one-byte
out-of-band buffer for this connection (pp. 986鈥?88 of TCPv2). The
only way for the process to read from this special one-byte buffer
is to call recv, recvfrom, or recvmsg
and specify the MSG_OOB flag. If a new OOB byte arrives
before the old byte is read, the previous value in this buffer is
discarded.
If, however, the process sets the
SO_OOBINLINE socket option, then the single byte of data
referred to by TCP's urgent pointer is left in the normal socket
receive buffer. The process cannot specify the MSG_OOB
flag to read the data byte in this case. The process will know when
it reaches this byte of data by checking the out-of-band mark for this connection, as we
will describe in Section 24.3.
Some of the following errors are possible:
-
If the process
asks for out-of-band data (e.g., specifying the MSG_OOB
flag), but the peer has not sent any, EINVAL is
returned.
-
If
the process has been notified that the peer has sent an out-of-band
byte (e.g., by SIGURG or select), and the process
tries to read it but that byte has not yet arrived,
EWOULDBLOCK is returned. All the process can do at this
point is read from the socket receive buffer (possibly discarding
the data if it has no room to store the data), to make space in the
buffer so that the peer TCP can send the out-of-band byte.
-
If
the process tries to read the same out-of-band byte multiple times,
EINVAL is returned.
-
If
the process has set the SO_OOBINLINE socket option and
then tries to read the out-of-band data by specifying
MSG_OOB, EINVAL is returned.
Simple Example Using
SIGURG
We now show a trivial example of sending and
receiving out-of-band data. Figure 24.3 shows the sending program.
Nine bytes are sent, with a one-second
sleep between each output operation. The purpose of the
pause is to let each write or send be transmitted
as a single TCP segment and received as such by the other end.
We'll talk later about some of the timing considerations with
out-of-band data. When we run this program, we see the expected
output.
macosx % tcpsend01 freebsd4 9999
wrote 3 bytes of normal data
wrote 1 byte of OOB data
wrote 2 bytes of normal data
wrote 1 byte of OOB data
wrote 2 bytes of normal data
Figure 24.3
Simple out-of-band sending program.
oob/tcpsend01.c
1 #include "unp.h"
2 int
3 main(int argc, char **argv)
4 {
5 int sockfd;
6 if (argc != 3)
7 err_quit("usage: tcpsend01 <host> <port#>");
8 sockfd = Tcp_connect(argv[1], argv[2]);
9 Write(sockfd, "123", 3);
10 printf("wrote 3 bytes of normal data\n");
11 sleep(1);
12 Send(sockfd, "4", 1, MSG_OOB);
13 printf("wrote 1 byte of OOB data\n");
14 sleep(1);
15 Write(sockfd, "56", 2);
16 printf("wrote 2 bytes of normal data\n");
17 sleep(1);
18 Send(sockfd, "7", 1, MSG_OOB);
19 printf("wrote 1 byte of OOB data\n");
20 sleep(1);
21 Write(sockfd, "89", 2);
22 printf("wrote 2 bytes of normal data\n");
23 sleep(1);
24 exit(0);
25 }
Figure
24.4 is the receiving program.
Establish signal handler and socket
owner
16鈥?7
The signal handler for SIGURG is established, and
fcntl sets the owner of the connected socket.
Notice that we do not establish the signal
handler until accept returns. There is a small probability
that out-of-band data can arrive after our TCP completes the
three-way handshake, but before accept returns, which we
would miss. But if we established the signal handler before calling
accept and also set the owner of the listening socket
(which carries over to the connected socket), then if out-of-band
data arrives before accept returns, our signal handler
won't yet have a value for connfd. If this scenario is
important for the application, it should initialize connfd
to 鈥?, check for this value in the signal handler, and if true,
just set a flag for the main loop to check after accept
returns. Alternately, it could block the signal around the call to
accept, but this is subject to all the signal race
conditions we discussed in Section 20.5.
Figure 24.4
Simple out-of-band receiving program.
oob/tcprecv01.c
1 #include "unp.h"
2 int listenfd, connfd;
3 void sig_urg(int);
4 int
5 main(int argc, char **argv)
6 {
7 int n;
8 char buff[100];
9 if (argc == 2)
10 listenfd = Tcp_listen(NULL, argv[1], NULL);
11 else if (argc == 3)
12 listenfd = Tcp_listen(argv[1], argv[2], NULL);
13 else
14 err_quit("usage: tcprecv01 [ <host> ] <port#>");
15 connfd = Accept(listenfd, NULL, NULL);
16 Signal(SIGURG, sig_urg);
17 Fcntl(connfd, F_SETOWN, getpid());
18 for ( ; ; ) {
19 if ( (n = Read(connfd, buff, sizeof(buff) - 1)) == 0) {
20 printf("received EOF\n");
21 exit(0);
22 }
23 buff[n] = 0; /* null terminate */
24 printf("read %d bytes: %s\n", n, buff);
25 }
26 }
27 void
28 sig_urg(int signo)
29 {
30 int n;
31 char buff[100];
32 printf("SIGURG received\n");
33 n = Recv(connfd, buff, sizeof(buff) - 1, MSG_OOB);
34 buff[n] = 0; /* null terminate */
35 printf("read %d OOB byte: %s\n", n, buff);
36 }
18鈥?5
The process reads from the socket, printing each string that is
returned by read. When the sender terminates the
connection, the receiver then terminates.
SIGURG handler
27鈥?6
Our signal handler calls printf, reads the out-of-band
byte by specifying the MSG_OOB flag, and then prints the
returned data. Notice that we ask for up to 100 bytes in the call
to recv, but as we will see shortly, only 1 byte is ever
returned as out-of-band data.
As stated earlier, calling the unsafe
printf function from a signal handler is not recommended.
We do it just to see what's happening with our programs.
Here is the output when we run the receiving
program, and then run the sending program from Figure 24.3:
freebsd4 % tcprecv01 9999
read 3 bytes: 123
SIGURG received
read 1 OOB byte: 4
read 2 bytes: 56
SIGURG received
read 1 OOB byte: 7
read 2 bytes: 89
received EOF
The results are as we expect. Each sending of
out-of-band data by the sender generates SIGURG for the
receiver, which then reads the single out-of-band byte.
Simple Example Using
select
We now redo our out-of-band receiver to use
select instead of the SIGURG signal. Figure 24.5 is the receiving
program.
15鈥?0
The process calls select while waiting for either normal
data (the read set, rset) or out-of-band data (the
exception set, xset). In each case, the received data is
printed.
When we run this program and then run the same
sending program as earlier (Figure 24.3), we encounter the following
error:
freebsd4 % tcprecv02 9999
read 3 bytes: 123
read 1 OOB byte: 4
recv error: Invalid argument
The problem is that select indicates an
exception condition until the process reads beyond the out-of-band data (pp. 530鈥?31 of
TCPv2). We cannot read the out-of-band data more than once because
after we read it the first time, the kernel clears the one-byte
out-of-band buffer. When we call recv specifying the
MSG_OOB flag the second time, it returns
EINVAL.
Figure 24.5
Receiving program that (incorrectly) uses select to be
notified of out-of-band data.
oob/tcprecv02.c
1 #include "unp.h"
2 int
3 main(int argc, char **argv)
4 {
5 int listenfd, connfd, n;
6 char buff[100];
7 fd_set rset, xset;
8 if (argc == 2)
9 listenfd = Tcp_listen(NULL, argv[1], NULL);
10 else if (argc == 3)
11 listenfd = Tcp_listen(argv[1], argv[2], NULL);
12 else
13 err_quit("usage: tcprecv02 [ <host> ] <port#>");
14 connfd = Accept(listenfd, NULL, NULL);
15 FD_ZERO(&rset);
16 FD_ZERO(&xset);
17 for ( ; ; ) {
18 FD_SET(connfd, &rset);
19 FD_SET(connfd, &xset);
20 Select(connfd + 1, &rset, NULL, &xset, NULL);
21 if (FD_ISSET(connfd, &xset)) {
22 n = Recv(connfd, buff, sizeof(buff) - 1, MSG_OOB);
23 buff[n] = 0; /* null terminate */
24 printf("read %d OOB byte: %s\n", n, buff);
25 }
26 if (FD_ISSET(connfd, &rset)) {
27 if ( (n = Read(connfd, buff, sizeof(buff) - 1)) == 0) {
28 printf("received EOF\n");
29 exit(0);
30 }
31 buff[n] = 0; /* null terminate */
32 printf("read %d bytes: %s\n", n, buff);
33 }
34 }
35 }
The solution is to select for an
exception condition only after reading normal data. Figure 24.6 is a modification of
Figure 24.5 that handles
this scenario correctly.
5 We
declare a new variable named justreadoob, which indicates
whether we just read out-of-band data or not. This flag determines
whether or not to select for an exception condition.
26鈥?7
When we set the justreadoob flag, we must also clear the
bit for this descriptor in the exception set.
The program now works as expected.
Figure 24.6
Modification of Figure
24.5 to select for an exception condition
correctly.
oob/tcprecv03.c
1 #include "unp.h"
2 int
3 main(int argc, char **argv)
4 {
5 int listenfd, connfd, n, justreadoob = 0;
6 char buff[100];
7 fd_set rset, xset;
8 if (argc == 2)
9 listenfd = Tcp_listen(NULL, argv[1], NULL);
10 else if (argc == 3)
11 listenfd = Tcp_listen(argv[1], argv[2], NULL);
12 else
13 err_quit("usage: tcprecv03 [ <host> ] <port#>");
14 connfd = Accept(listenfd, NULL, NULL);
15 FD_ZERO(&rset);
16 FD_ZERO(&xset);
17 for ( ; ; ) {
18 FD_SET(connfd, &rset);
19 if (justreadoob == 0)
20 FD_SET(connfd, &xset);
21 Select(connfd + 1, &rset, NULL, &xset, NULL);
22 if (FD_ISSET(connfd, &xset)) {
23 n = Recv(connfd, buff, sizeof(buff) - 1, MSG_OOB);
24 buff[n] = 0; /* null terminate */
25 printf("read %d OOB byte: %s\n", n, buff);
26 justreadoob = 1;
27 FD_CLR(connfd, &xset);
28 }
29 if (FD_ISSET(connfd, &rset)) {
30 if ( (n = Read(connfd, buff, sizeof(buff) - 1) ) == 0) {
31 printf("received EOF\n");
32 exit(0);
33 }
34 buff[n] = 0; /* null terminate */
35 printf("read %d bytes: %s\n", n, buff);
36 justreadoob = 0;
37 }
38 }
39 }
|
