26.5 Thread-Specific Data

When converting existing functions to run in a threads environment, a common problem encountered is due to static variables. A function that keeps state in a private buffer, or one that returns a result in the form of a pointer to a static buffer, is not thread-safe because multiple threads cannot use the buffer to hold different things at the same time. When faced with this problem, there are various solutions:

• Use thread-specific data. This is nontrivial and then converts the function into one that works only on systems with threads support. The advantage to this approach is that the calling sequence does not change and all the changes go into the library function and not the applications that call the function. We show a version of readline that is thread-safe by using thread-specific data later in this section.

• Change the calling sequence so that the caller packages all the arguments into a structure, and also store in that structure the static variables from Figure 3.18. This was also done, and Figure 26.6 shows the new structure and new function prototypes.

Figure 26.6 Data structure and function prototype for re-entrant version of readline.

          typedef struct {
            int       read_fd;        /* caller's descriptor to read from */
            char     *read_ptr;       /* caller's buffer to read into */
            size_t    read_maxlen;    /* caller's max # bytes to read */
                          /* next three are used internally by the function */
            int       rl_cnt;         /* initialize to 0 */
            char     *rl_bufptr;      /* initialize to rl_buf */
            char      rl_buf[MAXLINE];
          } Rline;

          void    readline_rinit(int, void *, size_t, Rline *);
          ssize_t readline_r(Rline *);
          ssize_t Readline_r(Rline *);

These new functions can be used on threaded and nonthreaded systems, but all applications that call readline must change.

• Restructure the interface to avoid any static variables so that the function is thread-safe. For the readline example, this would be the equivalent of ignoring the speedups introduced in Figure 3.18 and going back to the older version in Figure 3.17. Since we said the older version was "painfully slow," taking this option is not always viable.

Thread-specific data is a common technique for making an existing function thread-safe. Before describing the Pthread functions that work with thread-specific data, we describe the concept and a possible implementation, because the functions appear more complicated than they really are.

Part of the complication in many texts on using threads is that their descriptions of thread-specific data read like the Pthreads standard, talking about key-value pairs and keys being opaque objects. We describe thread-specific data in terms of indexes and pointers because common implementations use a small integer index for the key, and the value associated with the index is just a pointer to a region that the thread mallocs.

Each system supports a limited number of thread-specific data items. POSIX requires this limit be no less than 128 (per process), and we assume this limit in the following example. The system (probably the threads library) maintains one array of structures per process, which we call key structures, as we show in Figure 26.7.

The flag in the Key structure indicates whether this array element is currently in use, and all the flags are initialized to be "not in use." When a thread calls pthread_key_create to create a new thread-specific data item, the system searches through its array of Key structures and finds the first one not in use. Its index, 0 through 127, is called the key, and this index is returned to the calling thread. We will talk about the "destructor pointer," the other member of the Key structure, shortly.

In addition to the process-wide array of Key structures, the system maintains numerous pieces of information about each thread within a process. We call this a Pthread structure and part of this information is a 128-element array of pointers, which we call the pkey array. We show this in Figure 26.8.

All entries in the pkey array are initialized to null pointers. These 128 pointers are the "values" associated with each of the possible 128 "keys" in the process.

When we create a key with pthread_key_create, the system tells us its key (index). Each thread can then store a value (pointer) for the key, and each thread normally obtains the pointer by calling malloc. Part of the confusion with thread-specific data is that the pointer is the value in the key-value pair, but the real thread-specific data is whatever this pointer points to.

We now go through an example of how thread-specific data is used, assuming that our readline function uses thread-specific data to maintain the per-thread state across successive calls to the function. Shortly we will show the code for this, modifying our readline function to follow these steps:

1. A process is started and multiple threads are created.

2. One of the threads will be the first to call readline, and it in turn calls pthread_key_create. The system finds the first unused Key structure in Figure 26.7 and returns its index (0鈥?27) to the caller. We assume an index of 1 in this example.

   We will use the pthread_once function to guarantee that pthread_key_create is called only by the first thread to call readline.

3. readline calls pthread_getspecific to get the pkey[1] value (the "pointer" in Figure 26.8 for this key of 1) for this thread, and the returned value is a null pointer. Therefore, readline calls malloc to allocate the memory that it needs to keep the per-thread information across successive calls to readline for this thread. readline initializes this memory as needed and calls pthread_setspecific to set the thread-specific data pointer (pkey[1]) for this key to point to the memory it just allocated. We show this in Figure 26.9, assuming that the calling thread is thread 0 in the process.

   Figure 26.9. Associating malloced region with thread-specific data pointer.

   

   In this figure, we note that the Pthread structure is maintained by the system (probably the thread library), but the actual thread-specific data that we malloc is maintained by our function (readline, in this case). All that pthread_setspecific does is set the pointer for this key in the Pthread structure to point to our allocated memory. Similarly, all that pthread_getspecific does is return that pointer to us.

4. Another thread, say thread n, calls readline, perhaps while thread 0 is still executing within readline.

   readline calls pthread_once to initialize the key for this thread-specific data item, but since it has already been called, it is not called again.

5. readline calls pthread_getspecific to fetch the pkey [1] pointer for this thread, and a null pointer is returned. This thread then calls malloc, followed by pthread_setspecific, just like thread 0, initializing its thread-specific data for this key (1). We show this in Figure 26.10.

   Figure 26.10. Data structures after thread n initializes its thread-specific data.

   

6. Thread n continues executing in readline, using and modifying its own thread-specific data.

One item we have not addressed is: What happens when a thread terminates? If the thread has called our readline function, that function has allocated a region of memory that needs to be freed. This is where the "destructor pointer" from Figure 26.7 is used. When the thread that creates the thread-specific data item calls pthread_key_create, one argument to this function is a pointer to a destructor function. When a thread terminates, the system goes through that thread's pkey array, calling the corresponding destructor function for each non-null pkey pointer. What we mean by "corresponding destructor" is the function pointer stored in the Key array in Figure 26.7. This is how the thread-specific data is freed when a thread terminates.

The first two functions that are normally called when dealing with thread-specific data are pthread_once and pthread_key_create.

pthread_once is normally called every time a function that uses thread-specific data is called, but pthread_once uses the value in the variable pointed to by onceptr to guarantee that the init function is called only one time per process.

pthread_key_create must be called only one time for a given key within a process. The key is returned through the keyptr pointer, and the destructor function, if the argument is a non-null pointer, will be called by each thread on termination if that thread has stored a value for this key.

Typical usage of these two functions (ignoring error returns) is as follows:

     pthread_key_t   rl_key;
     pthread_once_t  rl_once = PTHREAD_ONCE_INIT;

     void
     readline_destructor(void *ptr)
     {
         free(ptr);
     }

     void
     readline_once(void)
     {
         pthread_key_create(&rl_key, readline_destructor);
     }

     ssize_t
     readline( ... )
     {
         ...

         pthread_once(&rl_once, readline_once);

         if ( (ptr = pthread_getspecific(rl_key)) == NULL) {
             ptr = Malloc( ... );
             pthread_setspecific(rl_key, ptr);
             /* initialize memory pointed to by ptr */
         }
         ...
         /* use values pointed to by ptr */
     }

Every time readline is called, it calls pthread_once. This function uses the value pointed to by its onceptr argument (the contents of the variable rl_once) to make certain that its init function is called only one time. This initialization function, readline_once, creates the thread-specific data key that is stored in rl_key, and which readline then uses in calls to pthread_getspecific and pthread_setspecific.

The pthread_getspecific and pthread_setspecific functions are used to fetch and store the value associated with a key. This value is what we called the "pointer" in Figure 26.8. What this pointer points to is up to the application, but normally, it points to dynamically allocated memory.

Notice that the argument to pthread_key_create is a pointer to the key (because this function stores the value assigned to the key), while the arguments to the get and set functions are the key itself (probably a small integer index as discussed earlier).

Example: readline Function Using Thread-Specific Data

We now show a complete example of thread-specific data by converting the optimized version of our readline function from Figure 3.18 to be thread-safe, without changing the calling sequence.

Figure 26.11 shows the first part of the function: the pthread_key_t variable, the pthread_once_t variable, the readline_destructor function, the readline_once function, and our Rline structure that contains all the information we must maintain on a per-thread basis.

Figure 26.11 First part of thread-safe readline function.

threads/readline.c

 1 #include    "unpthread.h"

 2 static pthread_key_t rl_key;
 3 static pthread_once_t rl_once = PTHREAD_ONCE_INIT;

 4 static void
 5 readline_destructor(void *ptr)
 6 {
 7     free(ptr);
 8 }

 9 static void
10 readline_once(void)
11 {
12     Pthread_key_creat(&rl_key, readline_destructor);
13 }

14 typedef struct {
15     int     rl_cnt;                /* initialize to 0 */
16     char   *rl_bufptr;             /* initialize to rl_buf */
17     char    rl_buf[MAXLINE];
18 } Rline;

Destructor

4鈥? Our destructor function just frees the memory that was allocated for this thread.

One-time function

9鈥?3 We will see that our one-time function is called one time by pthread_once, and it just creates the key that is used by readline.

Rline structure

14鈥?8 Our Rline structure contains the three variables that caused the problem by being declared static in Figure 3.18. One of these structures will be dynamically allocated per thread and then released by our destructor function.

Figure 26.12 shows the actual readline function, plus the function my_read it calls. This figure is a modification of Figure 3.18.

my_read function

19鈥?5 The first argument to the function is now a pointer to the Rline structure that was allocated for this thread (the actual thread-specific data).

Allocate thread-specific data

42 We first call pthread_once so that the first thread that calls readline in this process calls readline_once to create the thread-specific data key.

Fetch thread-specific data pointer

43鈥?6 pthread_getspecific returns the pointer to the Rline structure for this thread. But if this is the first time this thread has called readline, the return value is a null pointer. In this case, we allocate space for an Rline structure and the rl_cnt member is initialized to 0 by calloc. We then store the pointer for this thread by calling pthread_setspecific. The next time this thread calls readline, pthread_getspecific will return this pointer that was just stored.

Figure 26.12 Second part of thread-safe readline function.

threads/readline.c

19 static ssize_t
20 my_read(Rline *tsd, int fd, char *ptr)
21 {
22     if (tsd->rl_cnt < = 0 {
23       again:
24         if ( (tsd->rl_cnt = read(fd, tsd->rl_buf, MAXLINE)) < 0) {
25             if (error == EINTR)
26                 goto again;
27             return (-1);
28         } else if (tsd->rl_cnt == 0)
29             return (0);
30         tsd->rl_bufptr = tsd->rl_buf;
31     }

32     tsd->rl_cnt--;
33     *ptr = *tsd->rl_bufptr++;
34     return (1);
35 }

36 ssize_t
37 readline(int fd, void *vptr, size_t maxlen)
38 {
39     size_t n, rc;
40     char     c, *ptr;
41     Rline *tsd;

42     Pthread_once(&rl_once, readline_once);
43     if ( (tsd = pthread_getspecific(rl_key)) == NULL) {
44         tsd = Calloc(1, sizeof(Rline)); /* init to 0 */
45         Pthread_setspecific(rl_key, tsd);
46     }
47     ptr = vptr;
48     for (n = 1; n < maxlen; n++) {
49         if ( (rc = my_read(tsd, fd, &c)) == 1) {
50             *ptr++ = c;
51             if (c == '\n')
52                 break;
53         } else if (rc == 0) {
54             *ptr = 0;
55             return (n - 1);     /* EOF, n - 1 bytes read */
56         } else
57             return (-1);       /* error, errno set by read() */
58     }
59     *ptr = 0;
60     return (n);
61 }