How Many Ways to Wait for RCU?

I recently noted that there are 11 ways to wait for RCU, and this note was duly noted by others. Please note that this is just vanilla RCU, not counting SRCU and the various flavors of Tasks RCU.

But what exactly are these ways?

Let Me List The Ways

The easy way to answer this is to look at rcutorture's module parameters:

  1. gp_normal waits asynchronously for normal grace periods, as in call_rcu().
  2. gp_sync waits synchronously for normal grace periods, as in synchronize_rcu().
  3. gp_exp waits synchronously for expedited grace periods, as in synchronize_rcu_expedited().
  4. gp_cond waits conditionally and synchronously for normal grace periods, as in: c = get_state_synchronize_rcu(); do_something(); cond_synchronize_rcu();
  5. gp_cond_exp waits conditionally and synchronously for expedited grace periods, as in: c = get_state_synchronize_rcu(); do_something(); cond_synchronize_rcu_expedited();
  6. gp_cond_full waits conditionally and synchronously with a full-sized “cookie” for normal grace periods, as in: get_state_synchronize_rcu_full(&c); do_something(); cond_synchronize_rcu_full(&c);
  7. gp_cond_exp_full waits conditionally and synchronously with a full-sized “cookie” for expedited grace periods, as in: get_state_synchronize_rcu_full(&c); do_something(); cond_synchronize_rcu_expedited_full(&c);
  8. gp_poll poll-waits for normal grace periods, as in: c = start_poll_synchronize_rcu(); do { do_something(); } while (!poll_state_synchronize_rcu());
  9. gp_poll_exp poll-waits for expedited grace periods, as in: c = start_poll_synchronize_rcu_expedited(); do { do_something(); } while (!poll_state_synchronize_rcu());
  10. gp_poll_full poll-waits with a full-sized “cookie” for normal grace periods, as in: start_poll_synchronize_rcu_full(&c); do { do_something(); } while (!poll_state_synchronize_rcu_full(&c));
  11. gp_poll_exp_full poll-waits with a full-sized “cookie” for expedited grace periods, as in: start_poll_synchronize_rcu_expedited_full(&c); do { do_something(); } while (!poll_state_synchronize_rcu_full(&c));

And that makes 11 of them, just as advertised!

In case you are curious, by default, rcutorture uses all eleven, making a random choice on each grace period. These module parameter can be used to restrict rcutorture's attention to the specified grace-period methods.

But Wait! There Are More!!!

There is a call_rcu_hurry() for kernels built with both CONFIG_RCU_LAZY=y and CONFIG_RCU_NOCB_CPU=y. These Kconfig options is used in kernels built for some battery-powered devices, and they make call_rcu() be lazy, as in a callback might wait up to 10 seconds before its grace period starts. Code that is in more of a hurry invokes call_rcu_hurry() in order to avoid this wait.

But shouldn't rcutorture also test call_rcu_hurry()?

It should and it does, for example, when doing callback flooding where a 10-second delay might result in OOM.

One might argue that queue_rcu_work() is yet another way of waiting asynchronously for a normal grace period, and from the user's point of view it most certainly is. But it invokes call_rcu_hurry() internally, and furthermore is implemented within workqueues.

One might further argue that kfree_rcu() and kfree_rcu_mightsleep() are also ways of waiting for a normal grace period. This argument quickly becomes philosophical: Is the user really waiting for that kfree() to happen? Maybe? Besides, Uladzislau Rezki recently posted a patch pulled in my Vlastimil Babka that moves these two functions to the slab allocator. Not only that, but these two functions use the aforementioned queue_rcu_work() function, which in turn uses call_rcu_hurry().

Therefore, rcutorture's testing of call_rcu_hurry() already covers all three of these functions.

How Many Are There Really???

Twelve!!!

Eleven that must be tested separately, and a twelfth in the form of call_rcu_hurry() that is exercised by rcutorture's callback flooding, queueing of callbacks from timer handlers, self-propagating callbacks, rcu_barrier() testing, and debug-objects testing.

Counting up the ways to wait for SRCU and the various Tasks RCU flavors is left as an exercise for the interested reader.