David Jeffery
dba0d9caa6
sbitmap: only queue kyber's wait callback if not already active
...
[ Upstream commit df034c93f15ee71df231ff9fe311d27ff08a2a52 ]
Under heavy loads where the kyber I/O scheduler hits the token limits for
its scheduling domains, kyber can become stuck. When active requests
complete, kyber may not be woken up leaving the I/O requests in kyber
stuck.
This stuck state is due to a race condition with kyber and the sbitmap
functions it uses to run a callback when enough requests have completed.
The running of a sbt_wait callback can race with the attempt to insert the
sbt_wait. Since sbitmap_del_wait_queue removes the sbt_wait from the list
first then sets the sbq field to NULL, kyber can see the item as not on a
list but the call to sbitmap_add_wait_queue will see sbq as non-NULL. This
results in the sbt_wait being inserted onto the wait list but ws_active
doesn't get incremented. So the sbitmap queue does not know there is a
waiter on a wait list.
Since sbitmap doesn't think there is a waiter, kyber may never be
informed that there are domain tokens available and the I/O never advances.
With the sbt_wait on a wait list, kyber believes it has an active waiter
so cannot insert a new waiter when reaching the domain's full state.
This race can be fixed by only adding the sbt_wait to the queue if the
sbq field is NULL. If sbq is not NULL, there is already an action active
which will trigger the re-running of kyber. Let it run and add the
sbt_wait to the wait list if still needing to wait.
Reviewed-by: Omar Sandoval <osandov@fb.com>
Signed-off-by: David Jeffery <djeffery@redhat.com>
Reported-by: John Pittman <jpittman@redhat.com>
Tested-by: John Pittman <jpittman@redhat.com>
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2020-01-12 12:21:44 +01:00
..
2019-07-03 16:57:17 +02:00
2019-09-05 14:54:54 +10:00
2019-07-25 11:34:39 -07:00
2019-06-20 10:11:24 +02:00
2019-02-06 11:01:57 +01:00
2019-09-20 09:06:26 -07:00
2019-09-25 17:51:41 -07:00
2019-05-21 10:50:46 +02:00
2019-08-22 14:39:36 +10:00
2019-12-17 19:56:09 +01:00
2019-07-02 08:41:37 +02:00
2019-10-23 14:48:23 +02:00
2019-11-15 18:34:00 -08:00
2019-05-21 10:50:46 +02:00
2019-05-21 10:50:46 +02:00
2019-09-15 19:42:16 +02:00
2018-08-22 10:52:48 -07:00
2018-06-12 16:19:22 -07:00
2019-05-24 17:39:02 +02:00
2019-05-24 17:39:02 +02:00
2019-05-24 17:27:11 +02:00
2019-05-24 17:27:11 +02:00
2019-05-30 11:26:32 -07:00
2019-06-03 12:32:56 +02:00
2017-11-02 11:10:55 +01:00
2017-11-02 11:10:55 +01:00
2018-10-12 09:17:46 +02:00
2019-07-25 15:47:37 +02:00
2019-05-21 10:50:45 +02:00
2019-06-05 17:37:10 +02:00
2019-06-05 17:37:16 +02:00
2018-08-16 12:14:42 -07:00
2019-09-25 17:51:41 -07:00
2019-05-24 17:27:11 +02:00
2018-11-30 07:22:05 +01:00
2019-05-30 11:26:32 -07:00
2017-11-02 11:10:55 +01:00
2019-05-30 11:26:32 -07:00
2019-06-19 17:09:55 +02:00
2017-11-02 11:10:55 +01:00
2019-06-19 17:09:06 +02:00
2019-05-24 17:39:02 +02:00
2017-11-02 11:10:55 +01:00
2019-06-19 17:09:55 +02:00
2019-03-12 10:04:02 -07:00
2019-06-19 17:09:06 +02:00
2019-06-19 17:09:06 +02:00
2017-09-26 15:01:20 -06:00
2019-06-19 17:09:06 +02:00
2019-02-15 19:50:07 +01:00
2018-07-27 19:04:33 +08:00
2017-02-24 17:46:57 -08:00
2018-08-22 10:52:48 -07:00
2019-06-19 17:09:06 +02:00
2019-06-19 17:09:06 +02:00
2019-06-19 17:09:06 +02:00
2017-11-02 11:10:55 +01:00
2017-11-02 11:10:55 +01:00
2019-05-21 10:50:45 +02:00
2019-06-14 14:51:16 +02:00
2018-06-12 23:33:24 +02:00
2018-07-27 19:16:38 +08:00
2017-11-02 11:10:55 +01:00
2019-06-19 17:09:55 +02:00
2019-05-24 17:37:53 +02:00
2017-11-02 11:10:55 +01:00
2019-07-05 11:12:07 +09:00
2019-07-10 18:43:43 -07:00
2019-11-06 08:47:50 -08:00
2019-06-18 13:47:24 +02:00
2017-11-15 11:56:19 -08:00
2019-06-05 17:36:37 +02:00
2019-04-09 14:19:06 +02:00
2018-04-27 08:51:26 -04:00
2019-09-25 17:51:39 -07:00
2019-06-18 13:47:24 +02:00
2019-06-05 17:36:38 +02:00
2019-05-30 11:26:32 -07:00
2017-11-02 11:10:55 +01:00
2018-07-27 19:04:33 +08:00
2018-12-29 11:36:44 -08:00
2019-07-12 15:13:55 -07:00
2019-10-14 15:04:00 -07:00
2017-02-24 17:46:57 -08:00
2017-02-24 17:46:57 -08:00
2019-09-25 17:51:39 -07:00
2019-05-13 11:07:33 +02:00
2019-11-01 22:26:34 -04:00
2017-11-02 11:10:55 +01:00
2019-05-21 10:50:45 +02:00
2019-05-21 10:50:45 +02:00
2019-06-19 17:09:56 +02:00
2019-01-22 13:39:59 +01:00
2018-05-09 06:55:44 +02:00
2019-07-16 19:23:22 -07:00
2019-09-24 15:54:08 -07:00
2019-02-19 20:52:19 -07:00
2019-05-30 11:26:32 -07:00
2019-05-24 17:27:11 +02:00
2017-11-02 11:10:55 +01:00
2019-11-10 10:38:43 -08:00
2019-09-25 17:51:40 -07:00
2019-09-24 15:54:07 -07:00
2019-05-21 10:50:46 +02:00
2019-05-21 10:50:46 +02:00
2019-08-30 18:47:15 -07:00
2019-06-19 17:09:08 +02:00
2019-04-01 07:37:12 +02:00
2019-06-19 19:27:39 +02:00
2018-10-31 08:54:13 -07:00
2017-11-02 11:10:55 +01:00
2019-05-30 11:26:32 -07:00
2018-04-11 10:28:35 -07:00
2019-06-20 14:07:34 -06:00
2019-06-05 17:37:06 +02:00
2017-11-02 11:10:55 +01:00
2017-11-02 11:10:55 +01:00
2017-11-02 11:10:55 +01:00
2017-11-02 11:10:55 +01:00
2017-11-02 11:10:55 +01:00
2017-11-02 11:10:55 +01:00
2017-11-02 11:10:55 +01:00
2017-11-02 11:10:55 +01:00
2017-11-02 11:10:55 +01:00
2019-02-28 07:55:43 +01:00
2019-06-07 13:15:06 -07:00
2019-08-13 14:54:24 +08:00
2019-05-24 17:37:53 +02:00
2019-05-24 17:39:02 +02:00
2019-09-06 07:30:23 +02:00
2018-10-16 13:45:44 +02:00
2019-05-21 10:50:45 +02:00
2017-11-02 11:10:55 +01:00
2019-05-24 17:39:02 +02:00
2017-11-02 11:10:55 +01:00
2019-05-21 10:50:45 +02:00
2019-05-04 01:27:11 -04:00
2018-07-09 14:10:40 +02:00
2017-11-02 11:10:55 +01:00
2019-07-03 16:57:18 +02:00
2017-11-02 11:10:55 +01:00
2019-06-05 19:03:39 -07:00
2019-05-21 10:50:45 +02:00
2019-05-24 17:27:11 +02:00
2017-11-04 09:26:51 +09:00
2019-05-03 10:49:17 -04:00
2017-02-03 16:35:42 -05:00
2019-06-19 17:09:06 +02:00
2018-01-26 11:45:16 -06:00
2018-08-30 12:56:40 +02:00
2019-05-21 10:50:45 +02:00
2019-07-14 16:17:18 -07:00
2019-05-21 11:28:45 +02:00
2019-05-21 10:50:45 +02:00
2019-11-03 06:36:50 -05:00
2019-05-17 11:32:47 -07:00
2019-06-05 17:37:16 +02:00
2019-09-25 17:51:39 -07:00
2019-07-16 19:23:22 -07:00
2019-02-04 09:03:31 +01:00
2019-06-19 17:09:55 +02:00
2020-01-12 12:21:44 +01:00
2019-07-11 15:17:41 -07:00
2018-12-22 08:21:03 -05:00
2019-07-11 15:17:41 -07:00
2019-08-08 07:45:01 -06:00
2017-11-02 11:10:55 +01:00
2019-09-24 15:54:09 -07:00
2019-04-25 19:47:24 +02:00
2019-06-03 11:49:37 +02:00
2019-08-19 13:14:53 -03:00
2019-08-19 12:57:28 +02:00
2019-05-30 11:26:32 -07:00
2019-07-16 19:23:22 -07:00
2019-10-08 13:27:05 +02:00
2019-09-25 17:51:41 -07:00
2019-10-01 15:45:03 +02:00
2019-04-05 09:26:43 -04:00
2018-06-27 18:58:49 +03:00
2019-05-21 10:50:45 +02:00
2019-07-01 19:34:46 -07:00
2019-08-20 17:07:29 +02:00
2019-05-21 10:50:45 +02:00
2019-09-04 13:31:28 +02:00
2019-05-21 10:50:45 +02:00
2018-11-30 12:13:15 -08:00
2018-10-15 16:31:29 -04:00
2019-09-24 15:54:08 -07:00
2019-02-01 15:46:23 -08:00
2019-05-21 10:50:45 +02:00
2018-10-16 13:45:44 +02:00
2019-10-14 15:04:01 -07:00
2019-05-21 10:50:45 +02:00
2019-02-13 22:13:29 -08:00
2019-07-16 19:23:22 -07:00
2017-02-26 11:03:38 -05:00
2019-08-15 17:03:16 +02:00
2019-06-19 17:09:55 +02:00
2017-01-09 13:58:57 -05:00
2019-05-21 10:50:45 +02:00
2019-06-05 07:36:43 -07:00
2019-06-05 17:36:37 +02:00
2019-06-05 17:36:37 +02:00
2019-07-16 19:23:22 -07:00
2019-04-08 16:44:21 -06:00
2019-05-14 19:52:51 -07:00
2019-03-07 18:32:00 -08:00
2019-10-16 14:56:21 +02:00
2017-07-21 09:38:30 +02:00
2019-05-14 19:52:49 -07:00
2019-07-01 17:11:16 -04:00
2016-02-03 08:28:43 -08:00
2019-10-03 12:12:23 -04:00
2019-07-24 17:38:01 +02:00
2019-05-30 11:26:32 -07:00
2019-05-30 11:26:32 -07:00
2019-05-30 11:26:32 -07:00
2020-01-09 10:20:07 +01:00
2019-05-06 11:12:09 -07:00
2019-05-24 17:39:02 +02:00
2018-06-07 17:34:39 -07:00
2019-10-01 15:45:03 +02:00
2019-05-30 11:26:41 -07:00
2019-08-15 16:40:10 +02:00
2017-11-02 11:10:55 +01:00
2019-07-01 17:11:16 -04:00
2017-08-15 09:02:07 -07:00