android_kernel_xiaomi_sm8450/block
Jia Cheng Hu 42fc1faf44 UPSTREAM: block, bfq: set next_rq to waker_bfqq->next_rq in waker injection
Since commit c5089591c3ba ("block, bfq: detect wakers and
unconditionally inject their I/O"), when the in-service bfq_queue, say
Q, is temporarily empty, BFQ checks whether there are I/O requests to
inject (also) from the waker bfq_queue for Q. To this goal, the value
pointed by bfqq->waker_bfqq->next_rq must be controlled. However, the
current implementation mistakenly looks at bfqq->next_rq, which
instead points to the next request of the currently served queue.

This mistake evidently causes losses of throughput in scenarios with
waker bfq_queues.

This commit corrects this mistake.

Fixes: c5089591c3ba ("block, bfq: detect wakers and unconditionally inject their I/O")
Signed-off-by: Jia Cheng Hu <jia.jiachenghu@gmail.com>
Signed-off-by: Jan Kara <jack@suse.cz>
Signed-off-by: Paolo Valente <paolo.valente@linaro.org>
Signed-off-by: Jens Axboe <axboe@kernel.dk>

Bug: 193744965
Change-Id: I2b31773e1f67e4828e33a32acfa29b59809be7e0
(cherry picked from commit d4fc3640ff361a09e359867e0bca898abd2b7ecb)
Signed-off-by: Ed Tsai <ed.tsai@mediatek.com>
2021-07-15 19:42:16 +00:00
..
partitions Merge 1f06959bd2 ("block: remove the unused q argument to part_in_flight and part_in_flight_rw") into android-mailine 2020-10-24 10:17:17 +02:00
badblocks.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
bfq-cgroup.c bfq: fix blkio cgroup leakage v4 2020-08-18 07:48:08 -07:00
bfq-iosched.c UPSTREAM: block, bfq: set next_rq to waker_bfqq->next_rq in waker injection 2021-07-15 19:42:16 +00:00
bfq-iosched.h bfq: fix blkio cgroup leakage v4 2020-08-18 07:48:08 -07:00
bfq-wf2q.c bfq: fix blkio cgroup leakage v4 2020-08-18 07:48:08 -07:00
bio-integrity.c block: make function __bio_integrity_free() static 2020-07-02 12:38:18 -06:00
bio.c Revert "BACKPORT: bio: limit bio max size" 2021-05-11 09:34:37 -07:00
blk-cgroup-rwstat.c blk-cgroup: Fix the recursive blkg rwstat 2021-03-30 14:31:48 +02:00
blk-cgroup-rwstat.h blk-cgroup: separate out blkg_rwstat under CONFIG_BLK_CGROUP_RWSTAT 2019-11-07 12:28:13 -07:00
blk-cgroup.c FROMGIT: block: Introduce the ioprio rq-qos policy 2021-07-01 22:31:51 -07:00
blk-core.c ANDROID: Revert "scsi: block: Do not accept any requests while suspended" 2021-07-13 10:52:35 -07:00
blk-crypto-fallback.c Merge commit 382625d0d4 ("Merge tag 'for-5.9/block-20200802' of git://git.kernel.dk/linux-block") into android-mainline 2020-08-06 10:07:17 -07:00
blk-crypto-internal.h block: make blk_crypto_rq_bio_prep() able to fail 2020-10-05 10:47:43 -06:00
blk-crypto.c Merge 3ad11d7ac8 ("Merge tag 'block-5.10-2020-10-12' of git://git.kernel.dk/linux-block") into android-mainline 2020-10-24 17:29:43 +02:00
blk-exec.c block: add a blk_account_io_merge_bio helper 2020-05-27 05:21:23 -06:00
blk-flush.c FROMGIT: block: avoid double io accounting for flush request 2021-05-14 13:50:36 -07:00
blk-integrity.c block: remove the unused blk_integrity_merge_bio export 2020-10-06 07:29:53 -06:00
blk-ioc.c block: remove retry loop in ioc_release_fn() 2020-07-16 10:22:15 -06:00
blk-iocost.c blk-iocost: fix weight updates of inner active iocgs 2021-05-19 10:13:11 +02:00
blk-iolatency.c block: Remove redundant 'return' statement 2020-10-08 07:59:48 -06:00
blk-ioprio.c FROMGIT: block: Introduce the ioprio rq-qos policy 2021-07-01 22:31:51 -07:00
blk-ioprio.h FROMGIT: block: Introduce the ioprio rq-qos policy 2021-07-01 22:31:51 -07:00
blk-lib.c block: add a bdev_is_partition helper 2020-09-25 08:18:57 -06:00
blk-map.c block: fix bmd->is_null_mapped initialization 2020-09-23 09:18:39 -06:00
blk-merge.c block: recalculate segment count for multi-segment discards correctly 2021-03-30 14:32:06 +02:00
blk-mq-cpumap.c blk-mq: remove the calling of local_memory_node() 2020-10-20 07:08:17 -06:00
blk-mq-debugfs-zoned.c block: Cleanup license notice 2019-01-17 21:21:40 -07:00
blk-mq-debugfs.c FROMGIT: block: Introduce the ioprio rq-qos policy 2021-07-01 22:31:51 -07:00
blk-mq-debugfs.h blk-mq: no need to check return value of debugfs_create functions 2019-06-13 03:00:30 -06:00
blk-mq-pci.c block: Fix blk_mq_*_map_queues() kernel-doc headers 2019-05-31 15:12:34 -06:00
blk-mq-rdma.c block: Fix blk_mq_*_map_queues() kernel-doc headers 2019-05-31 15:12:34 -06:00
blk-mq-sched.c Revert "Revert "kyber: fix out of bounds access when preempted"" 2021-06-04 11:15:17 -07:00
blk-mq-sched.h block-5.10-2020-10-12 2020-10-13 12:12:44 -07:00
blk-mq-sysfs.c blk-mq: move cancel of hctx->run_work to the front of blk_exit_queue 2020-10-09 12:46:28 -06:00
blk-mq-tag.c FROMGIT: blk-mq: clear stale request in tags->rq[] before freeing one request pool 2021-05-21 13:17:09 -07:00
blk-mq-tag.h ANDROID: GKI: Add ANDROID_OEM_DATA in struct blk_mq_tags 2021-07-01 22:32:02 -07:00
blk-mq-virtio.c Merge 4646de87d3 ("Merge tag 'mailbox-v5.7' of git://git.linaro.org/landing-teams/working/fujitsu/integration") into android-mainline 2020-04-04 15:55:11 +02:00
blk-mq.c ANDROID: vendor_hooks: add vendor hook in blk_mq_rq_ctx_init() 2021-07-14 14:17:35 +00:00
blk-mq.h ANDROID: GKI: Add ANDROID_OEM_DATA in struct blk_mq_ctx 2021-07-01 22:32:01 -07:00
blk-pm.c scsi: block: Fix a race in the runtime power management code 2021-01-06 14:56:50 +01:00
blk-pm.h ANDROID: Revert "scsi: block: Do not accept any requests while suspended" 2021-07-13 10:52:35 -07:00
blk-rq-qos.c Revert "blk-rq-qos: remove redundant finish_wait to rq_qos_wait." 2020-07-15 09:33:37 -06:00
blk-rq-qos.h FROMGIT: block: Introduce the ioprio rq-qos policy 2021-07-01 22:31:51 -07:00
blk-settings.c Revert "BACKPORT: bio: limit bio max size" 2021-05-11 09:34:37 -07:00
blk-stat.c blk-stat: make q->stats->lock irqsafe 2020-09-01 16:48:46 -06:00
blk-stat.h block: deactivate blk_stat timer in wbt_disable_default() 2018-12-12 06:47:51 -07:00
blk-sysfs.c blk-mq: move cancel of hctx->run_work to the front of blk_exit_queue 2020-10-09 12:46:28 -06:00
blk-throttle.c blk-throttle: Re-use the throtl_set_slice_end() 2020-10-08 08:01:38 -06:00
blk-timeout.c block: blk-timeout: delete duplicated word 2020-07-31 16:29:47 -06:00
blk-wbt.c treewide: Use fallthrough pseudo-keyword 2020-08-23 17:36:59 -05:00
blk-wbt.h blk-wbt: remove wbt_update_limits 2020-05-29 16:30:39 -06:00
blk-zoned.c block: Fix REQ_OP_ZONE_RESET_ALL handling 2021-03-30 14:31:51 +02:00
blk.h block: move blk_mq_sched_try_merge to blk-merge.c 2020-10-06 07:29:53 -06:00
bounce.c block: make bio_crypt_clone() able to fail 2020-10-05 10:47:43 -06:00
bsg-lib.c block: drop double zeroing 2020-09-23 09:18:13 -06:00
bsg.c Merge 5.10.20 into android12-5.10 2021-03-07 12:33:33 +01:00
cmdline-parser.c License cleanup: add SPDX GPL-2.0 license identifier to files with no license 2017-11-02 11:10:55 +01:00
elevator.c block: fix comment and add lockdep assert 2020-10-09 12:34:06 -06:00
genhd.c block: Suppress uevent for hidden device when removed 2021-03-30 14:31:52 +02:00
ioctl.c block: return -EBUSY when there are open partitions in blkdev_reread_part 2021-04-28 13:39:59 +02:00
ioprio.c Revert "block: grant IOPRIO_CLASS_RT to CAP_SYS_NICE" 2020-10-24 17:30:14 +02:00
Kconfig FROMGIT: block: Introduce the ioprio rq-qos policy 2021-07-01 22:31:51 -07:00
Kconfig.iosched FROMGIT: block/mq-deadline: Add cgroup support 2021-07-01 22:31:54 -07:00
keyslot-manager.c UPSTREAM: block/keyslot-manager: introduce devm_blk_ksm_init() 2021-02-23 08:10:56 +01:00
kyber-iosched.c FROMGIT: blk-mq: Improve performance of non-mq IO schedulers with multiple HW queues 2021-07-01 22:31:49 -07:00
Makefile FROMGIT: block/mq-deadline: Add cgroup support 2021-07-01 22:31:54 -07:00
mq-deadline-cgroup.c FROMGIT: block/mq-deadline: Add cgroup support 2021-07-01 22:31:54 -07:00
mq-deadline-cgroup.h FROMGIT: block/mq-deadline: Add cgroup support 2021-07-01 22:31:54 -07:00
mq-deadline-main.c FROMGIT: block/mq-deadline: Prioritize high-priority requests 2021-07-01 22:31:55 -07:00
opal_proto.h block: sed-opal: Change the check condition for regular session validity 2020-03-12 08:00:10 -06:00
OWNERS ANDROID: Add OWNERS files referring to the respective android-mainline OWNERS 2021-04-03 14:11:30 +00:00
scsi_ioctl.c Revert "Revert "iov_iter: transparently handle compat iovecs in import_iovec"" 2020-11-02 09:27:36 +01:00
sed-opal.c block: sed-opal: Change the check condition for regular session validity 2020-03-12 08:00:10 -06:00
t10-pi.c block: Allow t10-pi to be modular 2020-01-06 20:59:04 -07:00