android/android_kernel_samsung_sm8650
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge c547d89a9a ("Merge tag 'for-5.15/io_uring-2021-08-30' of git://git.kernel.dk/linux-block") into android-mainline

Browse Source 
Steps on the way to 5.15-rc1

Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: Ib9e751c8781657d1d96fe1ac8098b38023b50eeb
This commit is contained in:
Greg Kroah-Hartman 2021-09-06 13:35:49 +02:00
commit d8cb93f7eb
249 changed files with 4747 additions and 17053 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
Documentation/ABI/testing/sysfs-block
View File

@ -28,6 +28,18 @@ Description:
For more details refer Documentation/admin-guide/iostats.rst
What: /sys/block/<disk>/diskseq
Date: February 2021
Contact: Matteo Croce <mcroce@microsoft.com>
Description:
The /sys/block/<disk>/diskseq files reports the disk
sequence number, which is a monotonically increasing
number assigned to every drive.
Some devices, like the loop device, refresh such number
every time the backing file is changed.
The value type is 64 bit unsigned.
What: /sys/block/<disk>/<part>/stat
Date: February 2008
Contact: Jerome Marchand <jmarchan@redhat.com>

43
Documentation/ABI/testing/sysfs-block-device
View File

@ -55,6 +55,43 @@ Date: Oct, 2016
KernelVersion: v4.10
Contact: linux-ide@vger.kernel.org
Description:
(RW) Write to the file to turn on or off the SATA ncq (native
command queueing) support. By default this feature is turned
off.
(RW) Write to the file to turn on or off the SATA NCQ (native
command queueing) priority support. By default this feature is
turned off. If the device does not support the SATA NCQ
priority feature, writing "1" to this file results in an error
(see ncq_prio_supported).
What: /sys/block/*/device/sas_ncq_prio_enable
Date: Oct, 2016
KernelVersion: v4.10
Contact: linux-ide@vger.kernel.org
Description:
(RW) This is the equivalent of the ncq_prio_enable attribute
file for SATA devices connected to a SAS host-bus-adapter
(HBA) implementing support for the SATA NCQ priority feature.
This file does not exist if the HBA driver does not implement
support for the SATA NCQ priority feature, regardless of the
device support for this feature (see sas_ncq_prio_supported).
What: /sys/block/*/device/ncq_prio_supported
Date: Aug, 2021
KernelVersion: v5.15
Contact: linux-ide@vger.kernel.org
Description:
(RO) Indicates if the device supports the SATA NCQ (native
command queueing) priority feature.
What: /sys/block/*/device/sas_ncq_prio_supported
Date: Aug, 2021
KernelVersion: v5.15
Contact: linux-ide@vger.kernel.org
Description:
(RO) This is the equivalent of the ncq_prio_supported attribute
file for SATA devices connected to a SAS host-bus-adapter
(HBA) implementing support for the SATA NCQ priority feature.
This file does not exist if the HBA driver does not implement
support for the SATA NCQ priority feature, regardless of the
device support for this feature.

1
Documentation/admin-guide/hw-vuln/index.rst
View File

@ -16,3 +16,4 @@ are configurable at compile, boot or run time.
multihit.rst
special-register-buffer-data-sampling.rst
core-scheduling.rst
l1d_flush.rst

69
Documentation/admin-guide/hw-vuln/l1d_flush.rst Normal file
View File

@ -0,0 +1,69 @@
L1D Flushing
============
With an increasing number of vulnerabilities being reported around data
leaks from the Level 1 Data cache (L1D) the kernel provides an opt-in
mechanism to flush the L1D cache on context switch.
This mechanism can be used to address e.g. CVE-2020-0550. For applications
the mechanism keeps them safe from vulnerabilities, related to leaks
(snooping of) from the L1D cache.
Related CVEs
------------
The following CVEs can be addressed by this
mechanism
============= ======================== ==================
CVE-2020-0550 Improper Data Forwarding OS related aspects
============= ======================== ==================
Usage Guidelines
----------------
Please see document: :ref:`Documentation/userspace-api/spec_ctrl.rst
<set_spec_ctrl>` for details.
**NOTE**: The feature is disabled by default, applications need to
specifically opt into the feature to enable it.
Mitigation
----------
When PR_SET_L1D_FLUSH is enabled for a task a flush of the L1D cache is
performed when the task is scheduled out and the incoming task belongs to a
different process and therefore to a different address space.
If the underlying CPU supports L1D flushing in hardware, the hardware
mechanism is used, software fallback for the mitigation, is not supported.
Mitigation control on the kernel command line
---------------------------------------------
The kernel command line allows to control the L1D flush mitigations at boot
time with the option "l1d_flush=". The valid arguments for this option are:
============ =============================================================
on Enables the prctl interface, applications trying to use
the prctl() will fail with an error if l1d_flush is not
enabled
============ =============================================================
By default the mechanism is disabled.
Limitations
-----------
The mechanism does not mitigate L1D data leaks between tasks belonging to
different processes which are concurrently executing on sibling threads of
a physical CPU core when SMT is enabled on the system.
This can be addressed by controlled placement of processes on physical CPU
cores or by disabling SMT. See the relevant chapter in the L1TF mitigation
document: :ref:`Documentation/admin-guide/hw-vuln/l1tf.rst <smt_control>`.
**NOTE** : The opt-in of a task for L1D flushing works only when the task's
affinity is limited to cores running in non-SMT mode. If a task which
requested L1D flushing is scheduled on a SMT-enabled core the kernel sends
a SIGBUS to the task.

19
Documentation/admin-guide/kernel-parameters.txt
View File

@ -2425,6 +2425,23 @@
feature (tagged TLBs) on capable Intel chips.
Default is 1 (enabled)
l1d_flush= [X86,INTEL]
Control mitigation for L1D based snooping vulnerability.
Certain CPUs are vulnerable to an exploit against CPU
internal buffers which can forward information to a
disclosure gadget under certain conditions.
In vulnerable processors, the speculatively
forwarded data can be used in a cache side channel
attack, to access data to which the attacker does
not have direct access.
This parameter controls the mitigation. The
options are:
on - enable the interface for the mitigation
l1tf= [X86] Control mitigation of the L1TF vulnerability on
affected CPUs
@ -4781,7 +4798,7 @@
reboot= [KNL]
Format (x86 or x86_64):
[w[arm] | c[old] | h[ard] | s[oft] | g[pio]] \
[w[arm] | c[old] | h[ard] | s[oft] | g[pio]] | d[efault] \
[[,]s[mp]#### \
[[,]b[ios] | a[cpi] | k[bd] | t[riple] | e[fi] | p[ci]] \
[[,]f[orce]

27
Documentation/devicetree/bindings/timer/rockchip,rk-timer.txt
View File

@ -1,27 +0,0 @@
Rockchip rk timer
Required properties:
- compatible: should be:
"rockchip,rv1108-timer", "rockchip,rk3288-timer": for Rockchip RV1108
"rockchip,rk3036-timer", "rockchip,rk3288-timer": for Rockchip RK3036
"rockchip,rk3066-timer", "rockchip,rk3288-timer": for Rockchip RK3066
"rockchip,rk3188-timer", "rockchip,rk3288-timer": for Rockchip RK3188
"rockchip,rk3228-timer", "rockchip,rk3288-timer": for Rockchip RK3228
"rockchip,rk3229-timer", "rockchip,rk3288-timer": for Rockchip RK3229
"rockchip,rk3288-timer": for Rockchip RK3288
"rockchip,rk3368-timer", "rockchip,rk3288-timer": for Rockchip RK3368
"rockchip,rk3399-timer": for Rockchip RK3399
- reg: base address of the timer register starting with TIMERS CONTROL register
- interrupts: should contain the interrupts for Timer0
- clocks : must contain an entry for each entry in clock-names
- clock-names : must include the following entries:
"timer", "pclk"
Example:
timer: timer@ff810000 {
compatible = "rockchip,rk3288-timer";
reg = <0xff810000 0x20>;
interrupts = <GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&xin24m>, <&cru PCLK_TIMER>;
clock-names = "timer", "pclk";
};

64
Documentation/devicetree/bindings/timer/rockchip,rk-timer.yaml Normal file
View File

@ -0,0 +1,64 @@
# SPDX-License-Identifier: GPL-2.0
%YAML 1.2
---
$id: http://devicetree.org/schemas/timer/rockchip,rk-timer.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: Rockchip Timer Device Tree Bindings
maintainers:
- Daniel Lezcano <daniel.lezcano@linaro.org>
properties:
compatible:
oneOf:
- const: rockchip,rk3288-timer
- const: rockchip,rk3399-timer
- items:
- enum:
- rockchip,rv1108-timer
- rockchip,rk3036-timer
- rockchip,rk3066-timer
- rockchip,rk3188-timer
- rockchip,rk3228-timer
- rockchip,rk3229-timer
- rockchip,rk3288-timer
- rockchip,rk3368-timer
- rockchip,px30-timer
- const: rockchip,rk3288-timer
reg:
maxItems: 1
interrupts:
maxItems: 1
clocks:
minItems: 2
maxItems: 2
clock-names:
items:
- const: pclk
- const: timer
required:
- compatible
- reg
- interrupts
- clocks
- clock-names
additionalProperties: false
examples:
- |
#include <dt-bindings/interrupt-controller/arm-gic.h>
#include <dt-bindings/clock/rk3288-cru.h>
timer: timer@ff810000 {
compatible = "rockchip,rk3288-timer";
reg = <0xff810000 0x20>;
interrupts = <GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>;
clocks = <&cru PCLK_TIMER>, <&xin24m>;
clock-names = "pclk", "timer";
};

1
Documentation/driver-api/index.rst
View File

@ -85,7 +85,6 @@ available subsections can be seen below.
io-mapping
io_ordering
generic-counter
lightnvm-pblk
memory-devices/index
men-chameleon-bus
ntb

21
Documentation/driver-api/lightnvm-pblk.rst
View File

@ -1,21 +0,0 @@
pblk: Physical Block Device Target
==================================
pblk implements a fully associative, host-based FTL that exposes a traditional
block I/O interface. Its primary responsibilities are:
- Map logical addresses onto physical addresses (4KB granularity) in a
logical-to-physical (L2P) table.
- Maintain the integrity and consistency of the L2P table as well as its
recovery from normal tear down and power outage.
- Deal with controller- and media-specific constrains.
- Handle I/O errors.
- Implement garbage collection.
- Maintain consistency across the I/O stack during synchronization points.
For more information please refer to:
http://lightnvm.io
which maintains updated FAQs, manual pages, technical documentation, tools,
contacts, etc.

1
Documentation/userspace-api/ioctl/ioctl-number.rst
View File

@ -160,7 +160,6 @@ Code Seq# Include File Comments
'K' all linux/kd.h
'L' 00-1F linux/loop.h conflict!
'L' 10-1F drivers/scsi/mpt3sas/mpt3sas_ctl.h conflict!
'L' 20-2F linux/lightnvm.h
'L' E0-FF linux/ppdd.h encrypted disk device driver
<http://linux01.gwdg.de/~alatham/ppdd.html>
'M' all linux/soundcard.h conflict!

8
Documentation/userspace-api/spec_ctrl.rst
View File

@ -106,3 +106,11 @@ Speculation misfeature controls
* prctl(PR_SET_SPECULATION_CTRL, PR_SPEC_INDIRECT_BRANCH, PR_SPEC_ENABLE, 0, 0);
* prctl(PR_SET_SPECULATION_CTRL, PR_SPEC_INDIRECT_BRANCH, PR_SPEC_DISABLE, 0, 0);
* prctl(PR_SET_SPECULATION_CTRL, PR_SPEC_INDIRECT_BRANCH, PR_SPEC_FORCE_DISABLE, 0, 0);
- PR_SPEC_L1D_FLUSH: Flush L1D Cache on context switch out of the task
(works only when tasks run on non SMT cores)
Invocations:
* prctl(PR_GET_SPECULATION_CTRL, PR_SPEC_L1D_FLUSH, 0, 0, 0);
* prctl(PR_SET_SPECULATION_CTRL, PR_SPEC_L1D_FLUSH, PR_SPEC_ENABLE, 0, 0);
* prctl(PR_SET_SPECULATION_CTRL, PR_SPEC_L1D_FLUSH, PR_SPEC_DISABLE, 0, 0);

11
Documentation/x86/x86_64/boot-options.rst
View File

@ -126,7 +126,7 @@ Idle loop
Rebooting
=========
reboot=b[ios] | t[riple] | k[bd] | a[cpi] | e[fi] [, [w]arm | [c]old]
reboot=b[ios] | t[riple] | k[bd] | a[cpi] | e[fi] | p[ci] [, [w]arm | [c]old]
bios
Use the CPU reboot vector for warm reset
warm
@ -145,6 +145,8 @@ Rebooting
Use efi reset_system runtime service. If EFI is not configured or
the EFI reset does not work, the reboot path attempts the reset using
the keyboard controller.
pci
Use a write to the PCI config space register 0xcf9 to trigger reboot.
Using warm reset will be much faster especially on big memory
systems because the BIOS will not go through the memory check.
@ -155,6 +157,13 @@ Rebooting
Don't stop other CPUs on reboot. This can make reboot more reliable
in some cases.
reboot=default
There are some built-in platform specific "quirks" - you may see:
"reboot: <name> series board detected. Selecting <type> for reboots."
In the case where you think the quirk is in error (e.g. you have
newer BIOS, or newer board) using this option will ignore the built-in
quirk table, and use the generic default reboot actions.
Non Executable Mappings
=======================

9
MAINTAINERS
View File

@ -10626,15 +10626,6 @@ F: LICENSES/
F: scripts/spdxcheck-test.sh
F: scripts/spdxcheck.py
LIGHTNVM PLATFORM SUPPORT
M: Matias Bjorling <mb@lightnvm.io>
L: linux-block@vger.kernel.org
S: Maintained
W: http://github/OpenChannelSSD
F: drivers/lightnvm/
F: include/linux/lightnvm.h
F: include/uapi/linux/lightnvm.h
LINEAR RANGES HELPERS
M: Mark Brown <broonie@kernel.org>
R: Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>

3
arch/Kconfig
View File

@ -1282,6 +1282,9 @@ config ARCH_SPLIT_ARG64
config ARCH_HAS_ELFCORE_COMPAT
bool
config ARCH_HAS_PARANOID_L1D_FLUSH
bool
source "kernel/gcov/Kconfig"
source "scripts/gcc-plugins/Kconfig"

1
arch/m68k/configs/stmark2_defconfig
View File

@ -22,7 +22,6 @@ CONFIG_RAMSIZE=0x8000000
CONFIG_VECTORBASE=0x40000000
CONFIG_KERNELBASE=0x40001000
# CONFIG_BLK_DEV_BSG is not set
CONFIG_BLK_CMDLINE_PARSER=y
CONFIG_BINFMT_FLAT=y
CONFIG_BINFMT_ZFLAT=y
CONFIG_BINFMT_MISC=y

2
arch/mips/include/asm/mach-rc32434/rb.h
View File

@ -7,8 +7,6 @@
#ifndef __ASM_RC32434_RB_H
#define __ASM_RC32434_RB_H
#include <linux/genhd.h>
#define REGBASE 0x18000000
#define IDT434_REG_BASE ((volatile void *) KSEG1ADDR(REGBASE))
#define UART0BASE 0x58000

1
arch/riscv/configs/defconfig
View File

@ -132,7 +132,6 @@ CONFIG_DEBUG_PLIST=y
CONFIG_DEBUG_SG=y
# CONFIG_RCU_TRACE is not set
CONFIG_RCU_EQS_DEBUG=y
CONFIG_DEBUG_BLOCK_EXT_DEVT=y
# CONFIG_FTRACE is not set
# CONFIG_RUNTIME_TESTING_MENU is not set
CONFIG_MEMTEST=y

1
arch/riscv/configs/rv32_defconfig
View File

@ -127,7 +127,6 @@ CONFIG_DEBUG_PLIST=y
CONFIG_DEBUG_SG=y
# CONFIG_RCU_TRACE is not set
CONFIG_RCU_EQS_DEBUG=y
CONFIG_DEBUG_BLOCK_EXT_DEVT=y
# CONFIG_FTRACE is not set
# CONFIG_RUNTIME_TESTING_MENU is not set
CONFIG_MEMTEST=y

3
arch/um/drivers/ubd_kern.c
View File

@ -1268,8 +1268,7 @@ static void ubd_map_req(struct ubd *dev, struct io_thread_req *io_req,
rq_for_each_segment(bvec, req, iter) {
BUG_ON(i >= io_req->desc_cnt);
io_req->io_desc[i].buffer =
page_address(bvec.bv_page) + bvec.bv_offset;
io_req->io_desc[i].buffer = bvec_virt(&bvec);
io_req->io_desc[i].length = bvec.bv_len;
i++;
}

1
arch/x86/Kconfig
View File

@ -119,6 +119,7 @@ config X86
select ARCH_WANT_HUGE_PMD_SHARE
select ARCH_WANT_LD_ORPHAN_WARN
select ARCH_WANTS_THP_SWAP if X86_64
select ARCH_HAS_PARANOID_L1D_FLUSH
select BUILDTIME_TABLE_SORT
select CLKEVT_I8253
select CLOCKSOURCE_VALIDATE_LAST_CYCLE

2
arch/x86/include/asm/i8259.h
View File

@ -19,6 +19,8 @@ extern unsigned int cached_irq_mask;
#define PIC_MASTER_OCW3 PIC_MASTER_ISR
#define PIC_SLAVE_CMD 0xa0
#define PIC_SLAVE_IMR 0xa1
#define PIC_ELCR1 0x4d0
#define PIC_ELCR2 0x4d1
/* i8259A PIC related value */
#define PIC_CASCADE_IR 2

2
arch/x86/include/asm/nospec-branch.h
View File

@ -252,6 +252,8 @@ DECLARE_STATIC_KEY_FALSE(switch_mm_always_ibpb);
DECLARE_STATIC_KEY_FALSE(mds_user_clear);
DECLARE_STATIC_KEY_FALSE(mds_idle_clear);
DECLARE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush);
#include <asm/segment.h>
/**

33
arch/x86/include/asm/pc-conf-reg.h Normal file
View File

@ -0,0 +1,33 @@
/* SPDX-License-Identifier: GPL-2.0 */
/*
* Support for the configuration register space at port I/O locations
* 0x22 and 0x23 variously used by PC architectures, e.g. the MP Spec,
* Cyrix CPUs, numerous chipsets.
*/
#ifndef _ASM_X86_PC_CONF_REG_H
#define _ASM_X86_PC_CONF_REG_H
#include <linux/io.h>
#include <linux/spinlock.h>
#include <linux/types.h>
#define PC_CONF_INDEX 0x22
#define PC_CONF_DATA 0x23
#define PC_CONF_MPS_IMCR 0x70
extern raw_spinlock_t pc_conf_lock;
static inline u8 pc_conf_get(u8 reg)
{
outb(reg, PC_CONF_INDEX);
return inb(PC_CONF_DATA);
}
static inline void pc_conf_set(u8 reg, u8 data)
{
outb(reg, PC_CONF_INDEX);
outb(data, PC_CONF_DATA);
}
#endif /* _ASM_X86_PC_CONF_REG_H */

8
arch/x86/include/asm/processor-cyrix.h
View File

@ -5,14 +5,14 @@
* Access order is always 0x22 (=offset), 0x23 (=value)
*/
#include <asm/pc-conf-reg.h>
static inline u8 getCx86(u8 reg)
{
outb(reg, 0x22);
return inb(0x23);
return pc_conf_get(reg);
}
static inline void setCx86(u8 reg, u8 data)
{
outb(reg, 0x22);
outb(data, 0x23);
pc_conf_set(reg, data);
}

2
arch/x86/include/asm/processor.h
View File

@ -136,6 +136,8 @@ struct cpuinfo_x86 {
u16 logical_die_id;
/* Index into per_cpu list: */
u16 cpu_index;
/* Is SMT active on this core? */
bool smt_active;
u32 microcode;
/* Address space bits used by the cache internally */
u8 x86_cache_bits;

6
arch/x86/include/asm/thread_info.h
View File

@ -81,7 +81,7 @@ struct thread_info {
#define TIF_SINGLESTEP 4 /* reenable singlestep on user return*/
#define TIF_SSBD 5 /* Speculative store bypass disable */
#define TIF_SPEC_IB 9 /* Indirect branch speculation mitigation */
#define TIF_SPEC_FORCE_UPDATE 10 /* Force speculation MSR update in context switch */
#define TIF_SPEC_L1D_FLUSH 10 /* Flush L1D on mm switches (processes) */
#define TIF_USER_RETURN_NOTIFY 11 /* notify kernel of userspace return */
#define TIF_UPROBE 12 /* breakpointed or singlestepping */
#define TIF_PATCH_PENDING 13 /* pending live patching update */
@ -93,6 +93,7 @@ struct thread_info {
#define TIF_MEMDIE 20 /* is terminating due to OOM killer */
#define TIF_POLLING_NRFLAG 21 /* idle is polling for TIF_NEED_RESCHED */
#define TIF_IO_BITMAP 22 /* uses I/O bitmap */
#define TIF_SPEC_FORCE_UPDATE 23 /* Force speculation MSR update in context switch */
#define TIF_FORCED_TF 24 /* true if TF in eflags artificially */
#define TIF_BLOCKSTEP 25 /* set when we want DEBUGCTLMSR_BTF */
#define TIF_LAZY_MMU_UPDATES 27 /* task is updating the mmu lazily */
@ -104,7 +105,7 @@ struct thread_info {
#define _TIF_SINGLESTEP (1 << TIF_SINGLESTEP)
#define _TIF_SSBD (1 << TIF_SSBD)
#define _TIF_SPEC_IB (1 << TIF_SPEC_IB)
#define _TIF_SPEC_FORCE_UPDATE (1 << TIF_SPEC_FORCE_UPDATE)
#define _TIF_SPEC_L1D_FLUSH (1 << TIF_SPEC_L1D_FLUSH)
#define _TIF_USER_RETURN_NOTIFY (1 << TIF_USER_RETURN_NOTIFY)
#define _TIF_UPROBE (1 << TIF_UPROBE)
#define _TIF_PATCH_PENDING (1 << TIF_PATCH_PENDING)
@ -115,6 +116,7 @@ struct thread_info {
#define _TIF_SLD (1 << TIF_SLD)
#define _TIF_POLLING_NRFLAG (1 << TIF_POLLING_NRFLAG)
#define _TIF_IO_BITMAP (1 << TIF_IO_BITMAP)
#define _TIF_SPEC_FORCE_UPDATE (1 << TIF_SPEC_FORCE_UPDATE)
#define _TIF_FORCED_TF (1 << TIF_FORCED_TF)
#define _TIF_BLOCKSTEP (1 << TIF_BLOCKSTEP)
#define _TIF_LAZY_MMU_UPDATES (1 << TIF_LAZY_MMU_UPDATES)

2
arch/x86/include/asm/tlbflush.h
View File

@ -83,7 +83,7 @@ struct tlb_state {
/* Last user mm for optimizing IBPB */
union {
struct mm_struct *last_user_mm;
unsigned long last_user_mm_ibpb;
unsigned long last_user_mm_spec;
};
u16 loaded_mm_asid;

12
arch/x86/kernel/acpi/boot.c
View File

@ -558,10 +558,10 @@ acpi_parse_nmi_src(union acpi_subtable_headers * header, const unsigned long end
* If a PIC-mode SCI is not recognized or gives spurious IRQ7's
* it may require Edge Trigger -- use "acpi_sci=edge"
*
* Port 0x4d0-4d1 are ECLR1 and ECLR2, the Edge/Level Control Registers
* Port 0x4d0-4d1 are ELCR1 and ELCR2, the Edge/Level Control Registers
* for the 8259 PIC. bit[n] = 1 means irq[n] is Level, otherwise Edge.
* ECLR1 is IRQs 0-7 (IRQ 0, 1, 2 must be 0)
* ECLR2 is IRQs 8-15 (IRQ 8, 13 must be 0)
* ELCR1 is IRQs 0-7 (IRQ 0, 1, 2 must be 0)
* ELCR2 is IRQs 8-15 (IRQ 8, 13 must be 0)
*/
void __init acpi_pic_sci_set_trigger(unsigned int irq, u16 trigger)
@ -570,7 +570,7 @@ void __init acpi_pic_sci_set_trigger(unsigned int irq, u16 trigger)
unsigned int old, new;
/* Real old ELCR mask */
old = inb(0x4d0) | (inb(0x4d1) << 8);
old = inb(PIC_ELCR1) | (inb(PIC_ELCR2) << 8);
/*
* If we use ACPI to set PCI IRQs, then we should clear ELCR
@ -596,8 +596,8 @@ void __init acpi_pic_sci_set_trigger(unsigned int irq, u16 trigger)
return;
pr_warn("setting ELCR to %04x (from %04x)\n", new, old);
outb(new, 0x4d0);
outb(new >> 8, 0x4d1);
outb(new, PIC_ELCR1);
outb(new >> 8, PIC_ELCR2);
}
int acpi_gsi_to_irq(u32 gsi, unsigned int *irqp)

9
arch/x86/kernel/apic/apic.c
View File

@ -38,6 +38,7 @@
#include <asm/trace/irq_vectors.h>
#include <asm/irq_remapping.h>
#include <asm/pc-conf-reg.h>
#include <asm/perf_event.h>
#include <asm/x86_init.h>
#include <linux/atomic.h>
@ -132,18 +133,14 @@ static int enabled_via_apicbase __ro_after_init;
*/
static inline void imcr_pic_to_apic(void)
{
/* select IMCR register */
outb(0x70, 0x22);
/* NMI and 8259 INTR go through APIC */
outb(0x01, 0x23);
pc_conf_set(PC_CONF_MPS_IMCR, 0x01);
}
static inline void imcr_apic_to_pic(void)
{
/* select IMCR register */
outb(0x70, 0x22);
/* NMI and 8259 INTR go directly to BSP */
outb(0x00, 0x23);
pc_conf_set(PC_CONF_MPS_IMCR, 0x00);
}
#endif

2
arch/x86/kernel/apic/io_apic.c
View File

@ -764,7 +764,7 @@ static bool irq_active_low(int idx)
static bool EISA_ELCR(unsigned int irq)
{
if (irq < nr_legacy_irqs()) {
unsigned int port = 0x4d0 + (irq >> 3);
unsigned int port = PIC_ELCR1 + (irq >> 3);
return (inb(port) >> (irq & 7)) & 1;
}
apic_printk(APIC_VERBOSE, KERN_INFO

2
arch/x86/kernel/apic/vector.c
View File

@ -1299,7 +1299,7 @@ static void __init print_PIC(void)
pr_debug("... PIC ISR: %04x\n", v);
v = inb(0x4d1) << 8 | inb(0x4d0);
v = inb(PIC_ELCR2) << 8 | inb(PIC_ELCR1);
pr_debug("... PIC ELCR: %04x\n", v);
}

70
arch/x86/kernel/cpu/bugs.c
View File

@ -43,6 +43,7 @@ static void __init mds_select_mitigation(void);
static void __init mds_print_mitigation(void);
static void __init taa_select_mitigation(void);
static void __init srbds_select_mitigation(void);
static void __init l1d_flush_select_mitigation(void);
/* The base value of the SPEC_CTRL MSR that always has to be preserved. */
u64 x86_spec_ctrl_base;
@ -76,6 +77,13 @@ EXPORT_SYMBOL_GPL(mds_user_clear);
DEFINE_STATIC_KEY_FALSE(mds_idle_clear);
EXPORT_SYMBOL_GPL(mds_idle_clear);
/*
* Controls whether l1d flush based mitigations are enabled,
* based on hw features and admin setting via boot parameter
* defaults to false
*/
DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush);
void __init check_bugs(void)
{
identify_boot_cpu();
@ -111,6 +119,7 @@ void __init check_bugs(void)
mds_select_mitigation();
taa_select_mitigation();
srbds_select_mitigation();
l1d_flush_select_mitigation();
/*
* As MDS and TAA mitigations are inter-related, print MDS
@ -491,6 +500,34 @@ static int __init srbds_parse_cmdline(char *str)
}
early_param("srbds", srbds_parse_cmdline);
#undef pr_fmt
#define pr_fmt(fmt) "L1D Flush : " fmt
enum l1d_flush_mitigations {
L1D_FLUSH_OFF = 0,
L1D_FLUSH_ON,
};
static enum l1d_flush_mitigations l1d_flush_mitigation __initdata = L1D_FLUSH_OFF;
static void __init l1d_flush_select_mitigation(void)
{
if (!l1d_flush_mitigation || !boot_cpu_has(X86_FEATURE_FLUSH_L1D))
return;
static_branch_enable(&switch_mm_cond_l1d_flush);
pr_info("Conditional flush on switch_mm() enabled\n");
}
static int __init l1d_flush_parse_cmdline(char *str)
{
if (!strcmp(str, "on"))
l1d_flush_mitigation = L1D_FLUSH_ON;
return 0;
}
early_param("l1d_flush", l1d_flush_parse_cmdline);
#undef pr_fmt
#define pr_fmt(fmt) "Spectre V1 : " fmt
@ -1215,6 +1252,24 @@ static void task_update_spec_tif(struct task_struct *tsk)
speculation_ctrl_update_current();
}
static int l1d_flush_prctl_set(struct task_struct *task, unsigned long ctrl)
{
if (!static_branch_unlikely(&switch_mm_cond_l1d_flush))
return -EPERM;
switch (ctrl) {
case PR_SPEC_ENABLE:
set_ti_thread_flag(&task->thread_info, TIF_SPEC_L1D_FLUSH);
return 0;
case PR_SPEC_DISABLE:
clear_ti_thread_flag(&task->thread_info, TIF_SPEC_L1D_FLUSH);
return 0;
default:
return -ERANGE;
}
}
static int ssb_prctl_set(struct task_struct *task, unsigned long ctrl)
{
if (ssb_mode != SPEC_STORE_BYPASS_PRCTL &&
@ -1324,6 +1379,8 @@ int arch_prctl_spec_ctrl_set(struct task_struct *task, unsigned long which,
return ssb_prctl_set(task, ctrl);
case PR_SPEC_INDIRECT_BRANCH:
return ib_prctl_set(task, ctrl);
case PR_SPEC_L1D_FLUSH:
return l1d_flush_prctl_set(task, ctrl);
default:
return -ENODEV;
}
@ -1340,6 +1397,17 @@ void arch_seccomp_spec_mitigate(struct task_struct *task)
}
#endif
static int l1d_flush_prctl_get(struct task_struct *task)
{
if (!static_branch_unlikely(&switch_mm_cond_l1d_flush))
return PR_SPEC_FORCE_DISABLE;
if (test_ti_thread_flag(&task->thread_info, TIF_SPEC_L1D_FLUSH))
return PR_SPEC_PRCTL | PR_SPEC_ENABLE;
else
return PR_SPEC_PRCTL | PR_SPEC_DISABLE;
}
static int ssb_prctl_get(struct task_struct *task)
{
switch (ssb_mode) {
@ -1390,6 +1458,8 @@ int arch_prctl_spec_ctrl_get(struct task_struct *task, unsigned long which)
return ssb_prctl_get(task);
case PR_SPEC_INDIRECT_BRANCH:
return ib_prctl_get(task);
case PR_SPEC_L1D_FLUSH:
return l1d_flush_prctl_get(task);
default:
return -ENODEV;
}

8
arch/x86/kernel/i8259.c
View File

@ -235,15 +235,15 @@ static char irq_trigger[2];
*/
static void restore_ELCR(char *trigger)
{
outb(trigger[0], 0x4d0);
outb(trigger[1], 0x4d1);
outb(trigger[0], PIC_ELCR1);
outb(trigger[1], PIC_ELCR2);
}
static void save_ELCR(char *trigger)
{
/* IRQ 0,1,2,8,13 are marked as reserved */
trigger[0] = inb(0x4d0) & 0xF8;
trigger[1] = inb(0x4d1) & 0xDE;
trigger[0] = inb(PIC_ELCR1) & 0xF8;
trigger[1] = inb(PIC_ELCR2) & 0xDE;
}
static void i8259A_resume(void)

3
arch/x86/kernel/mpparse.c
View File

@ -19,6 +19,7 @@
#include <linux/smp.h>
#include <linux/pci.h>
#include <asm/i8259.h>
#include <asm/io_apic.h>
#include <asm/acpi.h>
#include <asm/irqdomain.h>
@ -251,7 +252,7 @@ static int __init ELCR_trigger(unsigned int irq)
{
unsigned int port;
port = 0x4d0 + (irq >> 3);
port = PIC_ELCR1 + (irq >> 3);
return (inb(port) >> (irq & 7)) & 1;
}

3
arch/x86/kernel/reboot.c
View File

@ -388,10 +388,11 @@ static const struct dmi_system_id reboot_dmi_table[] __initconst = {
},
{ /* Handle problems with rebooting on the OptiPlex 990. */
.callback = set_pci_reboot,
.ident = "Dell OptiPlex 990",
.ident = "Dell OptiPlex 990 BIOS A0x",
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
DMI_MATCH(DMI_PRODUCT_NAME, "OptiPlex 990"),
DMI_MATCH(DMI_BIOS_VERSION, "A0"),
},
},
{ /* Handle problems with rebooting on Dell 300's */

10
arch/x86/kernel/smpboot.c
View File

@ -610,6 +610,9 @@ void set_cpu_sibling_map(int cpu)
if (threads > __max_smt_threads)
__max_smt_threads = threads;
for_each_cpu(i, topology_sibling_cpumask(cpu))
cpu_data(i).smt_active = threads > 1;
/*
* This needs a separate iteration over the cpus because we rely on all
* topology_sibling_cpumask links to be set-up.
@ -1552,8 +1555,13 @@ static void remove_siblinginfo(int cpu)
for_each_cpu(sibling, topology_die_cpumask(cpu))
cpumask_clear_cpu(cpu, topology_die_cpumask(sibling));
for_each_cpu(sibling, topology_sibling_cpumask(cpu))
for_each_cpu(sibling, topology_sibling_cpumask(cpu)) {
cpumask_clear_cpu(cpu, topology_sibling_cpumask(sibling));
if (cpumask_weight(topology_sibling_cpumask(sibling)) == 1)
cpu_data(sibling).smt_active = false;
}
for_each_cpu(sibling, cpu_llc_shared_mask(cpu))
cpumask_clear_cpu(cpu, cpu_llc_shared_mask(sibling));
cpumask_clear(cpu_llc_shared_mask(cpu));

20
arch/x86/kvm/i8259.c
View File

@ -541,17 +541,17 @@ static int picdev_slave_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
addr, len, val);
}
static int picdev_eclr_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
static int picdev_elcr_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
gpa_t addr, int len, const void *val)
{
return picdev_write(container_of(dev, struct kvm_pic, dev_eclr),
return picdev_write(container_of(dev, struct kvm_pic, dev_elcr),
addr, len, val);
}
static int picdev_eclr_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
static int picdev_elcr_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
gpa_t addr, int len, void *val)
{
return picdev_read(container_of(dev, struct kvm_pic, dev_eclr),
return picdev_read(container_of(dev, struct kvm_pic, dev_elcr),
addr, len, val);
}
@ -577,9 +577,9 @@ static const struct kvm_io_device_ops picdev_slave_ops = {
.write = picdev_slave_write,
};
static const struct kvm_io_device_ops picdev_eclr_ops = {
.read = picdev_eclr_read,
.write = picdev_eclr_write,
static const struct kvm_io_device_ops picdev_elcr_ops = {
.read = picdev_elcr_read,
.write = picdev_elcr_write,
};
int kvm_pic_init(struct kvm *kvm)
@ -602,7 +602,7 @@ int kvm_pic_init(struct kvm *kvm)
*/
kvm_iodevice_init(&s->dev_master, &picdev_master_ops);
kvm_iodevice_init(&s->dev_slave, &picdev_slave_ops);
kvm_iodevice_init(&s->dev_eclr, &picdev_eclr_ops);
kvm_iodevice_init(&s->dev_elcr, &picdev_elcr_ops);
mutex_lock(&kvm->slots_lock);
ret = kvm_io_bus_register_dev(kvm, KVM_PIO_BUS, 0x20, 2,
&s->dev_master);
@ -613,7 +613,7 @@ int kvm_pic_init(struct kvm *kvm)
if (ret < 0)
goto fail_unreg_2;
ret = kvm_io_bus_register_dev(kvm, KVM_PIO_BUS, 0x4d0, 2, &s->dev_eclr);
ret = kvm_io_bus_register_dev(kvm, KVM_PIO_BUS, 0x4d0, 2, &s->dev_elcr);
if (ret < 0)
goto fail_unreg_1;
@ -647,7 +647,7 @@ void kvm_pic_destroy(struct kvm *kvm)
mutex_lock(&kvm->slots_lock);
kvm_io_bus_unregister_dev(vpic->kvm, KVM_PIO_BUS, &vpic->dev_master);
kvm_io_bus_unregister_dev(vpic->kvm, KVM_PIO_BUS, &vpic->dev_slave);
kvm_io_bus_unregister_dev(vpic->kvm, KVM_PIO_BUS, &vpic->dev_eclr);
kvm_io_bus_unregister_dev(vpic->kvm, KVM_PIO_BUS, &vpic->dev_elcr);
mutex_unlock(&kvm->slots_lock);
kvm->arch.vpic = NULL;

2
arch/x86/kvm/irq.h
View File

@ -55,7 +55,7 @@ struct kvm_pic {
int output; /* intr from master PIC */
struct kvm_io_device dev_master;
struct kvm_io_device dev_slave;
struct kvm_io_device dev_eclr;
struct kvm_io_device dev_elcr;
void (*ack_notifier)(void *opaque, int irq);
unsigned long irq_states[PIC_NUM_PINS];
};

1
arch/x86/lib/Makefile
View File

@ -44,6 +44,7 @@ obj-$(CONFIG_SMP) += msr-smp.o cache-smp.o
lib-y := delay.o misc.o cmdline.o cpu.o
lib-y += usercopy_$(BITS).o usercopy.o getuser.o putuser.o
lib-y += memcpy_$(BITS).o
lib-y += pc-conf-reg.o
lib-$(CONFIG_ARCH_HAS_COPY_MC) += copy_mc.o copy_mc_64.o
lib-$(CONFIG_INSTRUCTION_DECODER) += insn.o inat.o insn-eval.o
lib-$(CONFIG_RANDOMIZE_BASE) += kaslr.o

13
arch/x86/lib/pc-conf-reg.c Normal file
View File

@ -0,0 +1,13 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Support for the configuration register space at port I/O locations
* 0x22 and 0x23 variously used by PC architectures, e.g. the MP Spec,
* Cyrix CPUs, numerous chipsets. As the space is indirectly addressed
* it may have to be protected with a spinlock, depending on the context.
*/
#include <linux/spinlock.h>
#include <asm/pc-conf-reg.h>
DEFINE_RAW_SPINLOCK(pc_conf_lock);

113
arch/x86/mm/tlb.c
View File

@ -8,11 +8,13 @@
#include <linux/export.h>
#include <linux/cpu.h>
#include <linux/debugfs.h>
#include <linux/sched/smt.h>
#include <asm/tlbflush.h>
#include <asm/mmu_context.h>
#include <asm/nospec-branch.h>
#include <asm/cache.h>
#include <asm/cacheflush.h>
#include <asm/apic.h>
#include <asm/perf_event.h>
@ -43,10 +45,15 @@
*/
/*
* Use bit 0 to mangle the TIF_SPEC_IB state into the mm pointer which is
* stored in cpu_tlb_state.last_user_mm_ibpb.
* Bits to mangle the TIF_SPEC_* state into the mm pointer which is
* stored in cpu_tlb_state.last_user_mm_spec.
*/
#define LAST_USER_MM_IBPB 0x1UL
#define LAST_USER_MM_L1D_FLUSH 0x2UL
#define LAST_USER_MM_SPEC_MASK (LAST_USER_MM_IBPB | LAST_USER_MM_L1D_FLUSH)
/* Bits to set when tlbstate and flush is (re)initialized */
#define LAST_USER_MM_INIT LAST_USER_MM_IBPB
/*
* The x86 feature is called PCID (Process Context IDentifier). It is similar
@ -317,20 +324,70 @@ void switch_mm(struct mm_struct *prev, struct mm_struct *next,
local_irq_restore(flags);
}
static unsigned long mm_mangle_tif_spec_ib(struct task_struct *next)
/*
* Invoked from return to user/guest by a task that opted-in to L1D
* flushing but ended up running on an SMT enabled core due to wrong
* affinity settings or CPU hotplug. This is part of the paranoid L1D flush
* contract which this task requested.
*/
static void l1d_flush_force_sigbus(struct callback_head *ch)
{
unsigned long next_tif = task_thread_info(next)->flags;
unsigned long ibpb = (next_tif >> TIF_SPEC_IB) & LAST_USER_MM_IBPB;
return (unsigned long)next->mm | ibpb;
force_sig(SIGBUS);
}
static void cond_ibpb(struct task_struct *next)
static void l1d_flush_evaluate(unsigned long prev_mm, unsigned long next_mm,
struct task_struct *next)
{
if (!next || !next->mm)
/* Flush L1D if the outgoing task requests it */
if (prev_mm & LAST_USER_MM_L1D_FLUSH)
wrmsrl(MSR_IA32_FLUSH_CMD, L1D_FLUSH);
/* Check whether the incoming task opted in for L1D flush */
if (likely(!(next_mm & LAST_USER_MM_L1D_FLUSH)))
return;
/*
* Validate that it is not running on an SMT sibling as this would
* make the excercise pointless because the siblings share L1D. If
* it runs on a SMT sibling, notify it with SIGBUS on return to
* user/guest
*/
if (this_cpu_read(cpu_info.smt_active)) {
clear_ti_thread_flag(&next->thread_info, TIF_SPEC_L1D_FLUSH);
next->l1d_flush_kill.func = l1d_flush_force_sigbus;
task_work_add(next, &next->l1d_flush_kill, TWA_RESUME);
}
}
static unsigned long mm_mangle_tif_spec_bits(struct task_struct *next)
{
unsigned long next_tif = task_thread_info(next)->flags;
unsigned long spec_bits = (next_tif >> TIF_SPEC_IB) & LAST_USER_MM_SPEC_MASK;
/*
* Ensure that the bit shift above works as expected and the two flags
* end up in bit 0 and 1.
*/
BUILD_BUG_ON(TIF_SPEC_L1D_FLUSH != TIF_SPEC_IB + 1);
return (unsigned long)next->mm | spec_bits;
}
static void cond_mitigation(struct task_struct *next)
{
unsigned long prev_mm, next_mm;
if (!next || !next->mm)
return;
next_mm = mm_mangle_tif_spec_bits(next);
prev_mm = this_cpu_read(cpu_tlbstate.last_user_mm_spec);
/*
* Avoid user/user BTB poisoning by flushing the branch predictor
* when switching between processes. This stops one process from
* doing Spectre-v2 attacks on another.
*
* Both, the conditional and the always IBPB mode use the mm
* pointer to avoid the IBPB when switching between tasks of the
* same process. Using the mm pointer instead of mm->context.ctx_id
@ -340,8 +397,6 @@ static void cond_ibpb(struct task_struct *next)
* exposed data is not really interesting.
*/
if (static_branch_likely(&switch_mm_cond_ibpb)) {
unsigned long prev_mm, next_mm;
/*
* This is a bit more complex than the always mode because
* it has to handle two cases:
@ -371,20 +426,14 @@ static void cond_ibpb(struct task_struct *next)
* Optimize this with reasonably small overhead for the
* above cases. Mangle the TIF_SPEC_IB bit into the mm
* pointer of the incoming task which is stored in
* cpu_tlbstate.last_user_mm_ibpb for comparison.
*/
next_mm = mm_mangle_tif_spec_ib(next);
prev_mm = this_cpu_read(cpu_tlbstate.last_user_mm_ibpb);
/*
* cpu_tlbstate.last_user_mm_spec for comparison.
*
* Issue IBPB only if the mm's are different and one or
* both have the IBPB bit set.
*/
if (next_mm != prev_mm &&
(next_mm | prev_mm) & LAST_USER_MM_IBPB)
indirect_branch_prediction_barrier();
this_cpu_write(cpu_tlbstate.last_user_mm_ibpb, next_mm);
}
if (static_branch_unlikely(&switch_mm_always_ibpb)) {
@ -393,11 +442,22 @@ static void cond_ibpb(struct task_struct *next)
* different context than the user space task which ran
* last on this CPU.
*/
if (this_cpu_read(cpu_tlbstate.last_user_mm) != next->mm) {
if ((prev_mm & ~LAST_USER_MM_SPEC_MASK) !=
(unsigned long)next->mm)
indirect_branch_prediction_barrier();
this_cpu_write(cpu_tlbstate.last_user_mm, next->mm);
}
}
if (static_branch_unlikely(&switch_mm_cond_l1d_flush)) {
/*
* Flush L1D when the outgoing task requested it and/or
* check whether the incoming task requested L1D flushing
* and ended up on an SMT sibling.
*/
if (unlikely((prev_mm | next_mm) & LAST_USER_MM_L1D_FLUSH))
l1d_flush_evaluate(prev_mm, next_mm, next);
}
this_cpu_write(cpu_tlbstate.last_user_mm_spec, next_mm);
}
#ifdef CONFIG_PERF_EVENTS
@ -531,11 +591,10 @@ void switch_mm_irqs_off(struct mm_struct *prev, struct mm_struct *next,
need_flush = true;
} else {
/*
* Avoid user/user BTB poisoning by flushing the branch
* predictor when switching between processes. This stops
* one process from doing Spectre-v2 attacks on another.
* Apply process to process speculation vulnerability
* mitigations if applicable.
*/
cond_ibpb(tsk);
cond_mitigation(tsk);
/*
* Stop remote flushes for the previous mm.
@ -643,7 +702,7 @@ void initialize_tlbstate_and_flush(void)
write_cr3(build_cr3(mm->pgd, 0));
/* Reinitialize tlbstate. */
this_cpu_write(cpu_tlbstate.last_user_mm_ibpb, LAST_USER_MM_IBPB);
this_cpu_write(cpu_tlbstate.last_user_mm_spec, LAST_USER_MM_INIT);
this_cpu_write(cpu_tlbstate.loaded_mm_asid, 0);
this_cpu_write(cpu_tlbstate.next_asid, 1);
this_cpu_write(cpu_tlbstate.ctxs[0].ctx_id, mm->context.ctx_id);

279
arch/x86/pci/irq.c
View File

@ -13,9 +13,13 @@
#include <linux/dmi.h>
#include <linux/io.h>
#include <linux/smp.h>
#include <linux/spinlock.h>
#include <asm/io_apic.h>
#include <linux/irq.h>
#include <linux/acpi.h>
#include <asm/i8259.h>
#include <asm/pc-conf-reg.h>
#include <asm/pci_x86.h>
#define PIRQ_SIGNATURE (('$' << 0) + ('P' << 8) + ('I' << 16) + ('R' << 24))
@ -47,6 +51,8 @@ struct irq_router {
int (*get)(struct pci_dev *router, struct pci_dev *dev, int pirq);
int (*set)(struct pci_dev *router, struct pci_dev *dev, int pirq,
int new);
int (*lvl)(struct pci_dev *router, struct pci_dev *dev, int pirq,
int irq);
};
struct irq_router_handler {
@ -153,7 +159,7 @@ static void __init pirq_peer_trick(void)
void elcr_set_level_irq(unsigned int irq)
{
unsigned char mask = 1 << (irq & 7);
unsigned int port = 0x4d0 + (irq >> 3);
unsigned int port = PIC_ELCR1 + (irq >> 3);
unsigned char val;
static u16 elcr_irq_mask;
@ -169,6 +175,139 @@ void elcr_set_level_irq(unsigned int irq)
}
}
/*
* PIRQ routing for the M1487 ISA Bus Controller (IBC) ASIC used
* with the ALi FinALi 486 chipset. The IBC is not decoded in the
* PCI configuration space, so we identify it by the accompanying
* M1489 Cache-Memory PCI Controller (CMP) ASIC.
*
* There are four 4-bit mappings provided, spread across two PCI
* INTx Routing Table Mapping Registers, available in the port I/O
* space accessible indirectly via the index/data register pair at
* 0x22/0x23, located at indices 0x42 and 0x43 for the INT1/INT2
* and INT3/INT4 lines respectively. The INT1/INT3 and INT2/INT4
* lines are mapped in the low and the high 4-bit nibble of the
* corresponding register as follows:
*
* 0000 : Disabled
* 0001 : IRQ9
* 0010 : IRQ3
* 0011 : IRQ10
* 0100 : IRQ4
* 0101 : IRQ5
* 0110 : IRQ7
* 0111 : IRQ6
* 1000 : Reserved
* 1001 : IRQ11
* 1010 : Reserved
* 1011 : IRQ12
* 1100 : Reserved
* 1101 : IRQ14
* 1110 : Reserved
* 1111 : IRQ15
*
* In addition to the usual ELCR register pair there is a separate
* PCI INTx Sensitivity Register at index 0x44 in the same port I/O
* space, whose bits 3:0 select the trigger mode for INT[4:1] lines
* respectively. Any bit set to 1 causes interrupts coming on the
* corresponding line to be passed to ISA as edge-triggered and
* otherwise they are passed as level-triggered. Manufacturer's
* documentation says this register has to be set consistently with
* the relevant ELCR register.
*
* Accesses to the port I/O space concerned here need to be unlocked
* by writing the value of 0xc5 to the Lock Register at index 0x03
* beforehand. Any other value written to said register prevents
* further accesses from reaching the register file, except for the
* Lock Register being written with 0xc5 again.
*
* References:
*
* "M1489/M1487: 486 PCI Chip Set", Version 1.2, Acer Laboratories
* Inc., July 1997
*/
#define PC_CONF_FINALI_LOCK 0x03u
#define PC_CONF_FINALI_PCI_INTX_RT1 0x42u
#define PC_CONF_FINALI_PCI_INTX_RT2 0x43u
#define PC_CONF_FINALI_PCI_INTX_SENS 0x44u
#define PC_CONF_FINALI_LOCK_KEY 0xc5u
static u8 read_pc_conf_nybble(u8 base, u8 index)
{
u8 reg = base + (index >> 1);
u8 x;
x = pc_conf_get(reg);
return index & 1 ? x >> 4 : x & 0xf;
}
static void write_pc_conf_nybble(u8 base, u8 index, u8 val)
{
u8 reg = base + (index >> 1);
u8 x;
x = pc_conf_get(reg);
x = index & 1 ? (x & 0x0f) | (val << 4) : (x & 0xf0) | val;
pc_conf_set(reg, x);
}
static int pirq_finali_get(struct pci_dev *router, struct pci_dev *dev,
int pirq)
{
static const u8 irqmap[16] = {
0, 9, 3, 10, 4, 5, 7, 6, 0, 11, 0, 12, 0, 14, 0, 15
};
unsigned long flags;
u8 x;
raw_spin_lock_irqsave(&pc_conf_lock, flags);
pc_conf_set(PC_CONF_FINALI_LOCK, PC_CONF_FINALI_LOCK_KEY);
x = irqmap[read_pc_conf_nybble(PC_CONF_FINALI_PCI_INTX_RT1, pirq - 1)];
pc_conf_set(PC_CONF_FINALI_LOCK, 0);
raw_spin_unlock_irqrestore(&pc_conf_lock, flags);
return x;
}
static int pirq_finali_set(struct pci_dev *router, struct pci_dev *dev,
int pirq, int irq)
{
static const u8 irqmap[16] = {
0, 0, 0, 2, 4, 5, 7, 6, 0, 1, 3, 9, 11, 0, 13, 15
};
u8 val = irqmap[irq];
unsigned long flags;
if (!val)
return 0;
raw_spin_lock_irqsave(&pc_conf_lock, flags);
pc_conf_set(PC_CONF_FINALI_LOCK, PC_CONF_FINALI_LOCK_KEY);
write_pc_conf_nybble(PC_CONF_FINALI_PCI_INTX_RT1, pirq - 1, val);
pc_conf_set(PC_CONF_FINALI_LOCK, 0);
raw_spin_unlock_irqrestore(&pc_conf_lock, flags);
return 1;
}
static int pirq_finali_lvl(struct pci_dev *router, struct pci_dev *dev,
int pirq, int irq)
{
u8 mask = ~(1u << (pirq - 1));
unsigned long flags;
u8 trig;
elcr_set_level_irq(irq);
raw_spin_lock_irqsave(&pc_conf_lock, flags);
pc_conf_set(PC_CONF_FINALI_LOCK, PC_CONF_FINALI_LOCK_KEY);
trig = pc_conf_get(PC_CONF_FINALI_PCI_INTX_SENS);
trig &= mask;
pc_conf_set(PC_CONF_FINALI_PCI_INTX_SENS, trig);
pc_conf_set(PC_CONF_FINALI_LOCK, 0);
raw_spin_unlock_irqrestore(&pc_conf_lock, flags);
return 1;
}
/*
* Common IRQ routing practice: nibbles in config space,
* offset by some magic constant.
@ -219,6 +358,74 @@ static int pirq_ali_set(struct pci_dev *router, struct pci_dev *dev, int pirq, i
return 0;
}
/*
* PIRQ routing for the 82374EB/82374SB EISA System Component (ESC)
* ASIC used with the Intel 82420 and 82430 PCIsets. The ESC is not
* decoded in the PCI configuration space, so we identify it by the
* accompanying 82375EB/82375SB PCI-EISA Bridge (PCEB) ASIC.
*
* There are four PIRQ Route Control registers, available in the
* port I/O space accessible indirectly via the index/data register
* pair at 0x22/0x23, located at indices 0x60/0x61/0x62/0x63 for the
* PIRQ0/1/2/3# lines respectively. The semantics is the same as
* with the PIIX router.
*
* Accesses to the port I/O space concerned here need to be unlocked
* by writing the value of 0x0f to the ESC ID Register at index 0x02
* beforehand. Any other value written to said register prevents
* further accesses from reaching the register file, except for the
* ESC ID Register being written with 0x0f again.
*
* References:
*
* "82374EB/82374SB EISA System Component (ESC)", Intel Corporation,
* Order Number: 290476-004, March 1996
*
* "82375EB/82375SB PCI-EISA Bridge (PCEB)", Intel Corporation, Order
* Number: 290477-004, March 1996
*/
#define PC_CONF_I82374_ESC_ID 0x02u
#define PC_CONF_I82374_PIRQ_ROUTE_CONTROL 0x60u
#define PC_CONF_I82374_ESC_ID_KEY 0x0fu
static int pirq_esc_get(struct pci_dev *router, struct pci_dev *dev, int pirq)
{
unsigned long flags;
int reg;
u8 x;
reg = pirq;
if (reg >= 1 && reg <= 4)
reg += PC_CONF_I82374_PIRQ_ROUTE_CONTROL - 1;
raw_spin_lock_irqsave(&pc_conf_lock, flags);
pc_conf_set(PC_CONF_I82374_ESC_ID, PC_CONF_I82374_ESC_ID_KEY);
x = pc_conf_get(reg);
pc_conf_set(PC_CONF_I82374_ESC_ID, 0);
raw_spin_unlock_irqrestore(&pc_conf_lock, flags);
return (x < 16) ? x : 0;
}
static int pirq_esc_set(struct pci_dev *router, struct pci_dev *dev, int pirq,
int irq)
{
unsigned long flags;
int reg;
reg = pirq;
if (reg >= 1 && reg <= 4)
reg += PC_CONF_I82374_PIRQ_ROUTE_CONTROL - 1;
raw_spin_lock_irqsave(&pc_conf_lock, flags);
pc_conf_set(PC_CONF_I82374_ESC_ID, PC_CONF_I82374_ESC_ID_KEY);
pc_conf_set(reg, irq);
pc_conf_set(PC_CONF_I82374_ESC_ID, 0);
raw_spin_unlock_irqrestore(&pc_conf_lock, flags);
return 1;
}
/*
* The Intel PIIX4 pirq rules are fairly simple: "pirq" is
* just a pointer to the config space.
@ -237,6 +444,50 @@ static int pirq_piix_set(struct pci_dev *router, struct pci_dev *dev, int pirq,
return 1;
}
/*
* PIRQ routing for the 82426EX ISA Bridge (IB) ASIC used with the
* Intel 82420EX PCIset.
*
* There are only two PIRQ Route Control registers, available in the
* combined 82425EX/82426EX PCI configuration space, at 0x66 and 0x67
* for the PIRQ0# and PIRQ1# lines respectively. The semantics is
* the same as with the PIIX router.
*
* References:
*
* "82420EX PCIset Data Sheet, 82425EX PCI System Controller (PSC)
* and 82426EX ISA Bridge (IB)", Intel Corporation, Order Number:
* 290488-004, December 1995
*/
#define PCI_I82426EX_PIRQ_ROUTE_CONTROL 0x66u
static int pirq_ib_get(struct pci_dev *router, struct pci_dev *dev, int pirq)
{
int reg;
u8 x;
reg = pirq;
if (reg >= 1 && reg <= 2)
reg += PCI_I82426EX_PIRQ_ROUTE_CONTROL - 1;
pci_read_config_byte(router, reg, &x);
return (x < 16) ? x : 0;
}
static int pirq_ib_set(struct pci_dev *router, struct pci_dev *dev, int pirq,
int irq)
{
int reg;
reg = pirq;
if (reg >= 1 && reg <= 2)
reg += PCI_I82426EX_PIRQ_ROUTE_CONTROL - 1;
pci_write_config_byte(router, reg, irq);
return 1;
}
/*
* The VIA pirq rules are nibble-based, like ALI,
* but without the ugly irq number munging.
@ -549,6 +800,11 @@ static __init int intel_router_probe(struct irq_router *r, struct pci_dev *route
return 0;
switch (device) {
case PCI_DEVICE_ID_INTEL_82375:
r->name = "PCEB/ESC";
r->get = pirq_esc_get;
r->set = pirq_esc_set;
return 1;
case PCI_DEVICE_ID_INTEL_82371FB_0:
case PCI_DEVICE_ID_INTEL_82371SB_0:
case PCI_DEVICE_ID_INTEL_82371AB_0:
@ -594,6 +850,11 @@ static __init int intel_router_probe(struct irq_router *r, struct pci_dev *route
r->get = pirq_piix_get;
r->set = pirq_piix_set;
return 1;
case PCI_DEVICE_ID_INTEL_82425:
r->name = "PSC/IB";
r->get = pirq_ib_get;
r->set = pirq_ib_set;
return 1;
}
if ((device >= PCI_DEVICE_ID_INTEL_5_3400_SERIES_LPC_MIN &&
@ -745,6 +1006,12 @@ static __init int ite_router_probe(struct irq_router *r, struct pci_dev *router,
static __init int ali_router_probe(struct irq_router *r, struct pci_dev *router, u16 device)
{
switch (device) {
case PCI_DEVICE_ID_AL_M1489:
r->name = "FinALi";
r->get = pirq_finali_get;
r->set = pirq_finali_set;
r->lvl = pirq_finali_lvl;
return 1;
case PCI_DEVICE_ID_AL_M1533:
case PCI_DEVICE_ID_AL_M1563:
r->name = "ALI";
@ -968,11 +1235,17 @@ static int pcibios_lookup_irq(struct pci_dev *dev, int assign)
} else if (r->get && (irq = r->get(pirq_router_dev, dev, pirq)) && \
((!(pci_probe & PCI_USE_PIRQ_MASK)) || ((1 << irq) & mask))) {
msg = "found";
elcr_set_level_irq(irq);
if (r->lvl)
r->lvl(pirq_router_dev, dev, pirq, irq);
else
elcr_set_level_irq(irq);
} else if (newirq && r->set &&
(dev->class >> 8) != PCI_CLASS_DISPLAY_VGA) {
if (r->set(pirq_router_dev, dev, pirq, newirq)) {
elcr_set_level_irq(newirq);
if (r->lvl)
r->lvl(pirq_router_dev, dev, pirq, newirq);
else
elcr_set_level_irq(newirq);
msg = "assigned";
irq = newirq;
}

14
block/Kconfig
View File

@ -114,16 +114,6 @@ config BLK_DEV_THROTTLING_LOW
Note, this is an experimental interface and could be changed someday.
config BLK_CMDLINE_PARSER
bool "Block device command line partition parser"
help
Enabling this option allows you to specify the partition layout from
the kernel boot args. This is typically of use for embedded devices
which don't otherwise have any standardized method for listing the
partitions on a block device.
See Documentation/block/cmdline-partition.rst for more information.
config BLK_WBT
bool "Enable support for block device writeback throttling"
help
@ -251,4 +241,8 @@ config BLK_MQ_RDMA
config BLK_PM
def_bool BLOCK && PM
# do not use in new code
config BLOCK_HOLDER_DEPRECATED
bool
source "block/Kconfig.iosched"

2
block/Makefile
View File

@ -26,7 +26,6 @@ obj-$(CONFIG_MQ_IOSCHED_KYBER) += kyber-iosched.o
bfq-y := bfq-iosched.o bfq-wf2q.o bfq-cgroup.o
obj-$(CONFIG_IOSCHED_BFQ) += bfq.o
obj-$(CONFIG_BLK_CMDLINE_PARSER) += cmdline-parser.o
obj-$(CONFIG_BLK_DEV_INTEGRITY) += bio-integrity.o blk-integrity.o
obj-$(CONFIG_BLK_DEV_INTEGRITY_T10) += t10-pi.o
obj-$(CONFIG_BLK_MQ_PCI) += blk-mq-pci.o
@ -40,3 +39,4 @@ obj-$(CONFIG_BLK_SED_OPAL) += sed-opal.o
obj-$(CONFIG_BLK_PM) += blk-pm.o
obj-$(CONFIG_BLK_INLINE_ENCRYPTION) += keyslot-manager.o blk-crypto.o
obj-$(CONFIG_BLK_INLINE_ENCRYPTION_FALLBACK) += blk-crypto-fallback.o
obj-$(CONFIG_BLOCK_HOLDER_DEPRECATED) += holder.o

17
block/bfq-iosched.c
View File

@ -2361,6 +2361,9 @@ static int bfq_request_merge(struct request_queue *q, struct request **req,
__rq = bfq_find_rq_fmerge(bfqd, bio, q);
if (__rq && elv_bio_merge_ok(__rq, bio)) {
*req = __rq;
if (blk_discard_mergable(__rq))
return ELEVATOR_DISCARD_MERGE;
return ELEVATOR_FRONT_MERGE;
}
@ -2505,7 +2508,7 @@ void bfq_end_wr_async_queues(struct bfq_data *bfqd,
int i, j;
for (i = 0; i < 2; i++)
for (j = 0; j < IOPRIO_BE_NR; j++)
for (j = 0; j < IOPRIO_NR_LEVELS; j++)
if (bfqg->async_bfqq[i][j])
bfq_bfqq_end_wr(bfqg->async_bfqq[i][j]);
if (bfqg->async_idle_bfqq)
@ -5266,8 +5269,8 @@ bfq_set_next_ioprio_data(struct bfq_queue *bfqq, struct bfq_io_cq *bic)
switch (ioprio_class) {
default:
pr_err("bdi %s: bfq: bad prio class %d\n",
bdi_dev_name(bfqq->bfqd->queue->backing_dev_info),
ioprio_class);
bdi_dev_name(bfqq->bfqd->queue->disk->bdi),
ioprio_class);
fallthrough;
case IOPRIO_CLASS_NONE:
/*
@ -5290,10 +5293,10 @@ bfq_set_next_ioprio_data(struct bfq_queue *bfqq, struct bfq_io_cq *bic)
break;
}
if (bfqq->new_ioprio >= IOPRIO_BE_NR) {
if (bfqq->new_ioprio >= IOPRIO_NR_LEVELS) {
pr_crit("bfq_set_next_ioprio_data: new_ioprio %d\n",
bfqq->new_ioprio);
bfqq->new_ioprio = IOPRIO_BE_NR;
bfqq->new_ioprio = IOPRIO_NR_LEVELS - 1;
}
bfqq->entity.new_weight = bfq_ioprio_to_weight(bfqq->new_ioprio);
@ -5408,7 +5411,7 @@ static struct bfq_queue **bfq_async_queue_prio(struct bfq_data *bfqd,
case IOPRIO_CLASS_RT:
return &bfqg->async_bfqq[0][ioprio];
case IOPRIO_CLASS_NONE:
ioprio = IOPRIO_NORM;
ioprio = IOPRIO_BE_NORM;
fallthrough;
case IOPRIO_CLASS_BE:
return &bfqg->async_bfqq[1][ioprio];
@ -6822,7 +6825,7 @@ void bfq_put_async_queues(struct bfq_data *bfqd, struct bfq_group *bfqg)
int i, j;
for (i = 0; i < 2; i++)
for (j = 0; j < IOPRIO_BE_NR; j++)
for (j = 0; j < IOPRIO_NR_LEVELS; j++)
__bfq_put_async_bfqq(bfqd, &bfqg->async_bfqq[i][j]);
__bfq_put_async_bfqq(bfqd, &bfqg->async_idle_bfqq);

6
block/bfq-iosched.h
View File

@ -931,7 +931,7 @@ struct bfq_group {
void *bfqd;
struct bfq_queue *async_bfqq[2][IOPRIO_BE_NR];
struct bfq_queue *async_bfqq[2][IOPRIO_NR_LEVELS];
struct bfq_queue *async_idle_bfqq;
struct bfq_entity *my_entity;
@ -948,15 +948,13 @@ struct bfq_group {
struct bfq_entity entity;
struct bfq_sched_data sched_data;
struct bfq_queue *async_bfqq[2][IOPRIO_BE_NR];
struct bfq_queue *async_bfqq[2][IOPRIO_NR_LEVELS];
struct bfq_queue *async_idle_bfqq;
struct rb_root rq_pos_tree;
};
#endif
struct bfq_queue *bfq_entity_to_bfqq(struct bfq_entity *entity);
/* --------------- main algorithm interface ----------------- */
#define BFQ_SERVICE_TREE_INIT ((struct bfq_service_tree) \

6
block/bfq-wf2q.c
View File

@ -505,7 +505,7 @@ static void bfq_active_insert(struct bfq_service_tree *st,
*/
unsigned short bfq_ioprio_to_weight(int ioprio)
{
return (IOPRIO_BE_NR - ioprio) * BFQ_WEIGHT_CONVERSION_COEFF;
return (IOPRIO_NR_LEVELS - ioprio) * BFQ_WEIGHT_CONVERSION_COEFF;
}
/**
@ -514,12 +514,12 @@ unsigned short bfq_ioprio_to_weight(int ioprio)
*
* To preserve as much as possible the old only-ioprio user interface,
* 0 is used as an escape ioprio value for weights (numerically) equal or
* larger than IOPRIO_BE_NR * BFQ_WEIGHT_CONVERSION_COEFF.
* larger than IOPRIO_NR_LEVELS * BFQ_WEIGHT_CONVERSION_COEFF.
*/
static unsigned short bfq_weight_to_ioprio(int weight)
{
return max_t(int, 0,
IOPRIO_BE_NR * BFQ_WEIGHT_CONVERSION_COEFF - weight);
IOPRIO_NR_LEVELS * BFQ_WEIGHT_CONVERSION_COEFF - weight);
}
static void bfq_get_entity(struct bfq_entity *entity)

21
block/bio-integrity.c
View File

@ -104,8 +104,7 @@ void bio_integrity_free(struct bio *bio)
struct bio_set *bs = bio->bi_pool;
if (bip->bip_flags & BIP_BLOCK_INTEGRITY)
kfree(page_address(bip->bip_vec->bv_page) +
bip->bip_vec->bv_offset);
kfree(bvec_virt(bip->bip_vec));
__bio_integrity_free(bs, bip);
bio->bi_integrity = NULL;
@ -163,27 +162,23 @@ static blk_status_t bio_integrity_process(struct bio *bio,
struct bio_vec bv;
struct bio_integrity_payload *bip = bio_integrity(bio);
blk_status_t ret = BLK_STS_OK;
void *prot_buf = page_address(bip->bip_vec->bv_page) +
bip->bip_vec->bv_offset;
iter.disk_name = bio->bi_bdev->bd_disk->disk_name;
iter.interval = 1 << bi->interval_exp;
iter.seed = proc_iter->bi_sector;
iter.prot_buf = prot_buf;
iter.prot_buf = bvec_virt(bip->bip_vec);
__bio_for_each_segment(bv, bio, bviter, *proc_iter) {
void *kaddr = kmap_atomic(bv.bv_page);
void *kaddr = bvec_kmap_local(&bv);
iter.data_buf = kaddr + bv.bv_offset;
iter.data_buf = kaddr;
iter.data_size = bv.bv_len;
ret = proc_fn(&iter);
if (ret) {
kunmap_atomic(kaddr);
return ret;
}
kunmap_local(kaddr);
if (ret)
break;
kunmap_atomic(kaddr);
}
return ret;
}

50
block/bio.c
View File

@ -495,16 +495,11 @@ EXPORT_SYMBOL(bio_kmalloc);
void zero_fill_bio(struct bio *bio)
{
unsigned long flags;
struct bio_vec bv;
struct bvec_iter iter;
bio_for_each_segment(bv, bio, iter) {
char *data = bvec_kmap_irq(&bv, &flags);
memset(data, 0, bv.bv_len);
flush_dcache_page(bv.bv_page);
bvec_kunmap_irq(data, &flags);
}
bio_for_each_segment(bv, bio, iter)
memzero_bvec(&bv);
}
EXPORT_SYMBOL(zero_fill_bio);
@ -979,6 +974,14 @@ static int bio_iov_bvec_set_append(struct bio *bio, struct iov_iter *iter)
return 0;
}
static void bio_put_pages(struct page **pages, size_t size, size_t off)
{
size_t i, nr = DIV_ROUND_UP(size + (off & ~PAGE_MASK), PAGE_SIZE);
for (i = 0; i < nr; i++)
put_page(pages[i]);
}
#define PAGE_PTRS_PER_BVEC (sizeof(struct bio_vec) / sizeof(struct page *))
/**
@ -1023,8 +1026,10 @@ static int __bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter)
if (same_page)
put_page(page);
} else {
if (WARN_ON_ONCE(bio_full(bio, len)))
return -EINVAL;
if (WARN_ON_ONCE(bio_full(bio, len))) {
bio_put_pages(pages + i, left, offset);
return -EINVAL;
}
__bio_add_page(bio, page, len, offset);
}
offset = 0;
@ -1069,6 +1074,7 @@ static int __bio_iov_append_get_pages(struct bio *bio, struct iov_iter *iter)
len = min_t(size_t, PAGE_SIZE - offset, left);
if (bio_add_hw_page(q, bio, page, len, offset,
max_append_sectors, &same_page) != len) {
bio_put_pages(pages + i, left, offset);
ret = -EINVAL;
break;
}
@ -1191,27 +1197,15 @@ EXPORT_SYMBOL(bio_advance);
void bio_copy_data_iter(struct bio *dst, struct bvec_iter *dst_iter,
struct bio *src, struct bvec_iter *src_iter)
{
struct bio_vec src_bv, dst_bv;
void *src_p, *dst_p;
unsigned bytes;
while (src_iter->bi_size && dst_iter->bi_size) {
src_bv = bio_iter_iovec(src, *src_iter);
dst_bv = bio_iter_iovec(dst, *dst_iter);
struct bio_vec src_bv = bio_iter_iovec(src, *src_iter);
struct bio_vec dst_bv = bio_iter_iovec(dst, *dst_iter);
unsigned int bytes = min(src_bv.bv_len, dst_bv.bv_len);
void *src_buf;
bytes = min(src_bv.bv_len, dst_bv.bv_len);
src_p = kmap_atomic(src_bv.bv_page);
dst_p = kmap_atomic(dst_bv.bv_page);
memcpy(dst_p + dst_bv.bv_offset,
src_p + src_bv.bv_offset,
bytes);
kunmap_atomic(dst_p);
kunmap_atomic(src_p);
flush_dcache_page(dst_bv.bv_page);
src_buf = bvec_kmap_local(&src_bv);
memcpy_to_bvec(&dst_bv, src_buf);
kunmap_local(src_buf);
bio_advance_iter_single(src, src_iter, bytes);
bio_advance_iter_single(dst, dst_iter, bytes);

141
block/blk-cgroup.c
View File

@ -489,10 +489,9 @@ static int blkcg_reset_stats(struct cgroup_subsys_state *css,
const char *blkg_dev_name(struct blkcg_gq *blkg)
{
/* some drivers (floppy) instantiate a queue w/o disk registered */
if (blkg->q->backing_dev_info->dev)
return bdi_dev_name(blkg->q->backing_dev_info);
return NULL;
if (!blkg->q->disk || !blkg->q->disk->bdi->dev)
return NULL;
return bdi_dev_name(blkg->q->disk->bdi);
}
/**
@ -873,6 +872,63 @@ static void blkcg_fill_root_iostats(void)
}
}
static void blkcg_print_one_stat(struct blkcg_gq *blkg, struct seq_file *s)
{
struct blkg_iostat_set *bis = &blkg->iostat;
u64 rbytes, wbytes, rios, wios, dbytes, dios;
bool has_stats = false;
const char *dname;
unsigned seq;
int i;
if (!blkg->online)
return;
dname = blkg_dev_name(blkg);
if (!dname)
return;
seq_printf(s, "%s ", dname);
do {
seq = u64_stats_fetch_begin(&bis->sync);
rbytes = bis->cur.bytes[BLKG_IOSTAT_READ];
wbytes = bis->cur.bytes[BLKG_IOSTAT_WRITE];
dbytes = bis->cur.bytes[BLKG_IOSTAT_DISCARD];
rios = bis->cur.ios[BLKG_IOSTAT_READ];
wios = bis->cur.ios[BLKG_IOSTAT_WRITE];
dios = bis->cur.ios[BLKG_IOSTAT_DISCARD];
} while (u64_stats_fetch_retry(&bis->sync, seq));
if (rbytes || wbytes || rios || wios) {
has_stats = true;
seq_printf(s, "rbytes=%llu wbytes=%llu rios=%llu wios=%llu dbytes=%llu dios=%llu",
rbytes, wbytes, rios, wios,
dbytes, dios);
}
if (blkcg_debug_stats && atomic_read(&blkg->use_delay)) {
has_stats = true;
seq_printf(s, " use_delay=%d delay_nsec=%llu",
atomic_read(&blkg->use_delay),
atomic64_read(&blkg->delay_nsec));
}
for (i = 0; i < BLKCG_MAX_POLS; i++) {
struct blkcg_policy *pol = blkcg_policy[i];
if (!blkg->pd[i] || !pol->pd_stat_fn)
continue;
if (pol->pd_stat_fn(blkg->pd[i], s))
has_stats = true;
}
if (has_stats)
seq_printf(s, "\n");
}
static int blkcg_print_stat(struct seq_file *sf, void *v)
{
struct blkcg *blkcg = css_to_blkcg(seq_css(sf));
@ -884,86 +940,11 @@ static int blkcg_print_stat(struct seq_file *sf, void *v)
cgroup_rstat_flush(blkcg->css.cgroup);
rcu_read_lock();
hlist_for_each_entry_rcu(blkg, &blkcg->blkg_list, blkcg_node) {
struct blkg_iostat_set *bis = &blkg->iostat;
const char *dname;
char *buf;
u64 rbytes, wbytes, rios, wios, dbytes, dios;
size_t size = seq_get_buf(sf, &buf), off = 0;
int i;
bool has_stats = false;
unsigned seq;
spin_lock_irq(&blkg->q->queue_lock);
if (!blkg->online)
goto skip;
dname = blkg_dev_name(blkg);
if (!dname)
goto skip;
/*
* Hooray string manipulation, count is the size written NOT
* INCLUDING THE \0, so size is now count+1 less than what we
* had before, but we want to start writing the next bit from
* the \0 so we only add count to buf.
*/
off += scnprintf(buf+off, size-off, "%s ", dname);
do {
seq = u64_stats_fetch_begin(&bis->sync);
rbytes = bis->cur.bytes[BLKG_IOSTAT_READ];
wbytes = bis->cur.bytes[BLKG_IOSTAT_WRITE];
dbytes = bis->cur.bytes[BLKG_IOSTAT_DISCARD];
rios = bis->cur.ios[BLKG_IOSTAT_READ];
wios = bis->cur.ios[BLKG_IOSTAT_WRITE];
dios = bis->cur.ios[BLKG_IOSTAT_DISCARD];
} while (u64_stats_fetch_retry(&bis->sync, seq));
if (rbytes || wbytes || rios || wios) {
has_stats = true;
off += scnprintf(buf+off, size-off,
"rbytes=%llu wbytes=%llu rios=%llu wios=%llu dbytes=%llu dios=%llu",
rbytes, wbytes, rios, wios,
dbytes, dios);
}
if (blkcg_debug_stats && atomic_read(&blkg->use_delay)) {
has_stats = true;
off += scnprintf(buf+off, size-off,
" use_delay=%d delay_nsec=%llu",
atomic_read(&blkg->use_delay),
(unsigned long long)atomic64_read(&blkg->delay_nsec));
}
for (i = 0; i < BLKCG_MAX_POLS; i++) {
struct blkcg_policy *pol = blkcg_policy[i];
size_t written;
if (!blkg->pd[i] || !pol->pd_stat_fn)
continue;
written = pol->pd_stat_fn(blkg->pd[i], buf+off, size-off);
if (written)
has_stats = true;
off += written;
}
if (has_stats) {
if (off < size - 1) {
off += scnprintf(buf+off, size-off, "\n");
seq_commit(sf, off);
} else {
seq_commit(sf, -1);
}
}
skip:
blkcg_print_one_stat(blkg, sf);
spin_unlock_irq(&blkg->q->queue_lock);
}
rcu_read_unlock();
return 0;
}

18
block/blk-core.c
View File

@ -14,7 +14,6 @@
*/
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/backing-dev.h>
#include <linux/bio.h>
#include <linux/blkdev.h>
#include <linux/blk-mq.h>
@ -393,10 +392,7 @@ void blk_cleanup_queue(struct request_queue *q)
/* for synchronous bio-based driver finish in-flight integrity i/o */
blk_flush_integrity();
/* @q won't process any more request, flush async actions */
del_timer_sync(&q->backing_dev_info->laptop_mode_wb_timer);
blk_sync_queue(q);
if (queue_is_mq(q))
blk_mq_exit_queue(q);
@ -533,20 +529,14 @@ struct request_queue *blk_alloc_queue(int node_id)
if (ret)
goto fail_id;
q->backing_dev_info = bdi_alloc(node_id);
if (!q->backing_dev_info)
goto fail_split;
q->stats = blk_alloc_queue_stats();
if (!q->stats)
goto fail_stats;
goto fail_split;
q->node = node_id;
atomic_set(&q->nr_active_requests_shared_sbitmap, 0);
timer_setup(&q->backing_dev_info->laptop_mode_wb_timer,
laptop_mode_timer_fn, 0);
timer_setup(&q->timeout, blk_rq_timed_out_timer, 0);
INIT_WORK(&q->timeout_work, blk_timeout_work);
INIT_LIST_HEAD(&q->icq_list);
@ -571,7 +561,7 @@ struct request_queue *blk_alloc_queue(int node_id)
if (percpu_ref_init(&q->q_usage_counter,
blk_queue_usage_counter_release,
PERCPU_REF_INIT_ATOMIC, GFP_KERNEL))
goto fail_bdi;
goto fail_stats;
if (blkcg_init_queue(q))
goto fail_ref;
@ -584,10 +574,8 @@ struct request_queue *blk_alloc_queue(int node_id)
fail_ref:
percpu_ref_exit(&q->q_usage_counter);
fail_bdi:
blk_free_queue_stats(q->stats);
fail_stats:
bdi_put(q->backing_dev_info);
blk_free_queue_stats(q->stats);
fail_split:
bioset_exit(&q->bio_split);
fail_id:

2
block/blk-crypto.c
View File

@ -348,7 +348,7 @@ int blk_crypto_init_key(struct blk_crypto_key *blk_key,
return -EINVAL;
}
if (dun_bytes == 0 || dun_bytes > BLK_CRYPTO_MAX_IV_SIZE)
if (dun_bytes == 0 || dun_bytes > mode->ivsize)
return -EINVAL;
if (!is_power_of_2(data_unit_size))

12
block/blk-integrity.c
View File

@ -431,13 +431,15 @@ void blk_integrity_unregister(struct gendisk *disk)
}
EXPORT_SYMBOL(blk_integrity_unregister);
void blk_integrity_add(struct gendisk *disk)
int blk_integrity_add(struct gendisk *disk)
{
if (kobject_init_and_add(&disk->integrity_kobj, &integrity_ktype,
&disk_to_dev(disk)->kobj, "%s", "integrity"))
return;
int ret;
kobject_uevent(&disk->integrity_kobj, KOBJ_ADD);
ret = kobject_init_and_add(&disk->integrity_kobj, &integrity_ktype,
&disk_to_dev(disk)->kobj, "%s", "integrity");
if (!ret)
kobject_uevent(&disk->integrity_kobj, KOBJ_ADD);
return ret;
}
void blk_integrity_del(struct gendisk *disk)

23
block/blk-iocost.c
View File

@ -2988,34 +2988,29 @@ static void ioc_pd_free(struct blkg_policy_data *pd)
kfree(iocg);
}
static size_t ioc_pd_stat(struct blkg_policy_data *pd, char *buf, size_t size)
static bool ioc_pd_stat(struct blkg_policy_data *pd, struct seq_file *s)
{
struct ioc_gq *iocg = pd_to_iocg(pd);
struct ioc *ioc = iocg->ioc;
size_t pos = 0;
if (!ioc->enabled)
return 0;
return false;
if (iocg->level == 0) {
unsigned vp10k = DIV64_U64_ROUND_CLOSEST(
ioc->vtime_base_rate * 10000,
VTIME_PER_USEC);
pos += scnprintf(buf + pos, size - pos, " cost.vrate=%u.%02u",
vp10k / 100, vp10k % 100);
seq_printf(s, " cost.vrate=%u.%02u", vp10k / 100, vp10k % 100);
}
pos += scnprintf(buf + pos, size - pos, " cost.usage=%llu",
iocg->last_stat.usage_us);
seq_printf(s, " cost.usage=%llu", iocg->last_stat.usage_us);
if (blkcg_debug_stats)
pos += scnprintf(buf + pos, size - pos,
" cost.wait=%llu cost.indebt=%llu cost.indelay=%llu",
iocg->last_stat.wait_us,
iocg->last_stat.indebt_us,
iocg->last_stat.indelay_us);
return pos;
seq_printf(s, " cost.wait=%llu cost.indebt=%llu cost.indelay=%llu",
iocg->last_stat.wait_us,
iocg->last_stat.indebt_us,
iocg->last_stat.indelay_us);
return true;
}
static u64 ioc_weight_prfill(struct seq_file *sf, struct blkg_policy_data *pd,

38
block/blk-iolatency.c
View File

@ -890,8 +890,7 @@ static int iolatency_print_limit(struct seq_file *sf, void *v)
return 0;
}
static size_t iolatency_ssd_stat(struct iolatency_grp *iolat, char *buf,
size_t size)
static bool iolatency_ssd_stat(struct iolatency_grp *iolat, struct seq_file *s)
{
struct latency_stat stat;
int cpu;
@ -906,39 +905,40 @@ static size_t iolatency_ssd_stat(struct iolatency_grp *iolat, char *buf,
preempt_enable();
if (iolat->rq_depth.max_depth == UINT_MAX)
return scnprintf(buf, size, " missed=%llu total=%llu depth=max",
(unsigned long long)stat.ps.missed,
(unsigned long long)stat.ps.total);
return scnprintf(buf, size, " missed=%llu total=%llu depth=%u",
(unsigned long long)stat.ps.missed,
(unsigned long long)stat.ps.total,
iolat->rq_depth.max_depth);
seq_printf(s, " missed=%llu total=%llu depth=max",
(unsigned long long)stat.ps.missed,
(unsigned long long)stat.ps.total);
else
seq_printf(s, " missed=%llu total=%llu depth=%u",
(unsigned long long)stat.ps.missed,
(unsigned long long)stat.ps.total,
iolat->rq_depth.max_depth);
return true;
}
static size_t iolatency_pd_stat(struct blkg_policy_data *pd, char *buf,
size_t size)
static bool iolatency_pd_stat(struct blkg_policy_data *pd, struct seq_file *s)
{
struct iolatency_grp *iolat = pd_to_lat(pd);
unsigned long long avg_lat;
unsigned long long cur_win;
if (!blkcg_debug_stats)
return 0;
return false;
if (iolat->ssd)
return iolatency_ssd_stat(iolat, buf, size);
return iolatency_ssd_stat(iolat, s);
avg_lat = div64_u64(iolat->lat_avg, NSEC_PER_USEC);
cur_win = div64_u64(iolat->cur_win_nsec, NSEC_PER_MSEC);
if (iolat->rq_depth.max_depth == UINT_MAX)
return scnprintf(buf, size, " depth=max avg_lat=%llu win=%llu",
avg_lat, cur_win);
return scnprintf(buf, size, " depth=%u avg_lat=%llu win=%llu",
iolat->rq_depth.max_depth, avg_lat, cur_win);
seq_printf(s, " depth=max avg_lat=%llu win=%llu",
avg_lat, cur_win);
else
seq_printf(s, " depth=%u avg_lat=%llu win=%llu",
iolat->rq_depth.max_depth, avg_lat, cur_win);
return true;
}
static struct blkg_policy_data *iolatency_pd_alloc(gfp_t gfp,
struct request_queue *q,
struct blkcg *blkcg)

2
block/blk-map.c
View File

@ -400,7 +400,7 @@ static void bio_copy_kern_endio_read(struct bio *bio)
struct bvec_iter_all iter_all;
bio_for_each_segment_all(bvec, bio, iter_all) {
memcpy(p, page_address(bvec->bv_page), bvec->bv_len);
memcpy_from_bvec(p, bvec);
p += bvec->bv_len;
}

18
block/blk-merge.c
View File

@ -348,6 +348,8 @@ void __blk_queue_split(struct bio **bio, unsigned int *nr_segs)
trace_block_split(split, (*bio)->bi_iter.bi_sector);
submit_bio_noacct(*bio);
*bio = split;
blk_throtl_charge_bio_split(*bio);
}
}
@ -705,22 +707,6 @@ static void blk_account_io_merge_request(struct request *req)
}
}
/*
* Two cases of handling DISCARD merge:
* If max_discard_segments > 1, the driver takes every bio
* as a range and send them to controller together. The ranges
* needn't to be contiguous.
* Otherwise, the bios/requests will be handled as same as
* others which should be contiguous.
*/
static inline bool blk_discard_mergable(struct request *req)
{
if (req_op(req) == REQ_OP_DISCARD &&
queue_max_discard_segments(req->q) > 1)
return true;
return false;
}
static enum elv_merge blk_try_req_merge(struct request *req,
struct request *next)
{

55
block/blk-mq-sysfs.c
View File

@ -45,60 +45,12 @@ static void blk_mq_hw_sysfs_release(struct kobject *kobj)
kfree(hctx);
}
struct blk_mq_ctx_sysfs_entry {
struct attribute attr;
ssize_t (*show)(struct blk_mq_ctx *, char *);
ssize_t (*store)(struct blk_mq_ctx *, const char *, size_t);
};
struct blk_mq_hw_ctx_sysfs_entry {
struct attribute attr;
ssize_t (*show)(struct blk_mq_hw_ctx *, char *);
ssize_t (*store)(struct blk_mq_hw_ctx *, const char *, size_t);
};
static ssize_t blk_mq_sysfs_show(struct kobject *kobj, struct attribute *attr,
char *page)
{
struct blk_mq_ctx_sysfs_entry *entry;
struct blk_mq_ctx *ctx;
struct request_queue *q;
ssize_t res;
entry = container_of(attr, struct blk_mq_ctx_sysfs_entry, attr);
ctx = container_of(kobj, struct blk_mq_ctx, kobj);
q = ctx->queue;
if (!entry->show)
return -EIO;
mutex_lock(&q->sysfs_lock);
res = entry->show(ctx, page);
mutex_unlock(&q->sysfs_lock);
return res;
}
static ssize_t blk_mq_sysfs_store(struct kobject *kobj, struct attribute *attr,
const char *page, size_t length)
{
struct blk_mq_ctx_sysfs_entry *entry;
struct blk_mq_ctx *ctx;
struct request_queue *q;
ssize_t res;
entry = container_of(attr, struct blk_mq_ctx_sysfs_entry, attr);
ctx = container_of(kobj, struct blk_mq_ctx, kobj);
q = ctx->queue;
if (!entry->store)
return -EIO;
mutex_lock(&q->sysfs_lock);
res = entry->store(ctx, page, length);
mutex_unlock(&q->sysfs_lock);
return res;
}
static ssize_t blk_mq_hw_sysfs_show(struct kobject *kobj,
struct attribute *attr, char *page)
{
@ -198,23 +150,16 @@ static struct attribute *default_hw_ctx_attrs[] = {
};
ATTRIBUTE_GROUPS(default_hw_ctx);
static const struct sysfs_ops blk_mq_sysfs_ops = {
.show = blk_mq_sysfs_show,
.store = blk_mq_sysfs_store,
};
static const struct sysfs_ops blk_mq_hw_sysfs_ops = {
.show = blk_mq_hw_sysfs_show,
.store = blk_mq_hw_sysfs_store,
};
static struct kobj_type blk_mq_ktype = {
.sysfs_ops = &blk_mq_sysfs_ops,
.release = blk_mq_sysfs_release,
};
static struct kobj_type blk_mq_ctx_ktype = {
.sysfs_ops = &blk_mq_sysfs_ops,
.release = blk_mq_ctx_sysfs_release,
};

8
block/blk-mq.c
View File

@ -525,7 +525,7 @@ void blk_mq_free_request(struct request *rq)
__blk_mq_dec_active_requests(hctx);
if (unlikely(laptop_mode && !blk_rq_is_passthrough(rq)))
laptop_io_completion(q->backing_dev_info);
laptop_io_completion(q->disk->bdi);
rq_qos_done(q, rq);
@ -3115,7 +3115,8 @@ struct request_queue *blk_mq_init_queue(struct blk_mq_tag_set *set)
}
EXPORT_SYMBOL(blk_mq_init_queue);
struct gendisk *__blk_mq_alloc_disk(struct blk_mq_tag_set *set, void *queuedata)
struct gendisk *__blk_mq_alloc_disk(struct blk_mq_tag_set *set, void *queuedata,
struct lock_class_key *lkclass)
{
struct request_queue *q;
struct gendisk *disk;
@ -3124,12 +3125,11 @@ struct gendisk *__blk_mq_alloc_disk(struct blk_mq_tag_set *set, void *queuedata)
if (IS_ERR(q))
return ERR_CAST(q);
disk = __alloc_disk_node(0, set->numa_node);
disk = __alloc_disk_node(q, set->numa_node, lkclass);
if (!disk) {
blk_cleanup_queue(q);
return ERR_PTR(-ENOMEM);
}
disk->queue = q;
return disk;
}
EXPORT_SYMBOL(__blk_mq_alloc_disk);

34
block/blk-settings.c
View File

@ -8,6 +8,7 @@
#include <linux/bio.h>
#include <linux/blkdev.h>
#include <linux/pagemap.h>
#include <linux/backing-dev-defs.h>
#include <linux/gcd.h>
#include <linux/lcm.h>
#include <linux/jiffies.h>
@ -140,7 +141,9 @@ void blk_queue_max_hw_sectors(struct request_queue *q, unsigned int max_hw_secto
limits->logical_block_size >> SECTOR_SHIFT);
limits->max_sectors = max_sectors;
q->backing_dev_info->io_pages = max_sectors >> (PAGE_SHIFT - 9);
if (!q->disk)
return;
q->disk->bdi->io_pages = max_sectors >> (PAGE_SHIFT - 9);
}
EXPORT_SYMBOL(blk_queue_max_hw_sectors);
@ -380,18 +383,19 @@ void blk_queue_alignment_offset(struct request_queue *q, unsigned int offset)
}
EXPORT_SYMBOL(blk_queue_alignment_offset);
void blk_queue_update_readahead(struct request_queue *q)
void disk_update_readahead(struct gendisk *disk)
{
struct request_queue *q = disk->queue;
/*
* For read-ahead of large files to be effective, we need to read ahead
* at least twice the optimal I/O size.
*/
q->backing_dev_info->ra_pages =
disk->bdi->ra_pages =
max(queue_io_opt(q) * 2 / PAGE_SIZE, VM_READAHEAD_PAGES);
q->backing_dev_info->io_pages =
queue_max_sectors(q) >> (PAGE_SHIFT - 9);
disk->bdi->io_pages = queue_max_sectors(q) >> (PAGE_SHIFT - 9);
}
EXPORT_SYMBOL_GPL(blk_queue_update_readahead);
EXPORT_SYMBOL_GPL(disk_update_readahead);
/**
* blk_limits_io_min - set minimum request size for a device
@ -471,7 +475,9 @@ EXPORT_SYMBOL(blk_limits_io_opt);
void blk_queue_io_opt(struct request_queue *q, unsigned int opt)
{
blk_limits_io_opt(&q->limits, opt);
q->backing_dev_info->ra_pages =
if (!q->disk)
return;
q->disk->bdi->ra_pages =
max(queue_io_opt(q) * 2 / PAGE_SIZE, VM_READAHEAD_PAGES);
}
EXPORT_SYMBOL(blk_queue_io_opt);
@ -661,17 +667,11 @@ void disk_stack_limits(struct gendisk *disk, struct block_device *bdev,
struct request_queue *t = disk->queue;
if (blk_stack_limits(&t->limits, &bdev_get_queue(bdev)->limits,
get_start_sect(bdev) + (offset >> 9)) < 0) {
char top[BDEVNAME_SIZE], bottom[BDEVNAME_SIZE];
get_start_sect(bdev) + (offset >> 9)) < 0)
pr_notice("%s: Warning: Device %pg is misaligned\n",
disk->disk_name, bdev);
disk_name(disk, 0, top);
bdevname(bdev, bottom);
printk(KERN_NOTICE "%s: Warning: Device %s is misaligned\n",
top, bottom);
}
blk_queue_update_readahead(disk->queue);
disk_update_readahead(disk);
}
EXPORT_SYMBOL(disk_stack_limits);

35
block/blk-sysfs.c
View File

@ -88,9 +88,11 @@ queue_requests_store(struct request_queue *q, const char *page, size_t count)
static ssize_t queue_ra_show(struct request_queue *q, char *page)
{
unsigned long ra_kb = q->backing_dev_info->ra_pages <<
(PAGE_SHIFT - 10);
unsigned long ra_kb;
if (!q->disk)
return -EINVAL;
ra_kb = q->disk->bdi->ra_pages << (PAGE_SHIFT - 10);
return queue_var_show(ra_kb, page);
}
@ -98,13 +100,14 @@ static ssize_t
queue_ra_store(struct request_queue *q, const char *page, size_t count)
{
unsigned long ra_kb;
ssize_t ret = queue_var_store(&ra_kb, page, count);
ssize_t ret;
if (!q->disk)
return -EINVAL;
ret = queue_var_store(&ra_kb, page, count);
if (ret < 0)
return ret;
q->backing_dev_info->ra_pages = ra_kb >> (PAGE_SHIFT - 10);
q->disk->bdi->ra_pages = ra_kb >> (PAGE_SHIFT - 10);
return ret;
}
@ -251,7 +254,8 @@ queue_max_sectors_store(struct request_queue *q, const char *page, size_t count)
spin_lock_irq(&q->queue_lock);
q->limits.max_sectors = max_sectors_kb << 1;
q->backing_dev_info->io_pages = max_sectors_kb >> (PAGE_SHIFT - 10);
if (q->disk)
q->disk->bdi->io_pages = max_sectors_kb >> (PAGE_SHIFT - 10);
spin_unlock_irq(&q->queue_lock);
return ret;
@ -766,13 +770,6 @@ static void blk_exit_queue(struct request_queue *q)
* e.g. blkcg_print_blkgs() to crash.
*/
blkcg_exit_queue(q);
/*
* Since the cgroup code may dereference the @q->backing_dev_info
* pointer, only decrease its reference count after having removed the
* association with the block cgroup controller.
*/
bdi_put(q->backing_dev_info);
}
/**
@ -859,15 +856,6 @@ int blk_register_queue(struct gendisk *disk)
struct device *dev = disk_to_dev(disk);
struct request_queue *q = disk->queue;
if (WARN_ON(!q))
return -ENXIO;
WARN_ONCE(blk_queue_registered(q),
"%s is registering an already registered queue\n",
kobject_name(&dev->kobj));
blk_queue_update_readahead(q);
ret = blk_trace_init_sysfs(dev);
if (ret)
return ret;
@ -941,7 +929,6 @@ int blk_register_queue(struct gendisk *disk)
return ret;
}
EXPORT_SYMBOL_GPL(blk_register_queue);
/**
* blk_unregister_queue - counterpart of blk_register_queue()

32
block/blk-throttle.c
View File

@ -178,6 +178,9 @@ struct throtl_grp {
unsigned int bad_bio_cnt; /* bios exceeding latency threshold */
unsigned long bio_cnt_reset_time;
atomic_t io_split_cnt[2];
atomic_t last_io_split_cnt[2];
struct blkg_rwstat stat_bytes;
struct blkg_rwstat stat_ios;
};
@ -777,6 +780,8 @@ static inline void throtl_start_new_slice_with_credit(struct throtl_grp *tg,
tg->bytes_disp[rw] = 0;
tg->io_disp[rw] = 0;
atomic_set(&tg->io_split_cnt[rw], 0);
/*
* Previous slice has expired. We must have trimmed it after last
* bio dispatch. That means since start of last slice, we never used
@ -799,6 +804,9 @@ static inline void throtl_start_new_slice(struct throtl_grp *tg, bool rw)
tg->io_disp[rw] = 0;
tg->slice_start[rw] = jiffies;
tg->slice_end[rw] = jiffies + tg->td->throtl_slice;
atomic_set(&tg->io_split_cnt[rw], 0);
throtl_log(&tg->service_queue,
"[%c] new slice start=%lu end=%lu jiffies=%lu",
rw == READ ? 'R' : 'W', tg->slice_start[rw],
@ -1031,6 +1039,9 @@ static bool tg_may_dispatch(struct throtl_grp *tg, struct bio *bio,
jiffies + tg->td->throtl_slice);
}
if (iops_limit != UINT_MAX)
tg->io_disp[rw] += atomic_xchg(&tg->io_split_cnt[rw], 0);
if (tg_with_in_bps_limit(tg, bio, bps_limit, &bps_wait) &&
tg_with_in_iops_limit(tg, bio, iops_limit, &iops_wait)) {
if (wait)
@ -2052,12 +2063,14 @@ static void throtl_downgrade_check(struct throtl_grp *tg)
}
if (tg->iops[READ][LIMIT_LOW]) {
tg->last_io_disp[READ] += atomic_xchg(&tg->last_io_split_cnt[READ], 0);
iops = tg->last_io_disp[READ] * HZ / elapsed_time;
if (iops >= tg->iops[READ][LIMIT_LOW])
tg->last_low_overflow_time[READ] = now;
}
if (tg->iops[WRITE][LIMIT_LOW]) {
tg->last_io_disp[WRITE] += atomic_xchg(&tg->last_io_split_cnt[WRITE], 0);
iops = tg->last_io_disp[WRITE] * HZ / elapsed_time;
if (iops >= tg->iops[WRITE][LIMIT_LOW])
tg->last_low_overflow_time[WRITE] = now;
@ -2176,6 +2189,25 @@ static inline void throtl_update_latency_buckets(struct throtl_data *td)
}
#endif
void blk_throtl_charge_bio_split(struct bio *bio)
{
struct blkcg_gq *blkg = bio->bi_blkg;
struct throtl_grp *parent = blkg_to_tg(blkg);
struct throtl_service_queue *parent_sq;
bool rw = bio_data_dir(bio);
do {
if (!parent->has_rules[rw])
break;
atomic_inc(&parent->io_split_cnt[rw]);
atomic_inc(&parent->last_io_split_cnt[rw]);
parent_sq = parent->service_queue.parent_sq;
parent = sq_to_tg(parent_sq);
} while (parent);
}
bool blk_throtl_bio(struct bio *bio)
{
struct request_queue *q = bio->bi_bdev->bd_disk->queue;

8
block/blk-wbt.c
View File

@ -97,7 +97,7 @@ static void wb_timestamp(struct rq_wb *rwb, unsigned long *var)
*/
static bool wb_recent_wait(struct rq_wb *rwb)
{
struct bdi_writeback *wb = &rwb->rqos.q->backing_dev_info->wb;
struct bdi_writeback *wb = &rwb->rqos.q->disk->bdi->wb;
return time_before(jiffies, wb->dirty_sleep + HZ);
}
@ -234,7 +234,7 @@ enum {
static int latency_exceeded(struct rq_wb *rwb, struct blk_rq_stat *stat)
{
struct backing_dev_info *bdi = rwb->rqos.q->backing_dev_info;
struct backing_dev_info *bdi = rwb->rqos.q->disk->bdi;
struct rq_depth *rqd = &rwb->rq_depth;
u64 thislat;
@ -287,7 +287,7 @@ static int latency_exceeded(struct rq_wb *rwb, struct blk_rq_stat *stat)
static void rwb_trace_step(struct rq_wb *rwb, const char *msg)
{
struct backing_dev_info *bdi = rwb->rqos.q->backing_dev_info;
struct backing_dev_info *bdi = rwb->rqos.q->disk->bdi;
struct rq_depth *rqd = &rwb->rq_depth;
trace_wbt_step(bdi, msg, rqd->scale_step, rwb->cur_win_nsec,
@ -359,7 +359,7 @@ static void wb_timer_fn(struct blk_stat_callback *cb)
status = latency_exceeded(rwb, cb->stat);
trace_wbt_timer(rwb->rqos.q->backing_dev_info, status, rqd->scale_step,
trace_wbt_timer(rwb->rqos.q->disk->bdi, status, rqd->scale_step,
inflight);
/*

6
block/blk-zoned.c
View File

@ -360,9 +360,6 @@ int blkdev_report_zones_ioctl(struct block_device *bdev, fmode_t mode,
if (!blk_queue_is_zoned(q))
return -ENOTTY;
if (!capable(CAP_SYS_ADMIN))
return -EACCES;
if (copy_from_user(&rep, argp, sizeof(struct blk_zone_report)))
return -EFAULT;
@ -421,9 +418,6 @@ int blkdev_zone_mgmt_ioctl(struct block_device *bdev, fmode_t mode,
if (!blk_queue_is_zoned(q))
return -ENOTTY;
if (!capable(CAP_SYS_ADMIN))
return -EACCES;
if (!(mode & FMODE_WRITE))
return -EBADF;

20
block/blk.h
View File

@ -128,7 +128,7 @@ static inline bool integrity_req_gap_front_merge(struct request *req,
bip_next->bip_vec[0].bv_offset);
}
void blk_integrity_add(struct gendisk *);
int blk_integrity_add(struct gendisk *disk);
void blk_integrity_del(struct gendisk *);
#else /* CONFIG_BLK_DEV_INTEGRITY */
static inline bool blk_integrity_merge_rq(struct request_queue *rq,
@ -162,8 +162,9 @@ static inline bool bio_integrity_endio(struct bio *bio)
static inline void bio_integrity_free(struct bio *bio)
{
}
static inline void blk_integrity_add(struct gendisk *disk)
static inline int blk_integrity_add(struct gendisk *disk)
{
return 0;
}
static inline void blk_integrity_del(struct gendisk *disk)
{
@ -289,11 +290,13 @@ int create_task_io_context(struct task_struct *task, gfp_t gfp_mask, int node);
extern int blk_throtl_init(struct request_queue *q);
extern void blk_throtl_exit(struct request_queue *q);
extern void blk_throtl_register_queue(struct request_queue *q);
extern void blk_throtl_charge_bio_split(struct bio *bio);
bool blk_throtl_bio(struct bio *bio);
#else /* CONFIG_BLK_DEV_THROTTLING */
static inline int blk_throtl_init(struct request_queue *q) { return 0; }
static inline void blk_throtl_exit(struct request_queue *q) { }
static inline void blk_throtl_register_queue(struct request_queue *q) { }
static inline void blk_throtl_charge_bio_split(struct bio *bio) { }
static inline bool blk_throtl_bio(struct bio *bio) { return false; }
#endif /* CONFIG_BLK_DEV_THROTTLING */
#ifdef CONFIG_BLK_DEV_THROTTLING_LOW
@ -340,15 +343,14 @@ static inline void blk_queue_clear_zone_settings(struct request_queue *q) {}
int blk_alloc_ext_minor(void);
void blk_free_ext_minor(unsigned int minor);
char *disk_name(struct gendisk *hd, int partno, char *buf);
#define ADDPART_FLAG_NONE 0
#define ADDPART_FLAG_RAID 1
#define ADDPART_FLAG_WHOLEDISK 2
int bdev_add_partition(struct block_device *bdev, int partno,
sector_t start, sector_t length);
int bdev_del_partition(struct block_device *bdev, int partno);
int bdev_resize_partition(struct block_device *bdev, int partno,
sector_t start, sector_t length);
int bdev_add_partition(struct gendisk *disk, int partno, sector_t start,
sector_t length);
int bdev_del_partition(struct gendisk *disk, int partno);
int bdev_resize_partition(struct gendisk *disk, int partno, sector_t start,
sector_t length);
int bio_add_hw_page(struct request_queue *q, struct bio *bio,
struct page *page, unsigned int len, unsigned int offset,
@ -356,7 +358,7 @@ int bio_add_hw_page(struct request_queue *q, struct bio *bio,
struct request_queue *blk_alloc_queue(int node_id);
void disk_alloc_events(struct gendisk *disk);
int disk_alloc_events(struct gendisk *disk);
void disk_add_events(struct gendisk *disk);
void disk_del_events(struct gendisk *disk);
void disk_release_events(struct gendisk *disk);

39
block/bounce.c
View File

@ -67,18 +67,6 @@ static __init int init_emergency_pool(void)
__initcall(init_emergency_pool);
/*
* highmem version, map in to vec
*/
static void bounce_copy_vec(struct bio_vec *to, unsigned char *vfrom)
{
unsigned char *vto;
vto = kmap_atomic(to->bv_page);
memcpy(vto + to->bv_offset, vfrom, to->bv_len);
kunmap_atomic(vto);
}
/*
* Simple bounce buffer support for highmem pages. Depending on the
* queue gfp mask set, *to may or may not be a highmem page. kmap it
@ -86,7 +74,6 @@ static void bounce_copy_vec(struct bio_vec *to, unsigned char *vfrom)
*/
static void copy_to_high_bio_irq(struct bio *to, struct bio *from)
{
unsigned char *vfrom;
struct bio_vec tovec, fromvec;
struct bvec_iter iter;
/*
@ -104,11 +91,8 @@ static void copy_to_high_bio_irq(struct bio *to, struct bio *from)
* been modified by the block layer, so use the original
* copy, bounce_copy_vec already uses tovec->bv_len
*/
vfrom = page_address(fromvec.bv_page) +
tovec.bv_offset;
bounce_copy_vec(&tovec, vfrom);
flush_dcache_page(tovec.bv_page);
memcpy_to_bvec(&tovec, page_address(fromvec.bv_page) +
tovec.bv_offset);
}
bio_advance_iter(from, &from_iter, tovec.bv_len);
}
@ -255,24 +239,19 @@ void __blk_queue_bounce(struct request_queue *q, struct bio **bio_orig)
* because the 'bio' is single-page bvec.
*/
for (i = 0, to = bio->bi_io_vec; i < bio->bi_vcnt; to++, i++) {
struct page *page = to->bv_page;
struct page *bounce_page;
if (!PageHighMem(page))
if (!PageHighMem(to->bv_page))
continue;
to->bv_page = mempool_alloc(&page_pool, GFP_NOIO);
inc_zone_page_state(to->bv_page, NR_BOUNCE);
bounce_page = mempool_alloc(&page_pool, GFP_NOIO);
inc_zone_page_state(bounce_page, NR_BOUNCE);
if (rw == WRITE) {
char *vto, *vfrom;
flush_dcache_page(page);
vto = page_address(to->bv_page) + to->bv_offset;
vfrom = kmap_atomic(page) + to->bv_offset;
memcpy(vto, vfrom, to->bv_len);
kunmap_atomic(vfrom);
flush_dcache_page(to->bv_page);
memcpy_from_bvec(page_address(bounce_page), to);
}
to->bv_page = bounce_page;
}
trace_block_bio_bounce(*bio_orig);

255
block/cmdline-parser.c
View File

@ -1,255 +0,0 @@
// SPDX-License-Identifier: GPL-2.0
/*
* Parse command line, get partition information
*
* Written by Cai Zhiyong <caizhiyong@huawei.com>
*
*/
#include <linux/export.h>
#include <linux/cmdline-parser.h>
static int parse_subpart(struct cmdline_subpart **subpart, char *partdef)
{
int ret = 0;
struct cmdline_subpart *new_subpart;
*subpart = NULL;
new_subpart = kzalloc(sizeof(struct cmdline_subpart), GFP_KERNEL);
if (!new_subpart)
return -ENOMEM;
if (*partdef == '-') {
new_subpart->size = (sector_t)(~0ULL);
partdef++;
} else {
new_subpart->size = (sector_t)memparse(partdef, &partdef);
if (new_subpart->size < (sector_t)PAGE_SIZE) {
pr_warn("cmdline partition size is invalid.");
ret = -EINVAL;
goto fail;
}
}
if (*partdef == '@') {
partdef++;
new_subpart->from = (sector_t)memparse(partdef, &partdef);
} else {
new_subpart->from = (sector_t)(~0ULL);
}
if (*partdef == '(') {
int length;
char *next = strchr(++partdef, ')');
if (!next) {
pr_warn("cmdline partition format is invalid.");
ret = -EINVAL;
goto fail;
}
length = min_t(int, next - partdef,
sizeof(new_subpart->name) - 1);
strncpy(new_subpart->name, partdef, length);
new_subpart->name[length] = '\0';
partdef = ++next;
} else
new_subpart->name[0] = '\0';
new_subpart->flags = 0;
if (!strncmp(partdef, "ro", 2)) {
new_subpart->flags |= PF_RDONLY;
partdef += 2;
}
if (!strncmp(partdef, "lk", 2)) {
new_subpart->flags |= PF_POWERUP_LOCK;
partdef += 2;
}
*subpart = new_subpart;
return 0;
fail:
kfree(new_subpart);
return ret;
}
static void free_subpart(struct cmdline_parts *parts)
{
struct cmdline_subpart *subpart;
while (parts->subpart) {
subpart = parts->subpart;
parts->subpart = subpart->next_subpart;
kfree(subpart);
}
}
static int parse_parts(struct cmdline_parts **parts, const char *bdevdef)
{
int ret = -EINVAL;
char *next;
int length;
struct cmdline_subpart **next_subpart;
struct cmdline_parts *newparts;
char buf[BDEVNAME_SIZE + 32 + 4];
*parts = NULL;
newparts = kzalloc(sizeof(struct cmdline_parts), GFP_KERNEL);
if (!newparts)
return -ENOMEM;
next = strchr(bdevdef, ':');
if (!next) {
pr_warn("cmdline partition has no block device.");
goto fail;
}
length = min_t(int, next - bdevdef, sizeof(newparts->name) - 1);
strncpy(newparts->name, bdevdef, length);
newparts->name[length] = '\0';
newparts->nr_subparts = 0;
next_subpart = &newparts->subpart;
while (next && *(++next)) {
bdevdef = next;
next = strchr(bdevdef, ',');
length = (!next) ? (sizeof(buf) - 1) :
min_t(int, next - bdevdef, sizeof(buf) - 1);
strncpy(buf, bdevdef, length);
buf[length] = '\0';
ret = parse_subpart(next_subpart, buf);
if (ret)
goto fail;
newparts->nr_subparts++;
next_subpart = &(*next_subpart)->next_subpart;
}
if (!newparts->subpart) {
pr_warn("cmdline partition has no valid partition.");
ret = -EINVAL;
goto fail;
}
*parts = newparts;
return 0;
fail:
free_subpart(newparts);
kfree(newparts);
return ret;
}
void cmdline_parts_free(struct cmdline_parts **parts)
{
struct cmdline_parts *next_parts;
while (*parts) {
next_parts = (*parts)->next_parts;
free_subpart(*parts);
kfree(*parts);
*parts = next_parts;
}
}
EXPORT_SYMBOL(cmdline_parts_free);
int cmdline_parts_parse(struct cmdline_parts **parts, const char *cmdline)
{
int ret;
char *buf;
char *pbuf;
char *next;
struct cmdline_parts **next_parts;
*parts = NULL;
next = pbuf = buf = kstrdup(cmdline, GFP_KERNEL);
if (!buf)
return -ENOMEM;
next_parts = parts;
while (next && *pbuf) {
next = strchr(pbuf, ';');
if (next)
*next = '\0';
ret = parse_parts(next_parts, pbuf);
if (ret)
goto fail;
if (next)
pbuf = ++next;
next_parts = &(*next_parts)->next_parts;
}
if (!*parts) {
pr_warn("cmdline partition has no valid partition.");
ret = -EINVAL;
goto fail;
}
ret = 0;
done:
kfree(buf);
return ret;
fail:
cmdline_parts_free(parts);
goto done;
}
EXPORT_SYMBOL(cmdline_parts_parse);
struct cmdline_parts *cmdline_parts_find(struct cmdline_parts *parts,
const char *bdev)
{
while (parts && strncmp(bdev, parts->name, sizeof(parts->name)))
parts = parts->next_parts;
return parts;
}
EXPORT_SYMBOL(cmdline_parts_find);
/*
* add_part()
* 0 success.
* 1 can not add so many partitions.
*/
int cmdline_parts_set(struct cmdline_parts *parts, sector_t disk_size,
int slot,
int (*add_part)(int, struct cmdline_subpart *, void *),
void *param)
{
sector_t from = 0;
struct cmdline_subpart *subpart;
for (subpart = parts->subpart; subpart;
subpart = subpart->next_subpart, slot++) {
if (subpart->from == (sector_t)(~0ULL))
subpart->from = from;
else
from = subpart->from;
if (from >= disk_size)
break;
if (subpart->size > (disk_size - from))
subpart->size = disk_size - from;
from += subpart->size;
if (add_part(slot, subpart, param))
break;
}
return slot;
}
EXPORT_SYMBOL(cmdline_parts_set);

69
block/disk-events.c
View File

@ -163,15 +163,31 @@ void disk_flush_events(struct gendisk *disk, unsigned int mask)
spin_unlock_irq(&ev->lock);
}
/*
* Tell userland about new events. Only the events listed in @disk->events are
* reported, and only if DISK_EVENT_FLAG_UEVENT is set. Otherwise, events are
* processed internally but never get reported to userland.
*/
static void disk_event_uevent(struct gendisk *disk, unsigned int events)
{
char *envp[ARRAY_SIZE(disk_uevents) + 1] = { };
int nr_events = 0, i;
for (i = 0; i < ARRAY_SIZE(disk_uevents); i++)
if (events & disk->events & (1 << i))
envp[nr_events++] = disk_uevents[i];
if (nr_events)
kobject_uevent_env(&disk_to_dev(disk)->kobj, KOBJ_CHANGE, envp);
}
static void disk_check_events(struct disk_events *ev,
unsigned int *clearing_ptr)
{
struct gendisk *disk = ev->disk;
char *envp[ARRAY_SIZE(disk_uevents) + 1] = { };
unsigned int clearing = *clearing_ptr;
unsigned int events;
unsigned long intv;
int nr_events = 0, i;
/* check events */
events = disk->fops->check_events(disk, clearing);
@ -190,19 +206,11 @@ static void disk_check_events(struct disk_events *ev,
spin_unlock_irq(&ev->lock);
/*
* Tell userland about new events. Only the events listed in
* @disk->events are reported, and only if DISK_EVENT_FLAG_UEVENT
* is set. Otherwise, events are processed internally but never
* get reported to userland.
*/
for (i = 0; i < ARRAY_SIZE(disk_uevents); i++)
if ((events & disk->events & (1 << i)) &&
(disk->event_flags & DISK_EVENT_FLAG_UEVENT))
envp[nr_events++] = disk_uevents[i];
if (events & DISK_EVENT_MEDIA_CHANGE)
inc_diskseq(disk);
if (nr_events)
kobject_uevent_env(&disk_to_dev(disk)->kobj, KOBJ_CHANGE, envp);
if (disk->event_flags & DISK_EVENT_FLAG_UEVENT)
disk_event_uevent(disk, events);
}
/**
@ -281,6 +289,32 @@ bool bdev_check_media_change(struct block_device *bdev)
}
EXPORT_SYMBOL(bdev_check_media_change);
/**
* disk_force_media_change - force a media change event
* @disk: the disk which will raise the event
* @events: the events to raise
*
* Generate uevents for the disk. If DISK_EVENT_MEDIA_CHANGE is present,
* attempt to free all dentries and inodes and invalidates all block
* device page cache entries in that case.
*
* Returns %true if DISK_EVENT_MEDIA_CHANGE was raised, or %false if not.
*/
bool disk_force_media_change(struct gendisk *disk, unsigned int events)
{
disk_event_uevent(disk, events);
if (!(events & DISK_EVENT_MEDIA_CHANGE))
return false;
if (__invalidate_device(disk->part0, true))
pr_warn("VFS: busy inodes on changed media %s\n",
disk->disk_name);
set_bit(GD_NEED_PART_SCAN, &disk->state);
return true;
}
EXPORT_SYMBOL_GPL(disk_force_media_change);
/*
* Separate this part out so that a different pointer for clearing_ptr can be
* passed in for disk_clear_events.
@ -410,17 +444,17 @@ module_param_cb(events_dfl_poll_msecs, &disk_events_dfl_poll_msecs_param_ops,
/*
* disk_{alloc|add|del|release}_events - initialize and destroy disk_events.
*/
void disk_alloc_events(struct gendisk *disk)
int disk_alloc_events(struct gendisk *disk)
{
struct disk_events *ev;
if (!disk->fops->check_events || !disk->events)
return;
return 0;
ev = kzalloc(sizeof(*ev), GFP_KERNEL);
if (!ev) {
pr_warn("%s: failed to initialize events\n", disk->disk_name);
return;
return -ENOMEM;
}
INIT_LIST_HEAD(&ev->node);
@ -432,6 +466,7 @@ void disk_alloc_events(struct gendisk *disk)
INIT_DELAYED_WORK(&ev->dwork, disk_events_workfn);
disk->ev = ev;
return 0;
}
void disk_add_events(struct gendisk *disk)

7
block/elevator.c
View File

@ -336,6 +336,9 @@ enum elv_merge elv_merge(struct request_queue *q, struct request **req,
__rq = elv_rqhash_find(q, bio->bi_iter.bi_sector);
if (__rq && elv_bio_merge_ok(__rq, bio)) {
*req = __rq;
if (blk_discard_mergable(__rq))
return ELEVATOR_DISCARD_MERGE;
return ELEVATOR_BACK_MERGE;
}
@ -630,6 +633,9 @@ static inline bool elv_support_iosched(struct request_queue *q)
*/
static struct elevator_type *elevator_get_default(struct request_queue *q)
{
if (q->tag_set && q->tag_set->flags & BLK_MQ_F_NO_SCHED_BY_DEFAULT)
return NULL;
if (q->nr_hw_queues != 1 &&
!blk_mq_is_sbitmap_shared(q->tag_set->flags))
return NULL;
@ -702,7 +708,6 @@ void elevator_init_mq(struct request_queue *q)
elevator_put(e);
}
}
EXPORT_SYMBOL_GPL(elevator_init_mq); /* only for dm-rq */
/*
* switch to new_e io scheduler. be careful not to introduce deadlocks -

385
block/genhd.c
View File

@ -29,6 +29,23 @@
static struct kobject *block_depr;
/*
* Unique, monotonically increasing sequential number associated with block
* devices instances (i.e. incremented each time a device is attached).
* Associating uevents with block devices in userspace is difficult and racy:
* the uevent netlink socket is lossy, and on slow and overloaded systems has
* a very high latency.
* Block devices do not have exclusive owners in userspace, any process can set
* one up (e.g. loop devices). Moreover, device names can be reused (e.g. loop0
* can be reused again and again).
* A userspace process setting up a block device and watching for its events
* cannot thus reliably tell whether an event relates to the device it just set
* up or another earlier instance with the same name.
* This sequential number allows userspace processes to solve this problem, and
* uniquely associate an uevent to the lifetime to a device.
*/
static atomic64_t diskseq;
/* for extended dynamic devt allocation, currently only one major is used */
#define NR_EXT_DEVT (1 << MINORBITS)
static DEFINE_IDA(ext_devt_ida);
@ -60,7 +77,8 @@ bool set_capacity_and_notify(struct gendisk *disk, sector_t size)
* initial capacity during probing.
*/
if (size == capacity ||
(disk->flags & (GENHD_FL_UP | GENHD_FL_HIDDEN)) != GENHD_FL_UP)
!disk_live(disk) ||
(disk->flags & GENHD_FL_HIDDEN))
return false;
pr_info("%s: detected capacity change from %lld to %lld\n",
@ -78,11 +96,17 @@ bool set_capacity_and_notify(struct gendisk *disk, sector_t size)
EXPORT_SYMBOL_GPL(set_capacity_and_notify);
/*
* Format the device name of the indicated disk into the supplied buffer and
* return a pointer to that same buffer for convenience.
* Format the device name of the indicated block device into the supplied buffer
* and return a pointer to that same buffer for convenience.
*
* Note: do not use this in new code, use the %pg specifier to sprintf and
* printk insted.
*/
char *disk_name(struct gendisk *hd, int partno, char *buf)
const char *bdevname(struct block_device *bdev, char *buf)
{
struct gendisk *hd = bdev->bd_disk;
int partno = bdev->bd_partno;
if (!partno)
snprintf(buf, BDEVNAME_SIZE, "%s", hd->disk_name);
else if (isdigit(hd->disk_name[strlen(hd->disk_name)-1]))
@ -92,11 +116,6 @@ char *disk_name(struct gendisk *hd, int partno, char *buf)
return buf;
}
const char *bdevname(struct block_device *bdev, char *buf)
{
return disk_name(bdev->bd_disk, bdev->bd_partno, buf);
}
EXPORT_SYMBOL(bdevname);
static void part_stat_read_all(struct block_device *part,
@ -294,54 +313,19 @@ void unregister_blkdev(unsigned int major, const char *name)
EXPORT_SYMBOL(unregister_blkdev);
/**
* blk_mangle_minor - scatter minor numbers apart
* @minor: minor number to mangle
*
* Scatter consecutively allocated @minor number apart if MANGLE_DEVT
* is enabled. Mangling twice gives the original value.
*
* RETURNS:
* Mangled value.
*
* CONTEXT:
* Don't care.
*/
static int blk_mangle_minor(int minor)
{
#ifdef CONFIG_DEBUG_BLOCK_EXT_DEVT
int i;
for (i = 0; i < MINORBITS / 2; i++) {
int low = minor & (1 << i);
int high = minor & (1 << (MINORBITS - 1 - i));
int distance = MINORBITS - 1 - 2 * i;
minor ^= low | high; /* clear both bits */
low <<= distance; /* swap the positions */
high >>= distance;
minor |= low | high; /* and set */
}
#endif
return minor;
}
int blk_alloc_ext_minor(void)
{
int idx;
idx = ida_alloc_range(&ext_devt_ida, 0, NR_EXT_DEVT, GFP_KERNEL);
if (idx < 0) {
if (idx == -ENOSPC)
return -EBUSY;
return idx;
}
return blk_mangle_minor(idx);
if (idx == -ENOSPC)
return -EBUSY;
return idx;
}
void blk_free_ext_minor(unsigned int minor)
{
ida_free(&ext_devt_ida, blk_mangle_minor(minor));
ida_free(&ext_devt_ida, minor);
}
static char *bdevt_str(dev_t devt, char *buf)
@ -390,78 +374,20 @@ static void disk_scan_partitions(struct gendisk *disk)
blkdev_put(bdev, FMODE_READ);
}
static void register_disk(struct device *parent, struct gendisk *disk,
const struct attribute_group **groups)
{
struct device *ddev = disk_to_dev(disk);
int err;
ddev->parent = parent;
dev_set_name(ddev, "%s", disk->disk_name);
/* delay uevents, until we scanned partition table */
dev_set_uevent_suppress(ddev, 1);
if (groups) {
WARN_ON(ddev->groups);
ddev->groups = groups;
}
if (device_add(ddev))
return;
if (!sysfs_deprecated) {
err = sysfs_create_link(block_depr, &ddev->kobj,
kobject_name(&ddev->kobj));
if (err) {
device_del(ddev);
return;
}
}
/*
* avoid probable deadlock caused by allocating memory with
* GFP_KERNEL in runtime_resume callback of its all ancestor
* devices
*/
pm_runtime_set_memalloc_noio(ddev, true);
disk->part0->bd_holder_dir =
kobject_create_and_add("holders", &ddev->kobj);
disk->slave_dir = kobject_create_and_add("slaves", &ddev->kobj);
if (disk->flags & GENHD_FL_HIDDEN)
return;
disk_scan_partitions(disk);
/* announce the disk and partitions after all partitions are created */
dev_set_uevent_suppress(ddev, 0);
disk_uevent(disk, KOBJ_ADD);
if (disk->queue->backing_dev_info->dev) {
err = sysfs_create_link(&ddev->kobj,
&disk->queue->backing_dev_info->dev->kobj,
"bdi");
WARN_ON(err);
}
}
/**
* __device_add_disk - add disk information to kernel list
* device_add_disk - add disk information to kernel list
* @parent: parent device for the disk
* @disk: per-device partitioning information
* @groups: Additional per-device sysfs groups
* @register_queue: register the queue if set to true
*
* This function registers the partitioning information in @disk
* with the kernel.
*
* FIXME: error handling
*/
static void __device_add_disk(struct device *parent, struct gendisk *disk,
const struct attribute_group **groups,
bool register_queue)
int device_add_disk(struct device *parent, struct gendisk *disk,
const struct attribute_group **groups)
{
struct device *ddev = disk_to_dev(disk);
int ret;
/*
@ -470,8 +396,7 @@ static void __device_add_disk(struct device *parent, struct gendisk *disk,
* elevator if one is needed, that is, for devices requesting queue
* registration.
*/
if (register_queue)
elevator_init_mq(disk->queue);
elevator_init_mq(disk->queue);
/*
* If the driver provides an explicit major number it also must provide
@ -481,7 +406,8 @@ static void __device_add_disk(struct device *parent, struct gendisk *disk,
* and all partitions from the extended dev_t space.
*/
if (disk->major) {
WARN_ON(!disk->minors);
if (WARN_ON(!disk->minors))
return -EINVAL;
if (disk->minors > DISK_MAX_PARTS) {
pr_err("block: can't allocate more than %d partitions\n",
@ -489,21 +415,65 @@ static void __device_add_disk(struct device *parent, struct gendisk *disk,
disk->minors = DISK_MAX_PARTS;
}
} else {
WARN_ON(disk->minors);
if (WARN_ON(disk->minors))
return -EINVAL;
ret = blk_alloc_ext_minor();
if (ret < 0) {
WARN_ON(1);
return;
}
if (ret < 0)
return ret;
disk->major = BLOCK_EXT_MAJOR;
disk->first_minor = MINOR(ret);
disk->first_minor = ret;
disk->flags |= GENHD_FL_EXT_DEVT;
}
disk->flags |= GENHD_FL_UP;
ret = disk_alloc_events(disk);
if (ret)
goto out_free_ext_minor;
disk_alloc_events(disk);
/* delay uevents, until we scanned partition table */
dev_set_uevent_suppress(ddev, 1);
ddev->parent = parent;
ddev->groups = groups;
dev_set_name(ddev, "%s", disk->disk_name);
if (!(disk->flags & GENHD_FL_HIDDEN))
ddev->devt = MKDEV(disk->major, disk->first_minor);
ret = device_add(ddev);
if (ret)
goto out_disk_release_events;
if (!sysfs_deprecated) {
ret = sysfs_create_link(block_depr, &ddev->kobj,
kobject_name(&ddev->kobj));
if (ret)
goto out_device_del;
}
/*
* avoid probable deadlock caused by allocating memory with
* GFP_KERNEL in runtime_resume callback of its all ancestor
* devices
*/
pm_runtime_set_memalloc_noio(ddev, true);
ret = blk_integrity_add(disk);
if (ret)
goto out_del_block_link;
disk->part0->bd_holder_dir =
kobject_create_and_add("holders", &ddev->kobj);
if (!disk->part0->bd_holder_dir)
goto out_del_integrity;
disk->slave_dir = kobject_create_and_add("slaves", &ddev->kobj);
if (!disk->slave_dir)
goto out_put_holder_dir;
ret = bd_register_pending_holders(disk);
if (ret < 0)
goto out_put_slave_dir;
ret = blk_register_queue(disk);
if (ret)
goto out_put_slave_dir;
if (disk->flags & GENHD_FL_HIDDEN) {
/*
@ -513,48 +483,56 @@ static void __device_add_disk(struct device *parent, struct gendisk *disk,
disk->flags |= GENHD_FL_SUPPRESS_PARTITION_INFO;
disk->flags |= GENHD_FL_NO_PART_SCAN;
} else {
struct backing_dev_info *bdi = disk->queue->backing_dev_info;
struct device *dev = disk_to_dev(disk);
/* Register BDI before referencing it from bdev */
dev->devt = MKDEV(disk->major, disk->first_minor);
ret = bdi_register(bdi, "%u:%u",
ret = bdi_register(disk->bdi, "%u:%u",
disk->major, disk->first_minor);
WARN_ON(ret);
bdi_set_owner(bdi, dev);
bdev_add(disk->part0, dev->devt);
if (ret)
goto out_unregister_queue;
bdi_set_owner(disk->bdi, ddev);
ret = sysfs_create_link(&ddev->kobj,
&disk->bdi->dev->kobj, "bdi");
if (ret)
goto out_unregister_bdi;
bdev_add(disk->part0, ddev->devt);
disk_scan_partitions(disk);
/*
* Announce the disk and partitions after all partitions are
* created. (for hidden disks uevents remain suppressed forever)
*/
dev_set_uevent_suppress(ddev, 0);
disk_uevent(disk, KOBJ_ADD);
}
register_disk(parent, disk, groups);
if (register_queue)
blk_register_queue(disk);
/*
* Take an extra ref on queue which will be put on disk_release()
* so that it sticks around as long as @disk is there.
*/
if (blk_get_queue(disk->queue))
set_bit(GD_QUEUE_REF, &disk->state);
else
WARN_ON_ONCE(1);
disk_update_readahead(disk);
disk_add_events(disk);
blk_integrity_add(disk);
}
return 0;
void device_add_disk(struct device *parent, struct gendisk *disk,
const struct attribute_group **groups)
{
__device_add_disk(parent, disk, groups, true);
out_unregister_bdi:
if (!(disk->flags & GENHD_FL_HIDDEN))
bdi_unregister(disk->bdi);
out_unregister_queue:
blk_unregister_queue(disk);
out_put_slave_dir:
kobject_put(disk->slave_dir);
out_put_holder_dir:
kobject_put(disk->part0->bd_holder_dir);
out_del_integrity:
blk_integrity_del(disk);
out_del_block_link:
if (!sysfs_deprecated)
sysfs_remove_link(block_depr, dev_name(ddev));
out_device_del:
device_del(ddev);
out_disk_release_events:
disk_release_events(disk);
out_free_ext_minor:
if (disk->major == BLOCK_EXT_MAJOR)
blk_free_ext_minor(disk->first_minor);
return WARN_ON_ONCE(ret); /* keep until all callers handle errors */
}
EXPORT_SYMBOL(device_add_disk);
void device_add_disk_no_queue_reg(struct device *parent, struct gendisk *disk)
{
__device_add_disk(parent, disk, NULL, false);
}
EXPORT_SYMBOL(device_add_disk_no_queue_reg);
/**
* del_gendisk - remove the gendisk
* @disk: the struct gendisk to remove
@ -578,26 +556,20 @@ void del_gendisk(struct gendisk *disk)
{
might_sleep();
if (WARN_ON_ONCE(!disk->queue))
if (WARN_ON_ONCE(!disk_live(disk) && !(disk->flags & GENHD_FL_HIDDEN)))
return;
blk_integrity_del(disk);
disk_del_events(disk);
mutex_lock(&disk->open_mutex);
disk->flags &= ~GENHD_FL_UP;
remove_inode_hash(disk->part0->bd_inode);
blk_drop_partitions(disk);
mutex_unlock(&disk->open_mutex);
fsync_bdev(disk->part0);
__invalidate_device(disk->part0, true);
/*
* Unhash the bdev inode for this device so that it can't be looked
* up any more even if openers still hold references to it.
*/
remove_inode_hash(disk->part0->bd_inode);
set_capacity(disk, 0);
if (!(disk->flags & GENHD_FL_HIDDEN)) {
@ -607,7 +579,7 @@ void del_gendisk(struct gendisk *disk)
* Unregister bdi before releasing device numbers (as they can
* get reused and we'd get clashes in sysfs).
*/
bdi_unregister(disk->queue->backing_dev_info);
bdi_unregister(disk->bdi);
}
blk_unregister_queue(disk);
@ -683,7 +655,6 @@ void __init printk_all_partitions(void)
while ((dev = class_dev_iter_next(&iter))) {
struct gendisk *disk = dev_to_disk(dev);
struct block_device *part;
char name_buf[BDEVNAME_SIZE];
char devt_buf[BDEVT_SIZE];
unsigned long idx;
@ -703,11 +674,10 @@ void __init printk_all_partitions(void)
xa_for_each(&disk->part_tbl, idx, part) {
if (!bdev_nr_sectors(part))
continue;
printk("%s%s %10llu %s %s",
printk("%s%s %10llu %pg %s",
bdev_is_partition(part) ? " " : "",
bdevt_str(part->bd_dev, devt_buf),
bdev_nr_sectors(part) >> 1,
disk_name(disk, part->bd_partno, name_buf),
bdev_nr_sectors(part) >> 1, part,
part->bd_meta_info ?
part->bd_meta_info->uuid : "");
if (bdev_is_partition(part))
@ -785,7 +755,6 @@ static int show_partition(struct seq_file *seqf, void *v)
struct gendisk *sgp = v;
struct block_device *part;
unsigned long idx;
char buf[BDEVNAME_SIZE];
/* Don't show non-partitionable removeable devices or empty devices */
if (!get_capacity(sgp) || (!disk_max_parts(sgp) &&
@ -798,10 +767,9 @@ static int show_partition(struct seq_file *seqf, void *v)
xa_for_each(&sgp->part_tbl, idx, part) {
if (!bdev_nr_sectors(part))
continue;
seq_printf(seqf, "%4d %7d %10llu %s\n",
seq_printf(seqf, "%4d %7d %10llu %pg\n",
MAJOR(part->bd_dev), MINOR(part->bd_dev),
bdev_nr_sectors(part) >> 1,
disk_name(sgp, part->bd_partno, buf));
bdev_nr_sectors(part) >> 1, part);
}
rcu_read_unlock();
return 0;
@ -968,6 +936,14 @@ static ssize_t disk_discard_alignment_show(struct device *dev,
return sprintf(buf, "%d\n", queue_discard_alignment(disk->queue));
}
static ssize_t diskseq_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct gendisk *disk = dev_to_disk(dev);
return sprintf(buf, "%llu\n", disk->diskseq);
}
static DEVICE_ATTR(range, 0444, disk_range_show, NULL);
static DEVICE_ATTR(ext_range, 0444, disk_ext_range_show, NULL);
static DEVICE_ATTR(removable, 0444, disk_removable_show, NULL);
@ -980,6 +956,7 @@ static DEVICE_ATTR(capability, 0444, disk_capability_show, NULL);
static DEVICE_ATTR(stat, 0444, part_stat_show, NULL);
static DEVICE_ATTR(inflight, 0444, part_inflight_show, NULL);
static DEVICE_ATTR(badblocks, 0644, disk_badblocks_show, disk_badblocks_store);
static DEVICE_ATTR(diskseq, 0444, diskseq_show, NULL);
#ifdef CONFIG_FAIL_MAKE_REQUEST
ssize_t part_fail_show(struct device *dev,
@ -1025,6 +1002,7 @@ static struct attribute *disk_attrs[] = {
&dev_attr_events.attr,
&dev_attr_events_async.attr,
&dev_attr_events_poll_msecs.attr,
&dev_attr_diskseq.attr,
#ifdef CONFIG_FAIL_MAKE_REQUEST
&dev_attr_fail.attr,
#endif
@ -1074,17 +1052,24 @@ static void disk_release(struct device *dev)
might_sleep();
if (MAJOR(dev->devt) == BLOCK_EXT_MAJOR)
blk_free_ext_minor(MINOR(dev->devt));
disk_release_events(disk);
kfree(disk->random);
xa_destroy(&disk->part_tbl);
if (test_bit(GD_QUEUE_REF, &disk->state) && disk->queue)
blk_put_queue(disk->queue);
bdput(disk->part0); /* frees the disk */
disk->queue->disk = NULL;
blk_put_queue(disk->queue);
iput(disk->part0->bd_inode); /* frees the disk */
}
static int block_uevent(struct device *dev, struct kobj_uevent_env *env)
{
struct gendisk *disk = dev_to_disk(dev);
return add_uevent_var(env, "DISKSEQ=%llu", disk->diskseq);
}
struct class block_class = {
.name = "block",
.dev_uevent = block_uevent,
};
static char *block_devnode(struct device *dev, umode_t *mode,
@ -1116,7 +1101,6 @@ static int diskstats_show(struct seq_file *seqf, void *v)
{
struct gendisk *gp = v;
struct block_device *hd;
char buf[BDEVNAME_SIZE];
unsigned int inflight;
struct disk_stats stat;
unsigned long idx;
@ -1139,15 +1123,14 @@ static int diskstats_show(struct seq_file *seqf, void *v)
else
inflight = part_in_flight(hd);
seq_printf(seqf, "%4d %7d %s "
seq_printf(seqf, "%4d %7d %pg "
"%lu %lu %lu %u "
"%lu %lu %lu %u "
"%u %u %u "
"%lu %lu %lu %u "
"%lu %u"
"\n",
MAJOR(hd->bd_dev), MINOR(hd->bd_dev),
disk_name(gp, hd->bd_partno, buf),
MAJOR(hd->bd_dev), MINOR(hd->bd_dev), hd,
stat.ios[STAT_READ],
stat.merges[STAT_READ],
stat.sectors[STAT_READ],
@ -1239,17 +1222,25 @@ dev_t blk_lookup_devt(const char *name, int partno)
return devt;
}
struct gendisk *__alloc_disk_node(int minors, int node_id)
struct gendisk *__alloc_disk_node(struct request_queue *q, int node_id,
struct lock_class_key *lkclass)
{
struct gendisk *disk;
if (!blk_get_queue(q))
return NULL;
disk = kzalloc_node(sizeof(struct gendisk), GFP_KERNEL, node_id);
if (!disk)
return NULL;
goto out_put_queue;
disk->bdi = bdi_alloc(node_id);
if (!disk->bdi)
goto out_free_disk;
disk->part0 = bdev_alloc(disk, 0);
if (!disk->part0)
goto out_free_disk;
goto out_free_bdi;
disk->node_id = node_id;
mutex_init(&disk->open_mutex);
@ -1257,23 +1248,33 @@ struct gendisk *__alloc_disk_node(int minors, int node_id)
if (xa_insert(&disk->part_tbl, 0, disk->part0, GFP_KERNEL))
goto out_destroy_part_tbl;
disk->minors = minors;
rand_initialize_disk(disk);
disk_to_dev(disk)->class = &block_class;
disk_to_dev(disk)->type = &disk_type;
device_initialize(disk_to_dev(disk));
inc_diskseq(disk);
disk->queue = q;
q->disk = disk;
lockdep_init_map(&disk->lockdep_map, "(bio completion)", lkclass, 0);
#ifdef CONFIG_BLOCK_HOLDER_DEPRECATED
INIT_LIST_HEAD(&disk->slave_bdevs);
#endif
return disk;
out_destroy_part_tbl:
xa_destroy(&disk->part_tbl);
bdput(disk->part0);
iput(disk->part0->bd_inode);
out_free_bdi:
bdi_put(disk->bdi);
out_free_disk:
kfree(disk);
out_put_queue:
blk_put_queue(q);
return NULL;
}
EXPORT_SYMBOL(__alloc_disk_node);
struct gendisk *__blk_alloc_disk(int node)
struct gendisk *__blk_alloc_disk(int node, struct lock_class_key *lkclass)
{
struct request_queue *q;
struct gendisk *disk;
@ -1282,12 +1283,11 @@ struct gendisk *__blk_alloc_disk(int node)
if (!q)
return NULL;
disk = __alloc_disk_node(0, node);
disk = __alloc_disk_node(q, node, lkclass);
if (!disk) {
blk_cleanup_queue(q);
return NULL;
}
disk->queue = q;
return disk;
}
EXPORT_SYMBOL(__blk_alloc_disk);
@ -1362,3 +1362,8 @@ int bdev_read_only(struct block_device *bdev)
return bdev->bd_read_only || get_disk_ro(bdev->bd_disk);
}
EXPORT_SYMBOL(bdev_read_only);
void inc_diskseq(struct gendisk *disk)
{
disk->diskseq = atomic64_inc_return(&diskseq);
}

174
block/holder.c Normal file
View File

@ -0,0 +1,174 @@
// SPDX-License-Identifier: GPL-2.0-only
#include <linux/genhd.h>
struct bd_holder_disk {
struct list_head list;
struct block_device *bdev;
int refcnt;
};
static struct bd_holder_disk *bd_find_holder_disk(struct block_device *bdev,
struct gendisk *disk)
{
struct bd_holder_disk *holder;
list_for_each_entry(holder, &disk->slave_bdevs, list)
if (holder->bdev == bdev)
return holder;
return NULL;
}
static int add_symlink(struct kobject *from, struct kobject *to)
{
return sysfs_create_link(from, to, kobject_name(to));
}
static void del_symlink(struct kobject *from, struct kobject *to)
{
sysfs_remove_link(from, kobject_name(to));
}
static int __link_disk_holder(struct block_device *bdev, struct gendisk *disk)
{
int ret;
ret = add_symlink(disk->slave_dir, bdev_kobj(bdev));
if (ret)
return ret;
ret = add_symlink(bdev->bd_holder_dir, &disk_to_dev(disk)->kobj);
if (ret)
del_symlink(disk->slave_dir, bdev_kobj(bdev));
return ret;
}
/**
* bd_link_disk_holder - create symlinks between holding disk and slave bdev
* @bdev: the claimed slave bdev
* @disk: the holding disk
*
* DON'T USE THIS UNLESS YOU'RE ALREADY USING IT.
*
* This functions creates the following sysfs symlinks.
*
* - from "slaves" directory of the holder @disk to the claimed @bdev
* - from "holders" directory of the @bdev to the holder @disk
*
* For example, if /dev/dm-0 maps to /dev/sda and disk for dm-0 is
* passed to bd_link_disk_holder(), then:
*
* /sys/block/dm-0/slaves/sda --> /sys/block/sda
* /sys/block/sda/holders/dm-0 --> /sys/block/dm-0
*
* The caller must have claimed @bdev before calling this function and
* ensure that both @bdev and @disk are valid during the creation and
* lifetime of these symlinks.
*
* CONTEXT:
* Might sleep.
*
* RETURNS:
* 0 on success, -errno on failure.
*/
int bd_link_disk_holder(struct block_device *bdev, struct gendisk *disk)
{
struct bd_holder_disk *holder;
int ret = 0;
mutex_lock(&disk->open_mutex);
WARN_ON_ONCE(!bdev->bd_holder);
/* FIXME: remove the following once add_disk() handles errors */
if (WARN_ON(!bdev->bd_holder_dir))
goto out_unlock;
holder = bd_find_holder_disk(bdev, disk);
if (holder) {
holder->refcnt++;
goto out_unlock;
}
holder = kzalloc(sizeof(*holder), GFP_KERNEL);
if (!holder) {
ret = -ENOMEM;
goto out_unlock;
}
INIT_LIST_HEAD(&holder->list);
holder->bdev = bdev;
holder->refcnt = 1;
if (disk->slave_dir) {
ret = __link_disk_holder(bdev, disk);
if (ret) {
kfree(holder);
goto out_unlock;
}
}
list_add(&holder->list, &disk->slave_bdevs);
/*
* del_gendisk drops the initial reference to bd_holder_dir, so we need
* to keep our own here to allow for cleanup past that point.
*/
kobject_get(bdev->bd_holder_dir);
out_unlock:
mutex_unlock(&disk->open_mutex);
return ret;
}
EXPORT_SYMBOL_GPL(bd_link_disk_holder);
static void __unlink_disk_holder(struct block_device *bdev,
struct gendisk *disk)
{
del_symlink(disk->slave_dir, bdev_kobj(bdev));
del_symlink(bdev->bd_holder_dir, &disk_to_dev(disk)->kobj);
}
/**
* bd_unlink_disk_holder - destroy symlinks created by bd_link_disk_holder()
* @bdev: the calimed slave bdev
* @disk: the holding disk
*
* DON'T USE THIS UNLESS YOU'RE ALREADY USING IT.
*
* CONTEXT:
* Might sleep.
*/
void bd_unlink_disk_holder(struct block_device *bdev, struct gendisk *disk)
{
struct bd_holder_disk *holder;
mutex_lock(&disk->open_mutex);
holder = bd_find_holder_disk(bdev, disk);
if (!WARN_ON_ONCE(holder == NULL) && !--holder->refcnt) {
if (disk->slave_dir)
__unlink_disk_holder(bdev, disk);
kobject_put(bdev->bd_holder_dir);
list_del_init(&holder->list);
kfree(holder);
}
mutex_unlock(&disk->open_mutex);
}
EXPORT_SYMBOL_GPL(bd_unlink_disk_holder);
int bd_register_pending_holders(struct gendisk *disk)
{
struct bd_holder_disk *holder;
int ret;
mutex_lock(&disk->open_mutex);
list_for_each_entry(holder, &disk->slave_bdevs, list) {
ret = __link_disk_holder(holder->bdev, disk);
if (ret)
goto out_undo;
}
mutex_unlock(&disk->open_mutex);
return 0;
out_undo:
list_for_each_entry_continue_reverse(holder, &disk->slave_bdevs, list)
__unlink_disk_holder(holder->bdev, disk);
mutex_unlock(&disk->open_mutex);
return ret;
}

16
block/ioctl.c
View File

@ -16,6 +16,7 @@
static int blkpg_do_ioctl(struct block_device *bdev,
struct blkpg_partition __user *upart, int op)
{
struct gendisk *disk = bdev->bd_disk;
struct blkpg_partition p;
long long start, length;
@ -30,7 +31,7 @@ static int blkpg_do_ioctl(struct block_device *bdev,
return -EINVAL;
if (op == BLKPG_DEL_PARTITION)
return bdev_del_partition(bdev, p.pno);
return bdev_del_partition(disk, p.pno);
start = p.start >> SECTOR_SHIFT;
length = p.length >> SECTOR_SHIFT;
@ -40,9 +41,9 @@ static int blkpg_do_ioctl(struct block_device *bdev,
/* check if partition is aligned to blocksize */
if (p.start & (bdev_logical_block_size(bdev) - 1))
return -EINVAL;
return bdev_add_partition(bdev, p.pno, start, length);
return bdev_add_partition(disk, p.pno, start, length);
case BLKPG_RESIZE_PARTITION:
return bdev_resize_partition(bdev, p.pno, start, length);
return bdev_resize_partition(disk, p.pno, start, length);
default:
return -EINVAL;
}
@ -469,6 +470,8 @@ static int blkdev_common_ioctl(struct block_device *bdev, fmode_t mode,
BLKDEV_DISCARD_SECURE);
case BLKZEROOUT:
return blk_ioctl_zeroout(bdev, mode, arg);
case BLKGETDISKSEQ:
return put_u64(argp, bdev->bd_disk->diskseq);
case BLKREPORTZONE:
return blkdev_report_zones_ioctl(bdev, mode, cmd, arg);
case BLKRESETZONE:
@ -504,7 +507,7 @@ static int blkdev_common_ioctl(struct block_device *bdev, fmode_t mode,
case BLKFRASET:
if(!capable(CAP_SYS_ADMIN))
return -EACCES;
bdev->bd_bdi->ra_pages = (arg * 512) / PAGE_SIZE;
bdev->bd_disk->bdi->ra_pages = (arg * 512) / PAGE_SIZE;
return 0;
case BLKRRPART:
return blkdev_reread_part(bdev, mode);
@ -554,7 +557,8 @@ int blkdev_ioctl(struct block_device *bdev, fmode_t mode, unsigned cmd,
case BLKFRAGET:
if (!argp)
return -EINVAL;
return put_long(argp, (bdev->bd_bdi->ra_pages*PAGE_SIZE) / 512);
return put_long(argp,
(bdev->bd_disk->bdi->ra_pages * PAGE_SIZE) / 512);
case BLKGETSIZE:
size = i_size_read(bdev->bd_inode);
if ((size >> 9) > ~0UL)
@ -626,7 +630,7 @@ long compat_blkdev_ioctl(struct file *file, unsigned cmd, unsigned long arg)
if (!argp)
return -EINVAL;
return compat_put_long(argp,
(bdev->bd_bdi->ra_pages * PAGE_SIZE) / 512);
(bdev->bd_disk->bdi->ra_pages * PAGE_SIZE) / 512);
case BLKGETSIZE:
size = i_size_read(bdev->bd_inode);
if ((size >> 9) > ~0UL)

9
block/ioprio.c
View File

@ -74,9 +74,8 @@ int ioprio_check_cap(int ioprio)
fallthrough;
/* rt has prio field too */
case IOPRIO_CLASS_BE:
if (data >= IOPRIO_BE_NR || data < 0)
if (data >= IOPRIO_NR_LEVELS || data < 0)
return -EINVAL;
break;
case IOPRIO_CLASS_IDLE:
break;
@ -171,7 +170,7 @@ static int get_task_ioprio(struct task_struct *p)
ret = security_task_getioprio(p);
if (ret)
goto out;
ret = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_NONE, IOPRIO_NORM);
ret = IOPRIO_DEFAULT;
task_lock(p);
if (p->io_context)
ret = p->io_context->ioprio;
@ -183,9 +182,9 @@ static int get_task_ioprio(struct task_struct *p)
int ioprio_best(unsigned short aprio, unsigned short bprio)
{
if (!ioprio_valid(aprio))
aprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_NORM);
aprio = IOPRIO_DEFAULT;
if (!ioprio_valid(bprio))
bprio = IOPRIO_PRIO_VALUE(IOPRIO_CLASS_BE, IOPRIO_NORM);
bprio = IOPRIO_DEFAULT;
return min(aprio, bprio);
}

2
block/mq-deadline.c
View File

@ -629,6 +629,8 @@ static int dd_request_merge(struct request_queue *q, struct request **rq,
if (elv_bio_merge_ok(__rq, bio)) {
*rq = __rq;
if (blk_discard_mergable(__rq))
return ELEVATOR_DISCARD_MERGE;
return ELEVATOR_FRONT_MERGE;
}
}

1
block/partitions/Kconfig
View File

@ -264,7 +264,6 @@ config SYSV68_PARTITION
config CMDLINE_PARTITION
bool "Command line partition support" if PARTITION_ADVANCED
select BLK_CMDLINE_PARSER
help
Say Y here if you want to read the partition table from bootargs.
The format for the command line is just like mtdparts.

4
block/partitions/acorn.c
View File

@ -275,7 +275,7 @@ int adfspart_check_ADFS(struct parsed_partitions *state)
/*
* Work out start of non-adfs partition.
*/
nr_sects = (state->bdev->bd_inode->i_size >> 9) - start_sect;
nr_sects = get_capacity(state->disk) - start_sect;
if (start_sect) {
switch (id) {
@ -540,7 +540,7 @@ int adfspart_check_EESOX(struct parsed_partitions *state)
if (i != 0) {
sector_t size;
size = get_capacity(state->bdev->bd_disk);
size = get_capacity(state->disk);
put_partition(state, slot++, start, size - start);
strlcat(state->pp_buf, "\n", PAGE_SIZE);
}

20
block/partitions/aix.c
View File

@ -66,22 +66,6 @@ struct pvd {
#define LVM_MAXLVS 256
/**
* last_lba(): return number of last logical block of device
* @bdev: block device
*
* Description: Returns last LBA value on success, 0 on error.
* This is stored (by sd and ide-geometry) in
* the part[0] entry for this disk, and is the number of
* physical sectors available on the disk.
*/
static u64 last_lba(struct block_device *bdev)
{
if (!bdev || !bdev->bd_inode)
return 0;
return (bdev->bd_inode->i_size >> 9) - 1ULL;
}
/**
* read_lba(): Read bytes from disk, starting at given LBA
* @state
@ -89,7 +73,7 @@ static u64 last_lba(struct block_device *bdev)
* @buffer
* @count
*
* Description: Reads @count bytes from @state->bdev into @buffer.
* Description: Reads @count bytes from @state->disk into @buffer.
* Returns number of bytes read on success, 0 on error.
*/
static size_t read_lba(struct parsed_partitions *state, u64 lba, u8 *buffer,
@ -97,7 +81,7 @@ static size_t read_lba(struct parsed_partitions *state, u64 lba, u8 *buffer,
{
size_t totalreadcount = 0;
if (!buffer || lba + count / 512 > last_lba(state->bdev))
if (!buffer || lba + count / 512 > get_capacity(state->disk) - 1ULL)
return 0;
while (count) {

7
block/partitions/amiga.c
View File

@ -34,7 +34,6 @@ int amiga_partition(struct parsed_partitions *state)
int start_sect, nr_sects, blk, part, res = 0;
int blksize = 1; /* Multiplier for disk block size */
int slot = 1;
char b[BDEVNAME_SIZE];
for (blk = 0; ; blk++, put_dev_sector(sect)) {
if (blk == RDB_ALLOCATION_LIMIT)
@ -42,7 +41,7 @@ int amiga_partition(struct parsed_partitions *state)
data = read_part_sector(state, blk, &sect);
if (!data) {
pr_err("Dev %s: unable to read RDB block %d\n",
bdevname(state->bdev, b), blk);
state->disk->disk_name, blk);
res = -1;
goto rdb_done;
}
@ -64,7 +63,7 @@ int amiga_partition(struct parsed_partitions *state)
}
pr_err("Dev %s: RDB in block %d has bad checksum\n",
bdevname(state->bdev, b), blk);
state->disk->disk_name, blk);
}
/* blksize is blocks per 512 byte standard block */
@ -84,7 +83,7 @@ int amiga_partition(struct parsed_partitions *state)
data = read_part_sector(state, blk, &sect);
if (!data) {
pr_err("Dev %s: unable to read partition block %d\n",
bdevname(state->bdev, b), blk);
state->disk->disk_name, blk);
res = -1;
goto rdb_done;
}

4
block/partitions/atari.c
View File

@ -47,7 +47,7 @@ int atari_partition(struct parsed_partitions *state)
* ATARI partition scheme supports 512 lba only. If this is not
* the case, bail early to avoid miscalculating hd_size.
*/
if (bdev_logical_block_size(state->bdev) != 512)
if (queue_logical_block_size(state->disk->queue) != 512)
return 0;
rs = read_part_sector(state, 0, &sect);
@ -55,7 +55,7 @@ int atari_partition(struct parsed_partitions *state)
return -1;
/* Verify this is an Atari rootsector: */
hd_size = state->bdev->bd_inode->i_size >> 9;
hd_size = get_capacity(state->disk);
if (!VALID_PARTITION(&rs->part[0], hd_size) &&
!VALID_PARTITION(&rs->part[1], hd_size) &&
!VALID_PARTITION(&rs->part[2], hd_size) &&

2
block/partitions/check.h
View File

@ -9,7 +9,7 @@
* description.
*/
struct parsed_partitions {
struct block_device *bdev;
struct gendisk *disk;
char name[BDEVNAME_SIZE];
struct {
sector_t from;

275
block/partitions/cmdline.c
View File

@ -14,20 +14,248 @@
* For further information, see "Documentation/block/cmdline-partition.rst"
*
*/
#include <linux/cmdline-parser.h>
#include <linux/blkdev.h>
#include <linux/fs.h>
#include <linux/slab.h>
#include "check.h"
/* partition flags */
#define PF_RDONLY 0x01 /* Device is read only */
#define PF_POWERUP_LOCK 0x02 /* Always locked after reset */
struct cmdline_subpart {
char name[BDEVNAME_SIZE]; /* partition name, such as 'rootfs' */
sector_t from;
sector_t size;
int flags;
struct cmdline_subpart *next_subpart;
};
struct cmdline_parts {
char name[BDEVNAME_SIZE]; /* block device, such as 'mmcblk0' */
unsigned int nr_subparts;
struct cmdline_subpart *subpart;
struct cmdline_parts *next_parts;
};
static int parse_subpart(struct cmdline_subpart **subpart, char *partdef)
{
int ret = 0;
struct cmdline_subpart *new_subpart;
*subpart = NULL;
new_subpart = kzalloc(sizeof(struct cmdline_subpart), GFP_KERNEL);
if (!new_subpart)
return -ENOMEM;
if (*partdef == '-') {
new_subpart->size = (sector_t)(~0ULL);
partdef++;
} else {
new_subpart->size = (sector_t)memparse(partdef, &partdef);
if (new_subpart->size < (sector_t)PAGE_SIZE) {
pr_warn("cmdline partition size is invalid.");
ret = -EINVAL;
goto fail;
}
}
if (*partdef == '@') {
partdef++;
new_subpart->from = (sector_t)memparse(partdef, &partdef);
} else {
new_subpart->from = (sector_t)(~0ULL);
}
if (*partdef == '(') {
int length;
char *next = strchr(++partdef, ')');
if (!next) {
pr_warn("cmdline partition format is invalid.");
ret = -EINVAL;
goto fail;
}
length = min_t(int, next - partdef,
sizeof(new_subpart->name) - 1);
strncpy(new_subpart->name, partdef, length);
new_subpart->name[length] = '\0';
partdef = ++next;
} else
new_subpart->name[0] = '\0';
new_subpart->flags = 0;
if (!strncmp(partdef, "ro", 2)) {
new_subpart->flags |= PF_RDONLY;
partdef += 2;
}
if (!strncmp(partdef, "lk", 2)) {
new_subpart->flags |= PF_POWERUP_LOCK;
partdef += 2;
}
*subpart = new_subpart;
return 0;
fail:
kfree(new_subpart);
return ret;
}
static void free_subpart(struct cmdline_parts *parts)
{
struct cmdline_subpart *subpart;
while (parts->subpart) {
subpart = parts->subpart;
parts->subpart = subpart->next_subpart;
kfree(subpart);
}
}
static int parse_parts(struct cmdline_parts **parts, const char *bdevdef)
{
int ret = -EINVAL;
char *next;
int length;
struct cmdline_subpart **next_subpart;
struct cmdline_parts *newparts;
char buf[BDEVNAME_SIZE + 32 + 4];
*parts = NULL;
newparts = kzalloc(sizeof(struct cmdline_parts), GFP_KERNEL);
if (!newparts)
return -ENOMEM;
next = strchr(bdevdef, ':');
if (!next) {
pr_warn("cmdline partition has no block device.");
goto fail;
}
length = min_t(int, next - bdevdef, sizeof(newparts->name) - 1);
strncpy(newparts->name, bdevdef, length);
newparts->name[length] = '\0';
newparts->nr_subparts = 0;
next_subpart = &newparts->subpart;
while (next && *(++next)) {
bdevdef = next;
next = strchr(bdevdef, ',');
length = (!next) ? (sizeof(buf) - 1) :
min_t(int, next - bdevdef, sizeof(buf) - 1);
strncpy(buf, bdevdef, length);
buf[length] = '\0';
ret = parse_subpart(next_subpart, buf);
if (ret)
goto fail;
newparts->nr_subparts++;
next_subpart = &(*next_subpart)->next_subpart;
}
if (!newparts->subpart) {
pr_warn("cmdline partition has no valid partition.");
ret = -EINVAL;
goto fail;
}
*parts = newparts;
return 0;
fail:
free_subpart(newparts);
kfree(newparts);
return ret;
}
static void cmdline_parts_free(struct cmdline_parts **parts)
{
struct cmdline_parts *next_parts;
while (*parts) {
next_parts = (*parts)->next_parts;
free_subpart(*parts);
kfree(*parts);
*parts = next_parts;
}
}
static int cmdline_parts_parse(struct cmdline_parts **parts,
const char *cmdline)
{
int ret;
char *buf;
char *pbuf;
char *next;
struct cmdline_parts **next_parts;
*parts = NULL;
next = pbuf = buf = kstrdup(cmdline, GFP_KERNEL);
if (!buf)
return -ENOMEM;
next_parts = parts;
while (next && *pbuf) {
next = strchr(pbuf, ';');
if (next)
*next = '\0';
ret = parse_parts(next_parts, pbuf);
if (ret)
goto fail;
if (next)
pbuf = ++next;
next_parts = &(*next_parts)->next_parts;
}
if (!*parts) {
pr_warn("cmdline partition has no valid partition.");
ret = -EINVAL;
goto fail;
}
ret = 0;
done:
kfree(buf);
return ret;
fail:
cmdline_parts_free(parts);
goto done;
}
static struct cmdline_parts *cmdline_parts_find(struct cmdline_parts *parts,
const char *bdev)
{
while (parts && strncmp(bdev, parts->name, sizeof(parts->name)))
parts = parts->next_parts;
return parts;
}
static char *cmdline;
static struct cmdline_parts *bdev_parts;
static int add_part(int slot, struct cmdline_subpart *subpart, void *param)
static int add_part(int slot, struct cmdline_subpart *subpart,
struct parsed_partitions *state)
{
int label_min;
struct partition_meta_info *info;
char tmp[sizeof(info->volname) + 4];
struct parsed_partitions *state = (struct parsed_partitions *)param;
if (slot >= state->limit)
return 1;
@ -50,6 +278,35 @@ static int add_part(int slot, struct cmdline_subpart *subpart, void *param)
return 0;
}
static int cmdline_parts_set(struct cmdline_parts *parts, sector_t disk_size,
struct parsed_partitions *state)
{
sector_t from = 0;
struct cmdline_subpart *subpart;
int slot = 1;
for (subpart = parts->subpart; subpart;
subpart = subpart->next_subpart, slot++) {
if (subpart->from == (sector_t)(~0ULL))
subpart->from = from;
else
from = subpart->from;
if (from >= disk_size)
break;
if (subpart->size > (disk_size - from))
subpart->size = disk_size - from;
from += subpart->size;
if (add_part(slot, subpart, state))
break;
}
return slot;
}
static int __init cmdline_parts_setup(char *s)
{
cmdline = s;
@ -123,7 +380,6 @@ static void cmdline_parts_verifier(int slot, struct parsed_partitions *state)
int cmdline_partition(struct parsed_partitions *state)
{
sector_t disk_size;
char bdev[BDEVNAME_SIZE];
struct cmdline_parts *parts;
if (cmdline) {
@ -140,14 +396,13 @@ int cmdline_partition(struct parsed_partitions *state)
if (!bdev_parts)
return 0;
bdevname(state->bdev, bdev);
parts = cmdline_parts_find(bdev_parts, bdev);
parts = cmdline_parts_find(bdev_parts, state->disk->disk_name);
if (!parts)
return 0;
disk_size = get_capacity(state->bdev->bd_disk) << 9;
disk_size = get_capacity(state->disk) << 9;
cmdline_parts_set(parts, disk_size, 1, add_part, (void *)state);
cmdline_parts_set(parts, disk_size, state);
cmdline_parts_verifier(1, state);
strlcat(state->pp_buf, "\n", PAGE_SIZE);

73
block/partitions/core.c
View File

@ -135,8 +135,8 @@ static struct parsed_partitions *check_partition(struct gendisk *hd)
}
state->pp_buf[0] = '\0';
state->bdev = hd->part0;
disk_name(hd, 0, state->name);
state->disk = hd;
snprintf(state->name, BDEVNAME_SIZE, "%s", hd->disk_name);
snprintf(state->pp_buf, PAGE_SIZE, " %s:", state->name);
if (isdigit(state->name[strlen(state->name)-1]))
sprintf(state->name, "p");
@ -259,9 +259,8 @@ static const struct attribute_group *part_attr_groups[] = {
static void part_release(struct device *dev)
{
if (MAJOR(dev->devt) == BLOCK_EXT_MAJOR)
blk_free_ext_minor(MINOR(dev->devt));
bdput(dev_to_bdev(dev));
put_disk(dev_to_bdev(dev)->bd_disk);
iput(dev_to_bdev(dev)->bd_inode);
}
static int part_uevent(struct device *dev, struct kobj_uevent_env *env)
@ -281,12 +280,10 @@ struct device_type part_type = {
.uevent = part_uevent,
};
/*
* Must be called either with open_mutex held, before a disk can be opened or
* after all disk users are gone.
*/
static void delete_partition(struct block_device *part)
{
lockdep_assert_held(&part->bd_disk->open_mutex);
fsync_bdev(part);
__invalidate_device(part, true);
@ -351,20 +348,17 @@ static struct block_device *add_partition(struct gendisk *disk, int partno,
if (xa_load(&disk->part_tbl, partno))
return ERR_PTR(-EBUSY);
/* ensure we always have a reference to the whole disk */
get_device(disk_to_dev(disk));
err = -ENOMEM;
bdev = bdev_alloc(disk, partno);
if (!bdev)
return ERR_PTR(-ENOMEM);
goto out_put_disk;
bdev->bd_start_sect = start;
bdev_set_nr_sectors(bdev, len);
if (info) {
err = -ENOMEM;
bdev->bd_meta_info = kmemdup(info, sizeof(*info), GFP_KERNEL);
if (!bdev->bd_meta_info)
goto out_bdput;
}
pdev = &bdev->bd_device;
dname = dev_name(ddev);
if (isdigit(dname[strlen(dname) - 1]))
@ -388,6 +382,13 @@ static struct block_device *add_partition(struct gendisk *disk, int partno,
}
pdev->devt = devt;
if (info) {
err = -ENOMEM;
bdev->bd_meta_info = kmemdup(info, sizeof(*info), GFP_KERNEL);
if (!bdev->bd_meta_info)
goto out_put;
}
/* delay uevent until 'holders' subdir is created */
dev_set_uevent_suppress(pdev, 1);
err = device_add(pdev);
@ -417,14 +418,13 @@ static struct block_device *add_partition(struct gendisk *disk, int partno,
kobject_uevent(&pdev->kobj, KOBJ_ADD);
return bdev;
out_bdput:
bdput(bdev);
return ERR_PTR(err);
out_del:
kobject_put(bdev->bd_holder_dir);
device_del(pdev);
out_put:
put_device(pdev);
out_put_disk:
put_disk(disk);
return ERR_PTR(err);
}
@ -449,15 +449,14 @@ static bool partition_overlaps(struct gendisk *disk, sector_t start,
return overlap;
}
int bdev_add_partition(struct block_device *bdev, int partno,
sector_t start, sector_t length)
int bdev_add_partition(struct gendisk *disk, int partno, sector_t start,
sector_t length)
{
struct block_device *part;
struct gendisk *disk = bdev->bd_disk;
int ret;
mutex_lock(&disk->open_mutex);
if (!(disk->flags & GENHD_FL_UP)) {
if (!disk_live(disk)) {
ret = -ENXIO;
goto out;
}
@ -475,13 +474,13 @@ int bdev_add_partition(struct block_device *bdev, int partno,
return ret;
}
int bdev_del_partition(struct block_device *bdev, int partno)
int bdev_del_partition(struct gendisk *disk, int partno)
{
struct block_device *part = NULL;
int ret = -ENXIO;
mutex_lock(&bdev->bd_disk->open_mutex);
part = xa_load(&bdev->bd_disk->part_tbl, partno);
mutex_lock(&disk->open_mutex);
part = xa_load(&disk->part_tbl, partno);
if (!part)
goto out_unlock;
@ -492,18 +491,18 @@ int bdev_del_partition(struct block_device *bdev, int partno)
delete_partition(part);
ret = 0;
out_unlock:
mutex_unlock(&bdev->bd_disk->open_mutex);
mutex_unlock(&disk->open_mutex);
return ret;
}
int bdev_resize_partition(struct block_device *bdev, int partno,
sector_t start, sector_t length)
int bdev_resize_partition(struct gendisk *disk, int partno, sector_t start,
sector_t length)
{
struct block_device *part = NULL;
int ret = -ENXIO;
mutex_lock(&bdev->bd_disk->open_mutex);
part = xa_load(&bdev->bd_disk->part_tbl, partno);
mutex_lock(&disk->open_mutex);
part = xa_load(&disk->part_tbl, partno);
if (!part)
goto out_unlock;
@ -512,14 +511,14 @@ int bdev_resize_partition(struct block_device *bdev, int partno,
goto out_unlock;
ret = -EBUSY;
if (partition_overlaps(bdev->bd_disk, start, length, partno))
if (partition_overlaps(disk, start, length, partno))
goto out_unlock;
bdev_set_nr_sectors(part, length);
ret = 0;
out_unlock:
mutex_unlock(&bdev->bd_disk->open_mutex);
mutex_unlock(&disk->open_mutex);
return ret;
}
@ -667,7 +666,7 @@ int bdev_disk_changed(struct gendisk *disk, bool invalidate)
lockdep_assert_held(&disk->open_mutex);
if (!(disk->flags & GENHD_FL_UP))
if (!disk_live(disk))
return -ENXIO;
rescan:
@ -715,10 +714,10 @@ EXPORT_SYMBOL_GPL(bdev_disk_changed);
void *read_part_sector(struct parsed_partitions *state, sector_t n, Sector *p)
{
struct address_space *mapping = state->bdev->bd_inode->i_mapping;
struct address_space *mapping = state->disk->part0->bd_inode->i_mapping;
struct page *page;
if (n >= get_capacity(state->bdev->bd_disk)) {
if (n >= get_capacity(state->disk)) {
state->access_beyond_eod = true;
return NULL;
}

48
block/partitions/efi.c
View File

@ -124,19 +124,17 @@ efi_crc32(const void *buf, unsigned long len)
/**
* last_lba(): return number of last logical block of device
* @bdev: block device
* @disk: block device
*
* Description: Returns last LBA value on success, 0 on error.
* This is stored (by sd and ide-geometry) in
* the part[0] entry for this disk, and is the number of
* physical sectors available on the disk.
*/
static u64 last_lba(struct block_device *bdev)
static u64 last_lba(struct gendisk *disk)
{
if (!bdev || !bdev->bd_inode)
return 0;
return div_u64(bdev->bd_inode->i_size,
bdev_logical_block_size(bdev)) - 1ULL;
return div_u64(disk->part0->bd_inode->i_size,
queue_logical_block_size(disk->queue)) - 1ULL;
}
static inline int pmbr_part_valid(gpt_mbr_record *part)
@ -231,17 +229,17 @@ static int is_pmbr_valid(legacy_mbr *mbr, sector_t total_sectors)
* @buffer: destination buffer
* @count: bytes to read
*
* Description: Reads @count bytes from @state->bdev into @buffer.
* Description: Reads @count bytes from @state->disk into @buffer.
* Returns number of bytes read on success, 0 on error.
*/
static size_t read_lba(struct parsed_partitions *state,
u64 lba, u8 *buffer, size_t count)
{
size_t totalreadcount = 0;
struct block_device *bdev = state->bdev;
sector_t n = lba * (bdev_logical_block_size(bdev) / 512);
sector_t n = lba *
(queue_logical_block_size(state->disk->queue) / 512);
if (!buffer || lba > last_lba(bdev))
if (!buffer || lba > last_lba(state->disk))
return 0;
while (count) {
@ -302,14 +300,14 @@ static gpt_entry *alloc_read_gpt_entries(struct parsed_partitions *state,
* @lba: the Logical Block Address of the partition table
*
* Description: returns GPT header on success, NULL on error. Allocates
* and fills a GPT header starting at @ from @state->bdev.
* and fills a GPT header starting at @ from @state->disk.
* Note: remember to free gpt when finished with it.
*/
static gpt_header *alloc_read_gpt_header(struct parsed_partitions *state,
u64 lba)
{
gpt_header *gpt;
unsigned ssz = bdev_logical_block_size(state->bdev);
unsigned ssz = queue_logical_block_size(state->disk->queue);
gpt = kmalloc(ssz, GFP_KERNEL);
if (!gpt)
@ -356,10 +354,10 @@ static int is_gpt_valid(struct parsed_partitions *state, u64 lba,
/* Check the GUID Partition Table header size is too big */
if (le32_to_cpu((*gpt)->header_size) >
bdev_logical_block_size(state->bdev)) {
queue_logical_block_size(state->disk->queue)) {
pr_debug("GUID Partition Table Header size is too large: %u > %u\n",
le32_to_cpu((*gpt)->header_size),
bdev_logical_block_size(state->bdev));
queue_logical_block_size(state->disk->queue));
goto fail;
}
@ -395,7 +393,7 @@ static int is_gpt_valid(struct parsed_partitions *state, u64 lba,
/* Check the first_usable_lba and last_usable_lba are
* within the disk.
*/
lastlba = last_lba(state->bdev);
lastlba = last_lba(state->disk);
if (le64_to_cpu((*gpt)->first_usable_lba) > lastlba) {
pr_debug("GPT: first_usable_lba incorrect: %lld > %lld\n",
(unsigned long long)le64_to_cpu((*gpt)->first_usable_lba),
@ -587,13 +585,15 @@ static int find_valid_gpt(struct parsed_partitions *state, gpt_header **gpt,
gpt_header *pgpt = NULL, *agpt = NULL;
gpt_entry *pptes = NULL, *aptes = NULL;
legacy_mbr *legacymbr;
sector_t total_sectors = i_size_read(state->bdev->bd_inode) >> 9;
struct gendisk *disk = state->disk;
const struct block_device_operations *fops = disk->fops;
sector_t total_sectors = get_capacity(state->disk);
u64 lastlba;
if (!ptes)
return 0;
lastlba = last_lba(state->bdev);
lastlba = last_lba(state->disk);
if (!force_gpt) {
/* This will be added to the EFI Spec. per Intel after v1.02. */
legacymbr = kzalloc(sizeof(*legacymbr), GFP_KERNEL);
@ -621,6 +621,16 @@ static int find_valid_gpt(struct parsed_partitions *state, gpt_header **gpt,
if (!good_agpt && force_gpt)
good_agpt = is_gpt_valid(state, lastlba, &agpt, &aptes);
if (!good_agpt && force_gpt && fops->alternative_gpt_sector) {
sector_t agpt_sector;
int err;
err = fops->alternative_gpt_sector(disk, &agpt_sector);
if (!err)
good_agpt = is_gpt_valid(state, agpt_sector,
&agpt, &aptes);
}
/* The obviously unsuccessful case */
if (!good_pgpt && !good_agpt)
goto fail;
@ -705,7 +715,7 @@ int efi_partition(struct parsed_partitions *state)
gpt_header *gpt = NULL;
gpt_entry *ptes = NULL;
u32 i;
unsigned ssz = bdev_logical_block_size(state->bdev) / 512;
unsigned ssz = queue_logical_block_size(state->disk->queue) / 512;
if (!find_valid_gpt(state, &gpt, &ptes) || !gpt || !ptes) {
kfree(gpt);
@ -722,7 +732,7 @@ int efi_partition(struct parsed_partitions *state)
u64 size = le64_to_cpu(ptes[i].ending_lba) -
le64_to_cpu(ptes[i].starting_lba) + 1ULL;
if (!is_pte_valid(&ptes[i], last_lba(state->bdev)))
if (!is_pte_valid(&ptes[i], last_lba(state->disk)))
continue;
put_partition(state, i+1, start * ssz, size * ssz);

4
block/partitions/ibm.c
View File

@ -290,8 +290,8 @@ static int find_cms1_partitions(struct parsed_partitions *state,
int ibm_partition(struct parsed_partitions *state)
{
int (*fn)(struct gendisk *disk, dasd_information2_t *info);
struct block_device *bdev = state->bdev;
struct gendisk *disk = bdev->bd_disk;
struct gendisk *disk = state->disk;
struct block_device *bdev = disk->part0;
int blocksize, res;
loff_t i_size, offset, size;
dasd_information2_t *info;

18
block/partitions/ldm.c
View File

@ -304,7 +304,7 @@ static bool ldm_validate_privheads(struct parsed_partitions *state,
}
}
num_sects = state->bdev->bd_inode->i_size >> 9;
num_sects = get_capacity(state->disk);
if ((ph[0]->config_start > num_sects) ||
((ph[0]->config_start + ph[0]->config_size) > num_sects)) {
@ -339,11 +339,11 @@ static bool ldm_validate_privheads(struct parsed_partitions *state,
/**
* ldm_validate_tocblocks - Validate the table of contents and its backups
* @state: Partition check state including device holding the LDM Database
* @base: Offset, into @state->bdev, of the database
* @base: Offset, into @state->disk, of the database
* @ldb: Cache of the database structures
*
* Find and compare the four tables of contents of the LDM Database stored on
* @state->bdev and return the parsed information into @toc1.
* @state->disk and return the parsed information into @toc1.
*
* The offsets and sizes of the configs are range-checked against a privhead.
*
@ -486,8 +486,8 @@ static bool ldm_validate_vmdb(struct parsed_partitions *state,
* only likely to happen if the underlying device is strange. If that IS
* the case we should return zero to let someone else try.
*
* Return: 'true' @state->bdev is a dynamic disk
* 'false' @state->bdev is not a dynamic disk, or an error occurred
* Return: 'true' @state->disk is a dynamic disk
* 'false' @state->disk is not a dynamic disk, or an error occurred
*/
static bool ldm_validate_partition_table(struct parsed_partitions *state)
{
@ -1340,7 +1340,7 @@ static bool ldm_frag_commit (struct list_head *frags, struct ldmdb *ldb)
/**
* ldm_get_vblks - Read the on-disk database of VBLKs into memory
* @state: Partition check state including device holding the LDM Database
* @base: Offset, into @state->bdev, of the database
* @base: Offset, into @state->disk, of the database
* @ldb: Cache of the database structures
*
* To use the information from the VBLKs, they need to be read from the disk,
@ -1432,10 +1432,10 @@ static void ldm_free_vblks (struct list_head *lh)
* example, if the device is hda, we would have: hda1: LDM database, hda2, hda3,
* and so on: the actual data containing partitions.
*
* Return: 1 Success, @state->bdev is a dynamic disk and we handled it
* 0 Success, @state->bdev is not a dynamic disk
* Return: 1 Success, @state->disk is a dynamic disk and we handled it
* 0 Success, @state->disk is not a dynamic disk
* -1 An error occurred before enough information had been read
* Or @state->bdev is a dynamic disk, but it may be corrupted
* Or @state->disk is a dynamic disk, but it may be corrupted
*/
int ldm_partition(struct parsed_partitions *state)
{

2
block/partitions/mac.c
View File

@ -133,7 +133,7 @@ int mac_partition(struct parsed_partitions *state)
}
#ifdef CONFIG_PPC_PMAC
if (found_root_goodness)
note_bootable_part(state->bdev->bd_dev, found_root,
note_bootable_part(state->disk->part0->bd_dev, found_root,
found_root_goodness);
#endif

6
block/partitions/msdos.c
View File

@ -135,11 +135,12 @@ static void parse_extended(struct parsed_partitions *state,
Sector sect;
unsigned char *data;
sector_t this_sector, this_size;
sector_t sector_size = bdev_logical_block_size(state->bdev) / 512;
sector_t sector_size;
int loopct = 0; /* number of links followed
without finding a data partition */
int i;
sector_size = queue_logical_block_size(state->disk->queue) / 512;
this_sector = first_sector;
this_size = first_size;
@ -579,7 +580,7 @@ static struct {
int msdos_partition(struct parsed_partitions *state)
{
sector_t sector_size = bdev_logical_block_size(state->bdev) / 512;
sector_t sector_size;
Sector sect;
unsigned char *data;
struct msdos_partition *p;
@ -587,6 +588,7 @@ int msdos_partition(struct parsed_partitions *state)
int slot;
u32 disksig;
sector_size = queue_logical_block_size(state->disk->queue) / 512;
data = read_part_sector(state, 0, &sect);
if (!data)
return -1;

5
block/partitions/sgi.c
View File

@ -43,7 +43,6 @@ int sgi_partition(struct parsed_partitions *state)
Sector sect;
struct sgi_disklabel *label;
struct sgi_partition *p;
char b[BDEVNAME_SIZE];
label = read_part_sector(state, 0, &sect);
if (!label)
@ -52,7 +51,7 @@ int sgi_partition(struct parsed_partitions *state)
magic = label->magic_mushroom;
if(be32_to_cpu(magic) != SGI_LABEL_MAGIC) {
/*printk("Dev %s SGI disklabel: bad magic %08x\n",
bdevname(bdev, b), be32_to_cpu(magic));*/
state->disk->disk_name, be32_to_cpu(magic));*/
put_dev_sector(sect);
return 0;
}
@ -63,7 +62,7 @@ int sgi_partition(struct parsed_partitions *state)
}
if(csum) {
printk(KERN_WARNING "Dev %s SGI disklabel: csum bad, label corrupted\n",
bdevname(state->bdev, b));
state->disk->disk_name);
put_dev_sector(sect);
return 0;
}

5
block/partitions/sun.c
View File

@ -65,7 +65,6 @@ int sun_partition(struct parsed_partitions *state)
} * label;
struct sun_partition *p;
unsigned long spc;
char b[BDEVNAME_SIZE];
int use_vtoc;
int nparts;
@ -76,7 +75,7 @@ int sun_partition(struct parsed_partitions *state)
p = label->partitions;
if (be16_to_cpu(label->magic) != SUN_LABEL_MAGIC) {
/* printk(KERN_INFO "Dev %s Sun disklabel: bad magic %04x\n",
bdevname(bdev, b), be16_to_cpu(label->magic)); */
state->disk->disk_name, be16_to_cpu(label->magic)); */
put_dev_sector(sect);
return 0;
}
@ -86,7 +85,7 @@ int sun_partition(struct parsed_partitions *state)
csum ^= *ush--;
if (csum) {
printk("Dev %s Sun disklabel: Csum bad, label corrupted\n",
bdevname(state->bdev, b));
state->disk->disk_name);
put_dev_sector(sect);
return 0;
}

16
block/t10-pi.c
View File

@ -147,11 +147,10 @@ static void t10_pi_type1_prepare(struct request *rq)
break;
bip_for_each_vec(iv, bip, iter) {
void *p, *pmap;
unsigned int j;
void *p;
pmap = kmap_atomic(iv.bv_page);
p = pmap + iv.bv_offset;
p = bvec_kmap_local(&iv);
for (j = 0; j < iv.bv_len; j += tuple_sz) {
struct t10_pi_tuple *pi = p;
@ -161,8 +160,7 @@ static void t10_pi_type1_prepare(struct request *rq)
ref_tag++;
p += tuple_sz;
}
kunmap_atomic(pmap);
kunmap_local(p);
}
bip->bip_flags |= BIP_MAPPED_INTEGRITY;
@ -195,11 +193,10 @@ static void t10_pi_type1_complete(struct request *rq, unsigned int nr_bytes)
struct bvec_iter iter;
bip_for_each_vec(iv, bip, iter) {
void *p, *pmap;
unsigned int j;
void *p;
pmap = kmap_atomic(iv.bv_page);
p = pmap + iv.bv_offset;
p = bvec_kmap_local(&iv);
for (j = 0; j < iv.bv_len && intervals; j += tuple_sz) {
struct t10_pi_tuple *pi = p;
@ -210,8 +207,7 @@ static void t10_pi_type1_complete(struct request *rq, unsigned int nr_bytes)
intervals--;
p += tuple_sz;
}
kunmap_atomic(pmap);
kunmap_local(p);
}
}
}

2
drivers/Kconfig
View File

@ -51,8 +51,6 @@ source "drivers/net/Kconfig"
source "drivers/isdn/Kconfig"
source "drivers/lightnvm/Kconfig"
# input before char - char/joystick depends on it. As does USB.
source "drivers/input/Kconfig"

1
drivers/Makefile
View File

@ -70,7 +70,6 @@ obj-$(CONFIG_FB_I810) += video/fbdev/i810/
obj-$(CONFIG_FB_INTEL) += video/fbdev/intelfb/
obj-$(CONFIG_PARPORT) += parport/
obj-$(CONFIG_NVM) += lightnvm/
obj-y += base/ block/ misc/ mfd/ nfc/
obj-$(CONFIG_LIBNVDIMM) += nvdimm/
obj-$(CONFIG_DAX) += dax/

1
drivers/ata/libahci.c
View File

@ -125,6 +125,7 @@ EXPORT_SYMBOL_GPL(ahci_shost_attrs);
struct device_attribute *ahci_sdev_attrs[] = {
&dev_attr_sw_activity,
&dev_attr_unload_heads,
&dev_attr_ncq_prio_supported,
&dev_attr_ncq_prio_enable,
NULL
};

272
drivers/ata/libata-core.c
View File

@ -159,6 +159,12 @@ MODULE_DESCRIPTION("Library module for ATA devices");
MODULE_LICENSE("GPL");
MODULE_VERSION(DRV_VERSION);
static inline bool ata_dev_print_info(struct ata_device *dev)
{
struct ata_eh_context *ehc = &dev->link->eh_context;
return ehc->i.flags & ATA_EHI_PRINTINFO;
}
static bool ata_sstatus_online(u32 sstatus)
{
@ -706,11 +712,9 @@ int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
if (tf->flags & ATA_TFLAG_FUA)
tf->device |= 1 << 7;
if (dev->flags & ATA_DFLAG_NCQ_PRIO) {
if (class == IOPRIO_CLASS_RT)
tf->hob_nsect |= ATA_PRIO_HIGH <<
ATA_SHIFT_PRIO;
}
if (dev->flags & ATA_DFLAG_NCQ_PRIO_ENABLE &&
class == IOPRIO_CLASS_RT)
tf->hob_nsect |= ATA_PRIO_HIGH << ATA_SHIFT_PRIO;
} else if (dev->flags & ATA_DFLAG_LBA) {
tf->flags |= ATA_TFLAG_LBA;
@ -1266,8 +1270,7 @@ static int ata_set_max_sectors(struct ata_device *dev, u64 new_sectors)
*/
static int ata_hpa_resize(struct ata_device *dev)
{
struct ata_eh_context *ehc = &dev->link->eh_context;
int print_info = ehc->i.flags & ATA_EHI_PRINTINFO;
bool print_info = ata_dev_print_info(dev);
bool unlock_hpa = ata_ignore_hpa || dev->flags & ATA_DFLAG_UNLOCK_HPA;
u64 sectors = ata_id_n_sectors(dev->id);
u64 native_sectors;
@ -2023,13 +2026,15 @@ unsigned int ata_read_log_page(struct ata_device *dev, u8 log,
err_mask = ata_exec_internal(dev, &tf, NULL, DMA_FROM_DEVICE,
buf, sectors * ATA_SECT_SIZE, 0);
if (err_mask && dma) {
dev->horkage |= ATA_HORKAGE_NO_DMA_LOG;
ata_dev_warn(dev, "READ LOG DMA EXT failed, trying PIO\n");
goto retry;
if (err_mask) {
if (dma) {
dev->horkage |= ATA_HORKAGE_NO_DMA_LOG;
goto retry;
}
ata_dev_err(dev, "Read log page 0x%02x failed, Emask 0x%x\n",
(unsigned int)page, err_mask);
}
DPRINTK("EXIT, err_mask=%x\n", err_mask);
return err_mask;
}
@ -2058,12 +2063,8 @@ static bool ata_identify_page_supported(struct ata_device *dev, u8 page)
*/
err = ata_read_log_page(dev, ATA_LOG_IDENTIFY_DEVICE, 0, ap->sector_buf,
1);
if (err) {
ata_dev_info(dev,
"failed to get Device Identify Log Emask 0x%x\n",
err);
if (err)
return false;
}
for (i = 0; i < ap->sector_buf[8]; i++) {
if (ap->sector_buf[9 + i] == page)
@ -2127,11 +2128,7 @@ static void ata_dev_config_ncq_send_recv(struct ata_device *dev)
}
err_mask = ata_read_log_page(dev, ATA_LOG_NCQ_SEND_RECV,
0, ap->sector_buf, 1);
if (err_mask) {
ata_dev_dbg(dev,
"failed to get NCQ Send/Recv Log Emask 0x%x\n",
err_mask);
} else {
if (!err_mask) {
u8 *cmds = dev->ncq_send_recv_cmds;
dev->flags |= ATA_DFLAG_NCQ_SEND_RECV;
@ -2157,11 +2154,7 @@ static void ata_dev_config_ncq_non_data(struct ata_device *dev)
}
err_mask = ata_read_log_page(dev, ATA_LOG_NCQ_NON_DATA,
0, ap->sector_buf, 1);
if (err_mask) {
ata_dev_dbg(dev,
"failed to get NCQ Non-Data Log Emask 0x%x\n",
err_mask);
} else {
if (!err_mask) {
u8 *cmds = dev->ncq_non_data_cmds;
memcpy(cmds, ap->sector_buf, ATA_LOG_NCQ_NON_DATA_SIZE);
@ -2173,30 +2166,24 @@ static void ata_dev_config_ncq_prio(struct ata_device *dev)
struct ata_port *ap = dev->link->ap;
unsigned int err_mask;
if (!(dev->flags & ATA_DFLAG_NCQ_PRIO_ENABLE)) {
dev->flags &= ~ATA_DFLAG_NCQ_PRIO;
return;
}
err_mask = ata_read_log_page(dev,
ATA_LOG_IDENTIFY_DEVICE,
ATA_LOG_SATA_SETTINGS,
ap->sector_buf,
1);
if (err_mask) {
ata_dev_dbg(dev,
"failed to get Identify Device data, Emask 0x%x\n",
err_mask);
return;
}
if (err_mask)
goto not_supported;
if (ap->sector_buf[ATA_LOG_NCQ_PRIO_OFFSET] & BIT(3)) {
dev->flags |= ATA_DFLAG_NCQ_PRIO;
} else {
dev->flags &= ~ATA_DFLAG_NCQ_PRIO;
ata_dev_dbg(dev, "SATA page does not support priority\n");
}
if (!(ap->sector_buf[ATA_LOG_NCQ_PRIO_OFFSET] & BIT(3)))
goto not_supported;
dev->flags |= ATA_DFLAG_NCQ_PRIO;
return;
not_supported:
dev->flags &= ~ATA_DFLAG_NCQ_PRIO_ENABLE;
dev->flags &= ~ATA_DFLAG_NCQ_PRIO;
}
static int ata_dev_config_ncq(struct ata_device *dev,
@ -2346,11 +2333,8 @@ static void ata_dev_config_trusted(struct ata_device *dev)
err = ata_read_log_page(dev, ATA_LOG_IDENTIFY_DEVICE, ATA_LOG_SECURITY,
ap->sector_buf, 1);
if (err) {
ata_dev_dbg(dev,
"failed to read Security Log, Emask 0x%x\n", err);
if (err)
return;
}
trusted_cap = get_unaligned_le64(&ap->sector_buf[40]);
if (!(trusted_cap & (1ULL << 63))) {
@ -2363,6 +2347,106 @@ static void ata_dev_config_trusted(struct ata_device *dev)
dev->flags |= ATA_DFLAG_TRUSTED;
}
static int ata_dev_config_lba(struct ata_device *dev)
{
struct ata_port *ap = dev->link->ap;
const u16 *id = dev->id;
const char *lba_desc;
char ncq_desc[24];
int ret;
dev->flags |= ATA_DFLAG_LBA;
if (ata_id_has_lba48(id)) {
lba_desc = "LBA48";
dev->flags |= ATA_DFLAG_LBA48;
if (dev->n_sectors >= (1UL << 28) &&
ata_id_has_flush_ext(id))
dev->flags |= ATA_DFLAG_FLUSH_EXT;
} else {
lba_desc = "LBA";
}
/* config NCQ */
ret = ata_dev_config_ncq(dev, ncq_desc, sizeof(ncq_desc));
/* print device info to dmesg */
if (ata_msg_drv(ap) && ata_dev_print_info(dev))
ata_dev_info(dev,
"%llu sectors, multi %u: %s %s\n",
(unsigned long long)dev->n_sectors,
dev->multi_count, lba_desc, ncq_desc);
return ret;
}
static void ata_dev_config_chs(struct ata_device *dev)
{
struct ata_port *ap = dev->link->ap;
const u16 *id = dev->id;
if (ata_id_current_chs_valid(id)) {
/* Current CHS translation is valid. */
dev->cylinders = id[54];
dev->heads = id[55];
dev->sectors = id[56];
} else {
/* Default translation */
dev->cylinders = id[1];
dev->heads = id[3];
dev->sectors = id[6];
}
/* print device info to dmesg */
if (ata_msg_drv(ap) && ata_dev_print_info(dev))
ata_dev_info(dev,
"%llu sectors, multi %u, CHS %u/%u/%u\n",
(unsigned long long)dev->n_sectors,
dev->multi_count, dev->cylinders,
dev->heads, dev->sectors);
}
static void ata_dev_config_devslp(struct ata_device *dev)
{
u8 *sata_setting = dev->link->ap->sector_buf;
unsigned int err_mask;
int i, j;
/*
* Check device sleep capability. Get DevSlp timing variables
* from SATA Settings page of Identify Device Data Log.
*/
if (!ata_id_has_devslp(dev->id))
return;
err_mask = ata_read_log_page(dev,
ATA_LOG_IDENTIFY_DEVICE,
ATA_LOG_SATA_SETTINGS,
sata_setting, 1);
if (err_mask)
return;
dev->flags |= ATA_DFLAG_DEVSLP;
for (i = 0; i < ATA_LOG_DEVSLP_SIZE; i++) {
j = ATA_LOG_DEVSLP_OFFSET + i;
dev->devslp_timing[i] = sata_setting[j];
}
}
static void ata_dev_print_features(struct ata_device *dev)
{
if (!(dev->flags & ATA_DFLAG_FEATURES_MASK))
return;
ata_dev_info(dev,
"Features:%s%s%s%s%s\n",
dev->flags & ATA_DFLAG_TRUSTED ? " Trust" : "",
dev->flags & ATA_DFLAG_DA ? " Dev-Attention" : "",
dev->flags & ATA_DFLAG_DEVSLP ? " Dev-Sleep" : "",
dev->flags & ATA_DFLAG_NCQ_SEND_RECV ? " NCQ-sndrcv" : "",
dev->flags & ATA_DFLAG_NCQ_PRIO ? " NCQ-prio" : "");
}
/**
* ata_dev_configure - Configure the specified ATA/ATAPI device
* @dev: Target device to configure
@ -2379,8 +2463,7 @@ static void ata_dev_config_trusted(struct ata_device *dev)
int ata_dev_configure(struct ata_device *dev)
{
struct ata_port *ap = dev->link->ap;
struct ata_eh_context *ehc = &dev->link->eh_context;
int print_info = ehc->i.flags & ATA_EHI_PRINTINFO;
bool print_info = ata_dev_print_info(dev);
const u16 *id = dev->id;
unsigned long xfer_mask;
unsigned int err_mask;
@ -2507,91 +2590,28 @@ int ata_dev_configure(struct ata_device *dev)
dev->multi_count = cnt;
}
/* print device info to dmesg */
if (ata_msg_drv(ap) && print_info)
ata_dev_info(dev, "%s: %s, %s, max %s\n",
revbuf, modelbuf, fwrevbuf,
ata_mode_string(xfer_mask));
if (ata_id_has_lba(id)) {
const char *lba_desc;
char ncq_desc[24];
lba_desc = "LBA";
dev->flags |= ATA_DFLAG_LBA;
if (ata_id_has_lba48(id)) {
dev->flags |= ATA_DFLAG_LBA48;
lba_desc = "LBA48";
if (dev->n_sectors >= (1UL << 28) &&
ata_id_has_flush_ext(id))
dev->flags |= ATA_DFLAG_FLUSH_EXT;
}
/* config NCQ */
rc = ata_dev_config_ncq(dev, ncq_desc, sizeof(ncq_desc));
rc = ata_dev_config_lba(dev);
if (rc)
return rc;
/* print device info to dmesg */
if (ata_msg_drv(ap) && print_info) {
ata_dev_info(dev, "%s: %s, %s, max %s\n",
revbuf, modelbuf, fwrevbuf,
ata_mode_string(xfer_mask));
ata_dev_info(dev,
"%llu sectors, multi %u: %s %s\n",
(unsigned long long)dev->n_sectors,
dev->multi_count, lba_desc, ncq_desc);
}
} else {
/* CHS */
/* Default translation */
dev->cylinders = id[1];
dev->heads = id[3];
dev->sectors = id[6];
if (ata_id_current_chs_valid(id)) {
/* Current CHS translation is valid. */
dev->cylinders = id[54];
dev->heads = id[55];
dev->sectors = id[56];
}
/* print device info to dmesg */
if (ata_msg_drv(ap) && print_info) {
ata_dev_info(dev, "%s: %s, %s, max %s\n",
revbuf, modelbuf, fwrevbuf,
ata_mode_string(xfer_mask));
ata_dev_info(dev,
"%llu sectors, multi %u, CHS %u/%u/%u\n",
(unsigned long long)dev->n_sectors,
dev->multi_count, dev->cylinders,
dev->heads, dev->sectors);
}
ata_dev_config_chs(dev);
}
/* Check and mark DevSlp capability. Get DevSlp timing variables
* from SATA Settings page of Identify Device Data Log.
*/
if (ata_id_has_devslp(dev->id)) {
u8 *sata_setting = ap->sector_buf;
int i, j;
dev->flags |= ATA_DFLAG_DEVSLP;
err_mask = ata_read_log_page(dev,
ATA_LOG_IDENTIFY_DEVICE,
ATA_LOG_SATA_SETTINGS,
sata_setting,
1);
if (err_mask)
ata_dev_dbg(dev,
"failed to get Identify Device Data, Emask 0x%x\n",
err_mask);
else
for (i = 0; i < ATA_LOG_DEVSLP_SIZE; i++) {
j = ATA_LOG_DEVSLP_OFFSET + i;
dev->devslp_timing[i] = sata_setting[j];
}
}
ata_dev_config_devslp(dev);
ata_dev_config_sense_reporting(dev);
ata_dev_config_zac(dev);
ata_dev_config_trusted(dev);
dev->cdb_len = 32;
if (ata_msg_drv(ap) && print_info)
ata_dev_print_features(dev);
}
/* ATAPI-specific feature tests */
@ -5573,7 +5593,7 @@ int ata_host_start(struct ata_host *host)
have_stop = 1;
}
if (host->ops->host_stop)
if (host->ops && host->ops->host_stop)
have_stop = 1;
if (have_stop) {

62
drivers/ata/libata-sata.c
View File

@ -834,28 +834,46 @@ DEVICE_ATTR(link_power_management_policy, S_IRUGO | S_IWUSR,
ata_scsi_lpm_show, ata_scsi_lpm_store);
EXPORT_SYMBOL_GPL(dev_attr_link_power_management_policy);
static ssize_t ata_ncq_prio_supported_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct scsi_device *sdev = to_scsi_device(device);
struct ata_port *ap = ata_shost_to_port(sdev->host);
struct ata_device *dev;
bool ncq_prio_supported;
int rc = 0;
spin_lock_irq(ap->lock);
dev = ata_scsi_find_dev(ap, sdev);
if (!dev)
rc = -ENODEV;
else
ncq_prio_supported = dev->flags & ATA_DFLAG_NCQ_PRIO;
spin_unlock_irq(ap->lock);
return rc ? rc : sysfs_emit(buf, "%u\n", ncq_prio_supported);
}
DEVICE_ATTR(ncq_prio_supported, S_IRUGO, ata_ncq_prio_supported_show, NULL);
EXPORT_SYMBOL_GPL(dev_attr_ncq_prio_supported);
static ssize_t ata_ncq_prio_enable_show(struct device *device,
struct device_attribute *attr,
char *buf)
{
struct scsi_device *sdev = to_scsi_device(device);
struct ata_port *ap;
struct ata_port *ap = ata_shost_to_port(sdev->host);
struct ata_device *dev;
bool ncq_prio_enable;
int rc = 0;
ap = ata_shost_to_port(sdev->host);
spin_lock_irq(ap->lock);
dev = ata_scsi_find_dev(ap, sdev);
if (!dev) {
if (!dev)
rc = -ENODEV;
goto unlock;
}
ncq_prio_enable = dev->flags & ATA_DFLAG_NCQ_PRIO_ENABLE;
unlock:
else
ncq_prio_enable = dev->flags & ATA_DFLAG_NCQ_PRIO_ENABLE;
spin_unlock_irq(ap->lock);
return rc ? rc : snprintf(buf, 20, "%u\n", ncq_prio_enable);
@ -869,7 +887,7 @@ static ssize_t ata_ncq_prio_enable_store(struct device *device,
struct ata_port *ap;
struct ata_device *dev;
long int input;
int rc;
int rc = 0;
rc = kstrtol(buf, 10, &input);
if (rc)
@ -883,27 +901,20 @@ static ssize_t ata_ncq_prio_enable_store(struct device *device,
return -ENODEV;
spin_lock_irq(ap->lock);
if (!(dev->flags & ATA_DFLAG_NCQ_PRIO)) {
rc = -EINVAL;
goto unlock;
}
if (input)
dev->flags |= ATA_DFLAG_NCQ_PRIO_ENABLE;
else
dev->flags &= ~ATA_DFLAG_NCQ_PRIO_ENABLE;
dev->link->eh_info.action |= ATA_EH_REVALIDATE;
dev->link->eh_info.flags |= ATA_EHI_QUIET;
ata_port_schedule_eh(ap);
unlock:
spin_unlock_irq(ap->lock);
ata_port_wait_eh(ap);
if (input) {
spin_lock_irq(ap->lock);
if (!(dev->flags & ATA_DFLAG_NCQ_PRIO)) {
dev->flags &= ~ATA_DFLAG_NCQ_PRIO_ENABLE;
rc = -EIO;
}
spin_unlock_irq(ap->lock);
}
return rc ? rc : len;
}
@ -914,6 +925,7 @@ EXPORT_SYMBOL_GPL(dev_attr_ncq_prio_enable);
struct device_attribute *ata_ncq_sdev_attrs[] = {
&dev_attr_unload_heads,
&dev_attr_ncq_prio_enable,
&dev_attr_ncq_prio_supported,
NULL
};
EXPORT_SYMBOL_GPL(ata_ncq_sdev_attrs);

60
drivers/ata/libata-scsi.c
View File

@ -1765,53 +1765,6 @@ struct ata_scsi_args {
struct scsi_cmnd *cmd;
};
/**
* ata_scsi_rbuf_get - Map response buffer.
* @cmd: SCSI command containing buffer to be mapped.
* @flags: unsigned long variable to store irq enable status
* @copy_in: copy in from user buffer
*
* Prepare buffer for simulated SCSI commands.
*
* LOCKING:
* spin_lock_irqsave(ata_scsi_rbuf_lock) on success
*
* RETURNS:
* Pointer to response buffer.
*/
static void *ata_scsi_rbuf_get(struct scsi_cmnd *cmd, bool copy_in,
unsigned long *flags)
{
spin_lock_irqsave(&ata_scsi_rbuf_lock, *flags);
memset(ata_scsi_rbuf, 0, ATA_SCSI_RBUF_SIZE);
if (copy_in)
sg_copy_to_buffer(scsi_sglist(cmd), scsi_sg_count(cmd),
ata_scsi_rbuf, ATA_SCSI_RBUF_SIZE);
return ata_scsi_rbuf;
}
/**
* ata_scsi_rbuf_put - Unmap response buffer.
* @cmd: SCSI command containing buffer to be unmapped.
* @copy_out: copy out result
* @flags: @flags passed to ata_scsi_rbuf_get()
*
* Returns rbuf buffer. The result is copied to @cmd's buffer if
* @copy_back is true.
*
* LOCKING:
* Unlocks ata_scsi_rbuf_lock.
*/
static inline void ata_scsi_rbuf_put(struct scsi_cmnd *cmd, bool copy_out,
unsigned long *flags)
{
if (copy_out)
sg_copy_from_buffer(scsi_sglist(cmd), scsi_sg_count(cmd),
ata_scsi_rbuf, ATA_SCSI_RBUF_SIZE);
spin_unlock_irqrestore(&ata_scsi_rbuf_lock, *flags);
}
/**
* ata_scsi_rbuf_fill - wrapper for SCSI command simulators
* @args: device IDENTIFY data / SCSI command of interest.
@ -1830,14 +1783,19 @@ static inline void ata_scsi_rbuf_put(struct scsi_cmnd *cmd, bool copy_out,
static void ata_scsi_rbuf_fill(struct ata_scsi_args *args,
unsigned int (*actor)(struct ata_scsi_args *args, u8 *rbuf))
{
u8 *rbuf;
unsigned int rc;
struct scsi_cmnd *cmd = args->cmd;
unsigned long flags;
rbuf = ata_scsi_rbuf_get(cmd, false, &flags);
rc = actor(args, rbuf);
ata_scsi_rbuf_put(cmd, rc == 0, &flags);
spin_lock_irqsave(&ata_scsi_rbuf_lock, flags);
memset(ata_scsi_rbuf, 0, ATA_SCSI_RBUF_SIZE);
rc = actor(args, ata_scsi_rbuf);
if (rc == 0)
sg_copy_from_buffer(scsi_sglist(cmd), scsi_sg_count(cmd),
ata_scsi_rbuf, ATA_SCSI_RBUF_SIZE);
spin_unlock_irqrestore(&ata_scsi_rbuf_lock, flags);
if (rc == 0)
cmd->result = SAM_STAT_GOOD;

12
drivers/ata/sata_dwc_460ex.c
View File

@ -1259,24 +1259,20 @@ static int sata_dwc_probe(struct platform_device *ofdev)
irq = irq_of_parse_and_map(np, 0);
if (irq == NO_IRQ) {
dev_err(&ofdev->dev, "no SATA DMA irq\n");
err = -ENODEV;
goto error_out;
return -ENODEV;
}
#ifdef CONFIG_SATA_DWC_OLD_DMA
if (!of_find_property(np, "dmas", NULL)) {
err = sata_dwc_dma_init_old(ofdev, hsdev);
if (err)
goto error_out;
return err;
}
#endif
hsdev->phy = devm_phy_optional_get(hsdev->dev, "sata-phy");
if (IS_ERR(hsdev->phy)) {
err = PTR_ERR(hsdev->phy);
hsdev->phy = NULL;
goto error_out;
}
if (IS_ERR(hsdev->phy))
return PTR_ERR(hsdev->phy);
err = phy_init(hsdev->phy);
if (err)

3
drivers/block/brd.c
View File

@ -27,9 +27,6 @@
#include <linux/uaccess.h>
#define PAGE_SECTORS_SHIFT (PAGE_SHIFT - SECTOR_SHIFT)
#define PAGE_SECTORS (1 << PAGE_SECTORS_SHIFT)
/*
* Each block ramdisk device has a radix_tree brd_pages of pages that stores
* the pages containing the block device's contents. A brd page's ->index is

2
drivers/block/drbd/drbd_nl.c
View File

@ -1364,7 +1364,7 @@ static void drbd_setup_queue_param(struct drbd_device *device, struct drbd_backi
if (b) {
blk_stack_limits(&q->limits, &b->limits, 0);
blk_queue_update_readahead(q);
disk_update_readahead(device->vdisk);
}
fixup_discard_if_not_supported(q);
fixup_write_zeroes(device, q);

5
drivers/block/drbd/drbd_req.c
View File

@ -905,13 +905,12 @@ static bool drbd_may_do_local_read(struct drbd_device *device, sector_t sector,
static bool remote_due_to_read_balancing(struct drbd_device *device, sector_t sector,
enum drbd_read_balancing rbm)
{
struct backing_dev_info *bdi;
int stripe_shift;
switch (rbm) {
case RB_CONGESTED_REMOTE:
bdi = device->ldev->backing_bdev->bd_disk->queue->backing_dev_info;
return bdi_read_congested(bdi);
return bdi_read_congested(
device->ldev->backing_bdev->bd_disk->bdi);
case RB_LEAST_PENDING:
return atomic_read(&device->local_cnt) >
atomic_read(&device->ap_pending_cnt) + atomic_read(&device->rs_pending_cnt);

Some files were not shown because too many files have changed in this diff Show More