android/android_kernel_asus_sm8350
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

BACKPORT: FROMLIST: mm: protect free_pgtables with mmap_lock write lock in exit_mmap

Browse Source 
oom-reaper and process_mrelease system call should protect against
races with exit_mmap which can destroy page tables while they
walk the VMA tree. oom-reaper protects from that race by setting
MMF_OOM_VICTIM and by relying on exit_mmap to set MMF_OOM_SKIP
before taking and releasing mmap_write_lock. process_mrelease has
to elevate mm->mm_users to prevent such race. Both oom-reaper and
process_mrelease hold mmap_read_lock when walking the VMA tree.
The locking rules and mechanisms could be simpler if exit_mmap takes
mmap_write_lock while executing destructive operations such as
free_pgtables.
Change exit_mmap to hold the mmap_write_lock when calling
free_pgtables. Operations like unmap_vmas() and unlock_range() are not
destructive and could run under mmap_read_lock but for simplicity we
take one mmap_write_lock during almost the entire operation. Note
also that because oom-reaper checks VM_LOCKED flag, unlock_range()
should not be allowed to race with it.
In most cases this lock should be uncontended. Previously, Kirill
reported ~4% regression caused by a similar change [1]. We reran the
same test and although the individual results are quite noisy, the
percentiles show lower regression with 1.6% being the worst case [2].
The change allows oom-reaper and process_mrelease to execute safely
under mmap_read_lock without worries that exit_mmap might destroy page
tables from under them.

[1] https://lore.kernel.org/all/20170725141723.ivukwhddk2voyhuc@node.shutemov.name/
[2] https://lore.kernel.org/all/CAJuCfpGC9-c9P40x7oy=jy5SphMcd0o0G_6U1-+JAziGKG6dGA@mail.gmail.com/

Signed-off-by: Suren Baghdasaryan <surenb@google.com>

Link: https://lore.kernel.org/all/20211124235906.14437-1-surenb@google.com/

Bug: 130172058
Bug: 189803002
Change-Id: Ic87272d09a0b68a1b0e968e8f1a1510fd6fc776a
Git-commit: 28358ebf2adb31117893813992fefcfd359a6a16
Git-repo: https://android.googlesource.com/kernel/common/
[quic_gkohli@quicinc.com: Resolved cherry-pick conflict in mm/mmap.c due
 to mmap lock was implemented differently in older kernel, and
 Although process_mrelease is not applicable in older kernel, but this
 patch is required to take exclusive lock in exit_mmap path so that
 SPF knows an isolated vma was freed from this path]
Signed-off-by: Gaurav Kohli <quic_gkohli@quicinc.com>
Signed-off-by: Srinivasarao Pathipati <quic_c_spathi@quicinc.com>
This commit is contained in:
Suren Baghdasaryan 2021-11-24 15:59:05 -08:00 committed by Srinivasarao Pathipati
parent 5305af0230
commit bb3bc96f35
1 changed files with 7 additions and 7 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
mm/mmap.c
View File

@ -3254,10 +3254,9 @@ void exit_mmap(struct mm_struct *mm)
(void)__oom_reap_task_mm(mm);
set_bit(MMF_OOM_SKIP, &mm->flags);
down_write(&mm->mmap_sem);
up_write(&mm->mmap_sem);
}
down_write(&mm->mmap_sem);
if (mm->locked_vm) {
vma = mm->mmap;
while (vma) {
@ -3270,8 +3269,11 @@ void exit_mmap(struct mm_struct *mm)
arch_exit_mmap(mm);
vma = mm->mmap;
if (!vma) /* Can happen if dup_mmap() received an OOM */
if (!vma) {
/* Can happen if dup_mmap() received an OOM */
up_write(&mm->mmap_sem);;
return;
}
lru_add_drain();
flush_cache_mm(mm);
@ -3282,16 +3284,14 @@ void exit_mmap(struct mm_struct *mm)
free_pgtables(&tlb, vma, FIRST_USER_ADDRESS, USER_PGTABLES_CEILING);
tlb_finish_mmu(&tlb, 0, -1);
/*
* Walk the list again, actually closing and freeing it,
* with preemption enabled, without holding any MM locks.
*/
/* Walk the list again, actually closing and freeing it. */
while (vma) {
if (vma->vm_flags & VM_ACCOUNT)
nr_accounted += vma_pages(vma);
vma = remove_vma(vma);
cond_resched();
}
up_write(&mm->mmap_sem);
vm_unacct_memory(nr_accounted);
}