android/android_kernel_xiaomi_sm8450
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
986,469 Commits 2 Branches 0 Tags 2.3 GiB
C 97.8%
Assembly 1.2%
Shell 0.3%
Makefile 0.3%
Python 0.2%
Other 0.1%
3d9ce6d28b
Go to file
SeongJae Park 3d9ce6d28b UPSTREAM: mm/damon/core: account age of target regions 
Patch series "Implement Data Access Monitoring-based Memory Operation Schemes".

Introduction
============

DAMON[1] can be used as a primitive for data access aware memory
management optimizations.  For that, users who want such optimizations
should run DAMON, read the monitoring results, analyze it, plan a new
memory management scheme, and apply the new scheme by themselves.  Such
efforts will be inevitable for some complicated optimizations.

However, in many other cases, the users would simply want the system to
apply a memory management action to a memory region of a specific size
having a specific access frequency for a specific time.  For example,
"page out a memory region larger than 100 MiB keeping only rare accesses
more than 2 minutes", or "Do not use THP for a memory region larger than
2 MiB rarely accessed for more than 1 seconds".

To make the works easier and non-redundant, this patchset implements a
new feature of DAMON, which is called Data Access Monitoring-based
Operation Schemes (DAMOS).  Using the feature, users can describe the
normal schemes in a simple way and ask DAMON to execute those on its
own.

[1] https://damonitor.github.io

Evaluations
===========

DAMOS is accurate and useful for memory management optimizations.  An
experimental DAMON-based operation scheme for THP, 'ethp', removes
76.15% of THP memory overheads while preserving 51.25% of THP speedup.
Another experimental DAMON-based 'proactive reclamation' implementation,
'prcl', reduces 93.38% of residential sets and 23.63% of system memory
footprint while incurring only 1.22% runtime overhead in the best case
(parsec3/freqmine).

NOTE that the experimental THP optimization and proactive reclamation
are not for production but only for proof of concepts.

Please refer to the showcase web site's evaluation document[1] for
detailed evaluation setup and results.

[1] https://damonitor.github.io/doc/html/v34/vm/damon/eval.html

Long-term Support Trees
-----------------------

For people who want to test DAMON but using LTS kernels, there are
another couple of trees based on two latest LTS kernels respectively and
containing the 'damon/master' backports.

- For v5.4.y: https://git.kernel.org/sj/h/damon/for-v5.4.y
- For v5.10.y: https://git.kernel.org/sj/h/damon/for-v5.10.y

Sequence Of Patches
===================

The 1st patch accounts age of each region.  The 2nd patch implements the
core of the DAMON-based operation schemes feature.  The 3rd patch makes
the default monitoring primitives for virtual address spaces to support
the schemes.  From this point, the kernel space users can use DAMOS.
The 4th patch exports the feature to the user space via the debugfs
interface.  The 5th patch implements schemes statistics feature for
easier tuning of the schemes and runtime access pattern analysis, and
the 6th patch adds selftests for these changes.  Finally, the 7th patch
documents this new feature.

This patch (of 7):

DAMON can be used for data access pattern aware memory management
optimizations.  For that, users should run DAMON, read the monitoring
results, analyze it, plan a new memory management scheme, and apply the
new scheme by themselves.  It would not be too hard, but still require
some level of effort.  For complicated cases, this effort is inevitable.

That said, in many cases, users would simply want to apply an actions to
a memory region of a specific size having a specific access frequency
for a specific time.  For example, "page out a memory region larger than
100 MiB but having a low access frequency more than 10 minutes", or "Use
THP for a memory region larger than 2 MiB having a high access frequency
for more than 2 seconds".

For such optimizations, users will need to first account the age of each
region themselves.  To reduce such efforts, this implements a simple age
account of each region in DAMON.  For each aggregation step, DAMON
compares the access frequency with that from last aggregation and reset
the age of the region if the change is significant.  Else, the age is
incremented.  Also, in case of the merge of regions, the region
size-weighted average of the ages is set as the age of merged new
region.

Link: https://lkml.kernel.org/r/20211001125604.29660-1-sj@kernel.org
Link: https://lkml.kernel.org/r/20211001125604.29660-2-sj@kernel.org
Signed-off-by: SeongJae Park <sj@kernel.org>
Cc: Jonathan Cameron <Jonathan.Cameron@huawei.com>
Cc: Amit Shah <amit@kernel.org>
Cc: Benjamin Herrenschmidt <benh@kernel.crashing.org>
Cc: Jonathan Corbet <corbet@lwn.net>
Cc: David Hildenbrand <david@redhat.com>
Cc: David Woodhouse <dwmw@amazon.com>
Cc: Marco Elver <elver@google.com>
Cc: Leonard Foerster <foersleo@amazon.de>
Cc: Greg Thelen <gthelen@google.com>
Cc: Markus Boehme <markubo@amazon.de>
Cc: David Rienjes <rientjes@google.com>
Cc: Shakeel Butt <shakeelb@google.com>
Cc: Shuah Khan <shuah@kernel.org>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

(cherry picked from commit fda504fade7f124858d7022341dc46ff35b45274)

Bug: 228223814
Signed-off-by: Hailong Tu <tuhailong@oppo.com>
Change-Id: Ia5ddb3b5ce9c0d14e098a0af55dabf4b6a609aaa
2022-04-28 23:09:15 +08:00
android ANDROID: ABI: Update allowed list for galaxy 2022-04-27 12:54:35 +02:00
arch UPSTREAM: kfence, x86: fix preemptible warning on KPTI-enabled systems 2022-04-27 11:12:09 -07:00
block Merge tag 'android12-5.10.101_r00' into android12-5.10 2022-03-21 14:29:02 +01:00
certs certs: Trigger creation of RSA module signing key if it's not an RSA key 2021-09-15 09:50:29 +02:00
crypto This is the 5.10.100 stable release 2022-02-11 09:19:01 +01:00
Documentation Merge tag 'android12-5.10.101_r00' into android12-5.10 2022-03-21 14:29:02 +01:00
drivers BACKPORT: dm: fix NULL pointer issue when free bio 2022-04-27 22:40:43 +00:00
fs BACKPORT: fuse: fix pipe buffer lifetime for direct_io 2022-04-28 11:33:22 +00:00
include UPSTREAM: mm/damon/core: account age of target regions 2022-04-28 23:09:15 +08:00
init ANDROID: Split ANDROID_STRUCT_PADDING into separate configs 2022-04-07 05:44:50 +00:00
ipc shm: extend forced shm destroy to support objects from several IPC nses 2021-12-01 09:19:10 +01:00
kernel UPSTREAM: locking/lockdep: Avoid potential access of invalid memory in lock_class 2022-04-01 17:14:44 +00:00
lib Merge tag 'android12-5.10.101_r00' into android12-5.10 2022-03-21 14:29:02 +01:00
LICENSES LICENSES/deprecated: add Zlib license text 2020-09-16 14:33:49 +02:00
mm UPSTREAM: mm/damon/core: account age of target regions 2022-04-28 23:09:15 +08:00
net BACKPORT: net/packet: fix slab-out-of-bounds access in packet_recvmsg() 2022-04-28 13:02:55 +00:00
samples Merge branch 'android12-5.10' into android12-5.10-lts 2022-01-17 18:47:02 +01:00
scripts This is the 5.10.101 stable release 2022-02-16 15:16:06 +01:00
security This is the 5.10.101 stable release 2022-02-16 15:16:06 +01:00
sound This is the 5.10.99 stable release 2022-02-09 12:14:04 +01:00
tools UPSTREAM: mm/damon: add user space selftests 2022-04-28 23:09:14 +08:00
usr usr/include/Makefile: add linux/nfc.h to the compile-test coverage 2022-02-01 17:25:48 +01:00
virt KVM: eventfd: Fix false positive RCU usage warning 2022-02-16 12:54:20 +01:00
.clang-format RDMA 5.10 pull request 2020-10-17 11:18:18 -07:00
.cocciconfig
.get_maintainer.ignore Opt out of scripts/get_maintainer.pl 2019-05-16 10:53:40 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore Merge 5.10.38 into android12-5.10 2021-05-20 15:35:25 +02:00
.mailmap mailmap: add two more addresses of Uwe Kleine-König 2020-12-06 10:19:07 -08:00
build.config.aarch64 ANDROID: arm64: enable compat vdso 2021-05-13 09:14:25 -07:00
build.config.allmodconfig ANDROID: Different fix for KABI breakage in 5.10.71 in struct sock 2021-10-12 14:48:00 +02:00
build.config.allmodconfig.aarch64 ANDROID: drop KERNEL_DIR setting in build.config.common 2020-08-31 15:20:37 +00:00
build.config.allmodconfig.arm ANDROID: drop KERNEL_DIR setting in build.config.common 2020-08-31 15:20:37 +00:00
build.config.allmodconfig.x86_64 ANDROID: drop KERNEL_DIR setting in build.config.common 2020-08-31 15:20:37 +00:00
build.config.amlogic ANDROID: GKI: amlogic: add DTB overlays 2021-02-12 10:08:04 +01:00
build.config.arm ANDROID: enable LLVM_IAS=1 for clang's integrated assembler for arm 2021-03-24 09:21:58 -07:00
build.config.common ANDROID: GKI: 7/14/2021 KMI update 2021-07-14 21:10:40 -07:00
build.config.db845c ANDROID: db845c: Use FRAGMENT_CONFIG 2021-02-10 06:08:17 +00:00
build.config.gki ANDROID: gki: Removed cf modules from gki_defconfig 2020-02-01 00:27:50 +00:00
build.config.gki_kasan ANDROID: build.config: Disable LTO for KASAN and Kprobes builds 2021-03-08 00:31:49 +00:00
build.config.gki_kasan.aarch64 ANDROID: drop KERNEL_DIR setting in build.config.common 2020-08-31 15:20:37 +00:00
build.config.gki_kasan.x86_64 ANDROID: drop KERNEL_DIR setting in build.config.common 2020-08-31 15:20:37 +00:00
build.config.gki_kprobes ANDROID: build.config: Disable LTO for KASAN and Kprobes builds 2021-03-08 00:31:49 +00:00
build.config.gki_kprobes.aarch64 ANDROID: Adding kprobes build configs for Cuttlefish 2021-03-03 02:03:02 +00:00
build.config.gki_kprobes.x86_64 ANDROID: Adding kprobes build configs for Cuttlefish 2021-03-03 02:03:02 +00:00
build.config.gki-debug.aarch64 ANDROID: drop KERNEL_DIR setting in build.config.common 2020-08-31 15:20:37 +00:00
build.config.gki-debug.x86_64 ANDROID: drop KERNEL_DIR setting in build.config.common 2020-08-31 15:20:37 +00:00
build.config.gki.aarch64 ANDROID: Add initial ASUS symbol list 2022-01-12 20:07:30 +00:00
build.config.gki.aarch64.fips140 ANDROID: ABI: add new symbols required by fips140.ko 2021-10-29 14:32:49 -07:00
build.config.gki.aarch64.fips140_eval_testing ANDROID: fips140: support "evaluation testing" builds via build.sh 2021-11-30 17:55:05 +00:00
build.config.gki.x86_64 ANDROID: drop KERNEL_DIR setting in build.config.common 2020-08-31 15:20:37 +00:00
build.config.hikey960 ANDROID: Add hikey960 build infrastructure file 2021-02-18 23:59:34 +00:00
build.config.khwasan ANDROID: Add a build config fragment for KHWASan. 2021-04-06 10:01:56 +00:00
build.config.rockchip ANDROID: GKI: rockchip: Add build script 2021-10-25 08:57:33 +08:00
build.config.x86_64 ANDROID: build.config: Add vmlinux.symvers and modules.builtin to DIST_DIR 2021-04-15 10:24:17 +01:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: Move Jason Cooper to CREDITS 2020-11-30 10:20:34 +01:00
Kbuild kbuild: rename hostprogs-y/always to hostprogs/always-y 2020-02-04 01:53:07 +09:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS This is the 5.10.85 stable release 2021-12-14 15:11:46 +01:00
Makefile This is the 5.10.101 stable release 2022-02-16 15:16:06 +01:00
OWNERS ANDROID: Use authoritative Dr. No list from android-mainline 2021-07-22 15:25:25 +00:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00
README.md ANDROID: README: Add requirement for "Bug:" tag for FROMLIST 2020-05-01 15:36:12 -07:00

README.md

How do I submit patches to Android Common Kernels

  1. BEST: Make all of your changes to upstream Linux. If appropriate, backport to the stable releases. These patches will be merged automatically in the corresponding common kernels. If the patch is already in upstream Linux, post a backport of the patch that conforms to the patch requirements below.

    • Do not send patches upstream that contain only symbol exports. To be considered for upstream Linux, additions of EXPORT_SYMBOL_GPL() require an in-tree modular driver that uses the symbol -- so include the new driver or changes to an existing driver in the same patchset as the export.
    • When sending patches upstream, the commit message must contain a clear case for why the patch is needed and beneficial to the community. Enabling out-of-tree drivers or functionality is not not a persuasive case.

  2. LESS GOOD: Develop your patches out-of-tree (from an upstream Linux point-of-view). Unless these are fixing an Android-specific bug, these are very unlikely to be accepted unless they have been coordinated with kernel-team@android.com. If you want to proceed, post a patch that conforms to the patch requirements below.

Common Kernel patch requirements

  • All patches must conform to the Linux kernel coding standards and pass script/checkpatch.pl
  • Patches shall not break gki_defconfig or allmodconfig builds for arm, arm64, x86, x86_64 architectures (see https://source.android.com/setup/build/building-kernels)
  • If the patch is not merged from an upstream branch, the subject must be tagged with the type of patch: UPSTREAM:, BACKPORT:, FROMGIT:, FROMLIST:, or ANDROID:.
  • All patches must have a Change-Id: tag (see https://gerrit-review.googlesource.com/Documentation/user-changeid.html)
  • If an Android bug has been assigned, there must be a Bug: tag.
  • All patches must have a Signed-off-by: tag by the author and the submitter

Additional requirements are listed below based on patch type

Requirements for backports from mainline Linux: UPSTREAM:, BACKPORT:

  • If the patch is a cherry-pick from Linux mainline with no changes at all
    • tag the patch subject with UPSTREAM:.
    • add upstream commit information with a (cherry picked from commit ...) line
    • Example:
      • if the upstream commit message is
        important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>
  • then Joe Smith would upload the patch for the common kernel as
        UPSTREAM: important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>

        Bug: 135791357
        Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
        (cherry picked from commit c31e73121f4c1ec41143423ac6ce3ce6dafdcec1)
        Signed-off-by: Joe Smith <joe.smith@foo.org>
  • If the patch requires any changes from the upstream version, tag the patch with BACKPORT: instead of UPSTREAM:.
    • use the same tags as UPSTREAM:
    • add comments about the changes under the (cherry picked from commit ...) line
    • Example:
        BACKPORT: important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>

        Bug: 135791357
        Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
        (cherry picked from commit c31e73121f4c1ec41143423ac6ce3ce6dafdcec1)
        [joe: Resolved minor conflict in drivers/foo/bar.c ]
        Signed-off-by: Joe Smith <joe.smith@foo.org>

Requirements for other backports: FROMGIT:, FROMLIST:,

  • If the patch has been merged into an upstream maintainer tree, but has not yet been merged into Linux mainline
    • tag the patch subject with FROMGIT:
    • add info on where the patch came from as (cherry picked from commit <sha1> <repo> <branch>). This must be a stable maintainer branch (not rebased, so don't use linux-next for example).
    • if changes were required, use BACKPORT: FROMGIT:
    • Example:
      • if the commit message in the maintainer tree is
        important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>
  • then Joe Smith would upload the patch for the common kernel as
        FROMGIT: important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>

        Bug: 135791357
        (cherry picked from commit 878a2fd9de10b03d11d2f622250285c7e63deace
         https://git.kernel.org/pub/scm/linux/kernel/git/foo/bar.git test-branch)
        Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
        Signed-off-by: Joe Smith <joe.smith@foo.org>
  • If the patch has been submitted to LKML, but not accepted into any maintainer tree
    • tag the patch subject with FROMLIST:
    • add a Link: tag with a link to the submittal on lore.kernel.org
    • add a Bug: tag with the Android bug (required for patches not accepted into a maintainer tree)
    • if changes were required, use BACKPORT: FROMLIST:
    • Example:
        FROMLIST: important patch from upstream

        This is the detailed description of the important patch

        Signed-off-by: Fred Jones <fred.jones@foo.org>

        Bug: 135791357
        Link: https://lore.kernel.org/lkml/20190619171517.GA17557@someone.com/
        Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
        Signed-off-by: Joe Smith <joe.smith@foo.org>

Requirements for Android-specific patches: ANDROID:

  • If the patch is fixing a bug to Android-specific code
    • tag the patch subject with ANDROID:
    • add a Fixes: tag that cites the patch with the bug
    • Example:
        ANDROID: fix android-specific bug in foobar.c

        This is the detailed description of the important fix

        Fixes: 1234abcd2468 ("foobar: add cool feature")
        Change-Id: I4caaaa566ea080fa148c5e768bb1a0b6f7201c01
        Signed-off-by: Joe Smith <joe.smith@foo.org>
  • If the patch is a new feature
    • tag the patch subject with ANDROID:
    • add a Bug: tag with the Android bug (required for android-specific features)