From fca353bdc06628dcabfbaec9772dce318cff2250 Mon Sep 17 00:00:00 2001 From: Minchan Kim Date: Fri, 8 Apr 2022 14:29:47 -0700 Subject: [PATCH] ANDROID: mm: freeing MIGRATE_ISOLATE page instantly Since Android has pcp list for MIGRATE_CMA[1], it could cause CMA allocation latency due to not freeing the MIGRATE_ISOLATE page immediately. Originally, MIGRATE_ISOLATED page is supposed to go buddy list with skipping pcp list. Otherwise, the page could be reallocated from pcp list or staying on the pcp list until the pcp is drained so that CMA keeps retrying since it couldn't find the freed page from buddy list. That worked before since the CMA pfnblocks changed only from MIGRATE_CMA to MIGRATE_ISOLATE and free function logic in page allocator has checked MIGRATE_ISOLATEness on every CMA pages using below. free_unref_page_commit if (migratetype >= MIGRATE_PCPTYPES) if(is_migrate_isolate(migratetype)) free_one_page(page); It worked since enum MIGRATE_CMA was bigger than enum MIGRATE_PCPTYPES but since [1], the enum MIGRATE_CMA is less than MIGRATE_PCPTYPES so the logic above doesn't work any more. It could cause following race CPU 0 CPU 1 free_unref_page migratetype = get_pfnblock_migratetype() set_pcppage_migratetype(MIGRATE_CMA) cma_alloc alloc_contig_range set_migrate_isolate(MIGRATE_ISOLATE) add the page into pcp list the page could be reallocated This patch couldn't fix the race completely due to missing zone->lock in order-0 page free(for performance reason). However, it's not a new problem so we need to deal with the issue separately. [1] ANDROID: mm: add cma pcp list Bug: 218731671 Signed-off-by: Minchan Kim Change-Id: Ibea20085ce5bfb4b74b83b041f9bda9a380120f9 (cherry picked from commit d9e4b67784866047e8cfb5598cdf1ebc0c71f3d9) Signed-off-by: Richard Chang --- mm/page_alloc.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/mm/page_alloc.c b/mm/page_alloc.c index 501d69ac8b95..9a4fa513cc35 100644 --- a/mm/page_alloc.c +++ b/mm/page_alloc.c @@ -3590,12 +3590,13 @@ void free_unref_page(struct page *page, unsigned int order) * excessively into the page allocator */ migratetype = get_pcppage_migratetype(page); - if (unlikely(migratetype >= MIGRATE_PCPTYPES)) { + if (unlikely(migratetype > MIGRATE_RECLAIMABLE)) { if (unlikely(is_migrate_isolate(migratetype))) { free_one_page(page_zone(page), page, pfn, order, migratetype, FPI_NONE); return; } - migratetype = MIGRATE_MOVABLE; + if (migratetype == MIGRATE_HIGHATOMIC) + migratetype = MIGRATE_MOVABLE; } zone = page_zone(page);