vmscan: Do not writeback pages in direct reclaim
in [Kernel]
|
Prev: [tip:x86/mce] x86, mce: Fix MSR_IA32_MCI_CTL2 CMCI threshold setup
Next: [PATCH] drivers/net/e1000/e1000_main.c: Fix message logging defect
From: Mel Gorman on 5 Jul 2010 10:00 On Fri, Jul 02, 2010 at 12:51:55PM -0700, Andrew Morton wrote: > On Tue, 29 Jun 2010 12:34:46 +0100 > Mel Gorman <mel(a)csn.ul.ie> wrote: > > > When memory is under enough pressure, a process may enter direct > > reclaim to free pages in the same manner kswapd does. If a dirty page is > > encountered during the scan, this page is written to backing storage using > > mapping->writepage. This can result in very deep call stacks, particularly > > if the target storage or filesystem are complex. It has already been observed > > on XFS that the stack overflows but the problem is not XFS-specific. > > > > This patch prevents direct reclaim writing back pages by not setting > > may_writepage in scan_control. Instead, dirty pages are placed back on the > > LRU lists for either background writing by the BDI threads or kswapd. If > > in direct lumpy reclaim and dirty pages are encountered, the process will > > stall for the background flusher before trying to reclaim the pages again. > > > > Memory control groups do not have a kswapd-like thread nor do pages get > > direct reclaimed from the page allocator. Instead, memory control group > > pages are reclaimed when the quota is being exceeded or the group is being > > shrunk. As it is not expected that the entry points into page reclaim are > > deep call chains memcg is still allowed to writeback dirty pages. > > I already had "[PATCH 01/14] vmscan: Fix mapping use after free" and > I'll send that in for 2.6.35. > Perfect, thanks. > I grabbed [02/14] up to [11/14]. Including "[PATCH 06/14] vmscan: kill > prev_priority completely", grumpyouallsuck. > > I wimped out at this, "Do not writeback pages in direct reclaim". It > really is a profound change and needs a bit more thought, discussion > and if possible testing which is designed to explore possible pathologies. > Ok, that's reasonable as I'm still working on that patch. For example, the patch disabled anonymous page writeback which is unnecessary as the stack usage for anon writeback is less than file writeback. Second, using systemtap, I was able to see that file-backed dirty pages have a tendency to be near the end of the LRU even though they are a small percentage of the overall pages in the LRU. I'm hoping to figure out why this is as it would make avoiding writeback a lot less controversial. -- Mel Gorman Part-time Phd Student Linux Technology Center University of Limerick IBM Dublin Software Lab -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: KOSAKI Motohiro on 5 Jul 2010 20:40 Hello, > Ok, that's reasonable as I'm still working on that patch. For example, the > patch disabled anonymous page writeback which is unnecessary as the stack > usage for anon writeback is less than file writeback. How do we examine swap-on-file? > Second, using systemtap, > I was able to see that file-backed dirty pages have a tendency to be near the > end of the LRU even though they are a small percentage of the overall pages > in the LRU. I'm hoping to figure out why this is as it would make avoiding > writeback a lot less controversial. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Minchan Kim on 6 Jul 2010 01:50 On Tue, Jul 6, 2010 at 9:36 AM, KOSAKI Motohiro <kosaki.motohiro(a)jp.fujitsu.com> wrote: > Hello, > >> Ok, that's reasonable as I'm still working on that patch. For example, the >> patch disabled anonymous page writeback which is unnecessary as the stack >> usage for anon writeback is less than file writeback. > > How do we examine swap-on-file? bool is_swap_on_file(struct page *page) { struct swap_info_struct *p; swp_entry_entry entry; entry.val = page_private(page); p = swap_info_get(entry); return !(p->flags & SWP_BLKDEV) } -- Kind regards, Minchan Kim -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: KOSAKI Motohiro on 6 Jul 2010 02:10 > On Tue, Jul 6, 2010 at 9:36 AM, KOSAKI Motohiro > <kosaki.motohiro(a)jp.fujitsu.com> wrote: > > Hello, > > > >> Ok, that's reasonable as I'm still working on that patch. For example, the > >> patch disabled anonymous page writeback which is unnecessary as the stack > >> usage for anon writeback is less than file writeback. > > > > How do we examine swap-on-file? > > bool is_swap_on_file(struct page *page) > { > struct swap_info_struct *p; > swp_entry_entry entry; > entry.val = page_private(page); > p = swap_info_get(entry); > return !(p->flags & SWP_BLKDEV) > } Well, do you suggested we traverse all pages in lru _before_ starting vmscan? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Minchan Kim on 6 Jul 2010 02:40
On Tue, Jul 6, 2010 at 3:02 PM, KOSAKI Motohiro <kosaki.motohiro(a)jp.fujitsu.com> wrote: >> On Tue, Jul 6, 2010 at 9:36 AM, KOSAKI Motohiro >> <kosaki.motohiro(a)jp.fujitsu.com> wrote: >> > Hello, >> > >> >> Ok, that's reasonable as I'm still working on that patch. For example, the >> >> patch disabled anonymous page writeback which is unnecessary as the stack >> >> usage for anon writeback is less than file writeback. >> > >> > How do we examine swap-on-file? >> >> bool is_swap_on_file(struct page *page) >> { >> struct swap_info_struct *p; >> swp_entry_entry entry; >> entry.val = page_private(page); >> p = swap_info_get(entry); >> return !(p->flags & SWP_BLKDEV) >> } > > Well, do you suggested we traverse all pages in lru _before_ > starting vmscan? > No. I don't suggest anything. What I say is just we can do it. If we have to implement it, Couldn't we do it in write_reclaim_page? -- Kind regards, Minchan Kim -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |