memcg: per cgroup dirty limit (v7)
in [Kernel]
|
Prev: linux-next: manual merge of the fsnotify tree with Linus' tree
Next: genirq: warn about IRQF_SHARED|IRQF_DISABLED at the right place
From: Balbir Singh on 17 Mar 2010 08:00 * Vivek Goyal <vgoyal(a)redhat.com> [2010-03-15 13:19:21]: > On Mon, Mar 15, 2010 at 01:12:09PM -0400, Vivek Goyal wrote: > > On Mon, Mar 15, 2010 at 12:26:37AM +0100, Andrea Righi wrote: > > > Control the maximum amount of dirty pages a cgroup can have at any given time. > > > > > > Per cgroup dirty limit is like fixing the max amount of dirty (hard to reclaim) > > > page cache used by any cgroup. So, in case of multiple cgroup writers, they > > > will not be able to consume more than their designated share of dirty pages and > > > will be forced to perform write-out if they cross that limit. > > > > > > > For me even with this version I see that group with 100M limit is getting > > much more BW. > > > > root cgroup > > ========== > > #time dd if=/dev/zero of=/root/zerofile bs=4K count=1M > > 4294967296 bytes (4.3 GB) copied, 55.7979 s, 77.0 MB/s > > > > real 0m56.209s > > > > test1 cgroup with memory limit of 100M > > ====================================== > > # time dd if=/dev/zero of=/root/zerofile1 bs=4K count=1M > > 4294967296 bytes (4.3 GB) copied, 20.9252 s, 205 MB/s > > > > real 0m21.096s > > > > Note, these two jobs are not running in parallel. These are running one > > after the other. > > > > Ok, here is the strange part. I am seeing similar behavior even without > your patches applied. > > root cgroup > ========== > #time dd if=/dev/zero of=/root/zerofile bs=4K count=1M > 4294967296 bytes (4.3 GB) copied, 56.098 s, 76.6 MB/s > > real 0m56.614s > > test1 cgroup with memory limit 100M > =================================== > # time dd if=/dev/zero of=/root/zerofile1 bs=4K count=1M > 4294967296 bytes (4.3 GB) copied, 19.8097 s, 217 MB/s > > real 0m19.992s > This is strange, did you flish the cache between the two runs? NOTE: Since the files are same, we reuse page cache from the other cgroup. -- Three Cheers, Balbir -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Vivek Goyal on 17 Mar 2010 09:40 On Wed, Mar 17, 2010 at 05:24:28PM +0530, Balbir Singh wrote: > * Vivek Goyal <vgoyal(a)redhat.com> [2010-03-15 13:19:21]: > > > On Mon, Mar 15, 2010 at 01:12:09PM -0400, Vivek Goyal wrote: > > > On Mon, Mar 15, 2010 at 12:26:37AM +0100, Andrea Righi wrote: > > > > Control the maximum amount of dirty pages a cgroup can have at any given time. > > > > > > > > Per cgroup dirty limit is like fixing the max amount of dirty (hard to reclaim) > > > > page cache used by any cgroup. So, in case of multiple cgroup writers, they > > > > will not be able to consume more than their designated share of dirty pages and > > > > will be forced to perform write-out if they cross that limit. > > > > > > > > > > For me even with this version I see that group with 100M limit is getting > > > much more BW. > > > > > > root cgroup > > > ========== > > > #time dd if=/dev/zero of=/root/zerofile bs=4K count=1M > > > 4294967296 bytes (4.3 GB) copied, 55.7979 s, 77.0 MB/s > > > > > > real 0m56.209s > > > > > > test1 cgroup with memory limit of 100M > > > ====================================== > > > # time dd if=/dev/zero of=/root/zerofile1 bs=4K count=1M > > > 4294967296 bytes (4.3 GB) copied, 20.9252 s, 205 MB/s > > > > > > real 0m21.096s > > > > > > Note, these two jobs are not running in parallel. These are running one > > > after the other. > > > > > > > Ok, here is the strange part. I am seeing similar behavior even without > > your patches applied. > > > > root cgroup > > ========== > > #time dd if=/dev/zero of=/root/zerofile bs=4K count=1M > > 4294967296 bytes (4.3 GB) copied, 56.098 s, 76.6 MB/s > > > > real 0m56.614s > > > > test1 cgroup with memory limit 100M > > =================================== > > # time dd if=/dev/zero of=/root/zerofile1 bs=4K count=1M > > 4294967296 bytes (4.3 GB) copied, 19.8097 s, 217 MB/s > > > > real 0m19.992s > > > > This is strange, did you flish the cache between the two runs? > NOTE: Since the files are same, we reuse page cache from the > other cgroup. Files are different. Note suffix "1". Vivek -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Balbir Singh on 17 Mar 2010 15:00 * Vivek Goyal <vgoyal(a)redhat.com> [2010-03-17 09:34:07]: > > > > > > root cgroup > > > ========== > > > #time dd if=/dev/zero of=/root/zerofile bs=4K count=1M > > > 4294967296 bytes (4.3 GB) copied, 56.098 s, 76.6 MB/s > > > > > > real 0m56.614s > > > > > > test1 cgroup with memory limit 100M > > > =================================== > > > # time dd if=/dev/zero of=/root/zerofile1 bs=4K count=1M > > > 4294967296 bytes (4.3 GB) copied, 19.8097 s, 217 MB/s > > > > > > real 0m19.992s > > > > > > > This is strange, did you flish the cache between the two runs? > > NOTE: Since the files are same, we reuse page cache from the > > other cgroup. > > Files are different. Note suffix "1". > Thanks, I'll get the perf output and see what I get. -- Three Cheers, Balbir -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Balbir Singh on 17 Mar 2010 15:20 * Vivek Goyal <vgoyal(a)redhat.com> [2010-03-17 09:34:07]: > On Wed, Mar 17, 2010 at 05:24:28PM +0530, Balbir Singh wrote: > > * Vivek Goyal <vgoyal(a)redhat.com> [2010-03-15 13:19:21]: > > > > > On Mon, Mar 15, 2010 at 01:12:09PM -0400, Vivek Goyal wrote: > > > > On Mon, Mar 15, 2010 at 12:26:37AM +0100, Andrea Righi wrote: > > > > > Control the maximum amount of dirty pages a cgroup can have at any given time. > > > > > > > > > > Per cgroup dirty limit is like fixing the max amount of dirty (hard to reclaim) > > > > > page cache used by any cgroup. So, in case of multiple cgroup writers, they > > > > > will not be able to consume more than their designated share of dirty pages and > > > > > will be forced to perform write-out if they cross that limit. > > > > > > > > > > > > > For me even with this version I see that group with 100M limit is getting > > > > much more BW. > > > > > > > > root cgroup > > > > ========== > > > > #time dd if=/dev/zero of=/root/zerofile bs=4K count=1M > > > > 4294967296 bytes (4.3 GB) copied, 55.7979 s, 77.0 MB/s > > > > > > > > real 0m56.209s > > > > > > > > test1 cgroup with memory limit of 100M > > > > ====================================== > > > > # time dd if=/dev/zero of=/root/zerofile1 bs=4K count=1M > > > > 4294967296 bytes (4.3 GB) copied, 20.9252 s, 205 MB/s > > > > > > > > real 0m21.096s > > > > > > > > Note, these two jobs are not running in parallel. These are running one > > > > after the other. > > > > > > > The data is not always repeatable at my end. Are you able to modify the order and get repeatable results? In fact, I saw for cgroup != root ------------------ 4294967296 bytes (4.3 GB) copied, 120.359 s, 35.7 MB/s for cgroup = root ----------------- 4294967296 bytes (4.3 GB) copied, 84.504 s, 50.8 MB/s This is without the patches applied. -- Three Cheers, Balbir -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Vivek Goyal on 17 Mar 2010 15:20
On Thu, Mar 18, 2010 at 12:23:27AM +0530, Balbir Singh wrote: > * Vivek Goyal <vgoyal(a)redhat.com> [2010-03-17 09:34:07]: > > > > > > > > > root cgroup > > > > ========== > > > > #time dd if=/dev/zero of=/root/zerofile bs=4K count=1M > > > > 4294967296 bytes (4.3 GB) copied, 56.098 s, 76.6 MB/s > > > > > > > > real 0m56.614s > > > > > > > > test1 cgroup with memory limit 100M > > > > =================================== > > > > # time dd if=/dev/zero of=/root/zerofile1 bs=4K count=1M > > > > 4294967296 bytes (4.3 GB) copied, 19.8097 s, 217 MB/s > > > > > > > > real 0m19.992s > > > > > > > > > > This is strange, did you flish the cache between the two runs? > > > NOTE: Since the files are same, we reuse page cache from the > > > other cgroup. > > > > Files are different. Note suffix "1". > > > > Thanks, I'll get the perf output and see what I get. One more thing I noticed and that is, it happens only if we limit the memory of cgroup to 100M. If same cgroup test1 is unlimited memory thing, then it did not happen. I also did not notice this happening on other system where I have 4G of memory. So it also seems to be related with only bigger configurations. Thanks Vivek > > -- > Three Cheers, > Balbir -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |