Btrfs: Add hot data tracking functionality
in [Kernel]
|
Prev: [PATCH 4/4] Checkpatch: warn about unexpectedly long msleep's
Next: [RFC][PATCH] PCI / PCIe: Ask BIOS for control of all native services at once (v2)
From: Tracy Reed on 27 Jul 2010 19:00 On Tue, Jul 27, 2010 at 05:00:18PM -0500, bchociej(a)gmail.com spake thusly: > The long-term goal of these patches, as discussed in the Motivation > section at the end of this message, is to enable Btrfs to perform > automagic relocation of hot data to fast media like SSD. This goal has > been motivated by the Project Ideas page on the Btrfs wiki. With disks being so highly virtualized away these days is there any way for btrfs to know which are the fast outer-tracks vs the slower inner-tracks of a physical disk? If so not only could this benefit SSD owners but it could also benefit the many more spinning platters out there. If not (which wouldn't be surprising) then disregard. Even just having that sort of functionality for SSD would be excellent. If I understand correctly not only would this work for SSD but if I have a SAN full of many large 7200rpm disks and a few 15k SAS disks I could effectively utilize that disk by allowing btrfs to place hot data on the 15k SAS. I understand Compellent does this as well. -- Tracy Reed http://tracyreed.org
From: Diego Calleja on 27 Jul 2010 19:20 On Mi�rcoles, 28 de Julio de 2010 00:00:18 bchociej(a)gmail.com escribi�: > With Btrfs's COW approach, an external cache (where data is moved to > SSD, rather than just cached there) makes a lot of sense. Though these As I understand it, what your proyect intends to do is to move "hot" data to a SSD which would be part of a Btrfs pool, and not do any kind of SSD caching, as bcache (http://lwn.net/Articles/394672/) does? -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Ben Chociej on 27 Jul 2010 19:20 On Tue, Jul 27, 2010 at 6:10 PM, Diego Calleja <diegocg(a)gmail.com> wrote: > On Mi�rcoles, 28 de Julio de 2010 00:00:18 bchociej(a)gmail.com escribi�: >> With Btrfs's COW approach, an external cache (where data is moved to >> SSD, rather than just cached there) makes a lot of sense. Though these > > As I understand it, what your proyect intends to do is to move "hot" > data to a SSD which would be part of a Btrfs pool, and not do any > kind of SSD caching, as bcache (http://lwn.net/Articles/394672/) does? > Yes, that's correct. It's likely not going to be a cache in the traditional sense, since the entire capacity of both HDD and SSD would be available. BC -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Christian Stroetmann on 27 Jul 2010 19:40 At the 28.07.2010 00:00, Ben Chociej wrote: > INTRODUCTION: > > This patch series adds experimental support for tracking data > temperature in Btrfs. Essentially, this means maintaining some key > stats (like number of reads/writes, last read/write time, frequency of > reads/writes), then distilling those numbers down to a single > "temperature" value that reflects what data is "hot." > > The long-term goal of these patches, as discussed in the Motivation > section at the end of this message, is to enable Btrfs to perform > automagic relocation of hot data to fast media like SSD. This goal has > been motivated by the Project Ideas page on the Btrfs wiki. > > Of course, users are warned not to run this code outside of development > environments. These patches are EXPERIMENTAL, and as such they might > eat your data and/or memory. > > > MOTIVATION: > > The overall goal of enabling hot data relocation to SSD has been > motivated by the Project Ideas page on the Btrfs wiki at > https://btrfs.wiki.kernel.org/index.php/Project_ideas. It is hoped that > this initial patchset will eventually mature into a usable hybrid > storage feature set for Btrfs. > > This is essentially the traditional cache argument: SSD is fast and > expensive; HDD is cheap but slow. ZFS, for example, can already take > advantage of SSD caching. Btrfs should also be able to take advantage > of hybrid storage without any broad, sweeping changes to existing code. > Wouldn't this feature be useful for other file systems as well, so that a more general and not an only Btrfs related solution is preferable? > With Btrfs's COW approach, an external cache (where data is *moved* to > SSD, rather than just cached there) makes a lot of sense. Though these > patches don't enable any relocation yet, they do lay an essential > foundation for enabling that functionality in the near future. We plan > to roll out an additional patchset introducing some of the automatic > migration functionality in the next few weeks. > > With all the best Christian Stroetmann -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Chris Samuel on 28 Jul 2010 08:40
On Wed, 28 Jul 2010 09:18:23 am Ben Chociej wrote: > Yes, that's correct. It's likely not going to be a cache in the > traditional sense, since the entire capacity of both HDD and SSD would > be available. To me that sounds like an HSM type arrangement, with most frequently used data on the highest performing media and less frequently touched data getting shunted down the chain to SAS, SATA and then tape and/or MAID type devices. Certainly interesting from my HPC point of view in that I can see it being useful to parallel filesystems like Ceph if this "just happens". I guess real HSM devotees would want policies for migration downstream.. ;-) cheers, Chris -- Chris Samuel : http://www.csamuel.org/ : Melbourne, VIC This email may come with a PGP signature as a file. Do not panic. For more info see: http://en.wikipedia.org/wiki/OpenPGP |