: Compress hibernation image with LZO (in-kernel)
in [Kernel]
|
Prev: powerpc: Use IRQF_NO_SUSPEND not IRQF_TIMER for non-timer interrupts
Next: [PATCH 3/9] staging: otus: check kmalloc() return value
From: Bojan Smojver on 30 Jul 2010 21:10 On Sat, 2010-07-31 at 09:40 +1000, Nigel Cunningham wrote: > How about vmallocing the cmp as well? That would greatly reduce the > potential for page allocation failures while still letting you use an > order 6 area. In save_image(), that worked. In load_image() it would cause a crash (something about kernel not being able to satisfy paging request). So, I just made it __get_free_pages() instead. But, yeah good point. Keep in mind that I have absolutely no idea how kernel memory allocation works. I'm kinda coping and pasting code and hoping it doesn't crash :-) > > PS. I guess with this, read_sync can simply disappear as well. > > I haven't looked at the code for a while, but it might still be needed > for the header? I know that in TuxOnIce, I need to read the first page > synchronously when bootstrapping reading the image (can't read the next > page until you know where it is, and its location is on the first page). > Since swsusp uses those index pages, I think it would have the same > issue - they would need to be read before it could read the following > pages. Of course I'm going off memory :) I think it can go, because the header is already read/written with &bio set to NULL (sync read). See patch to remove read_sync. -- Bojan
From: Nigel Cunningham on 30 Jul 2010 21:20 Hi. On 31/07/10 11:03, Bojan Smojver wrote: > On Sat, 2010-07-31 at 09:40 +1000, Nigel Cunningham wrote: > >> How about vmallocing the cmp as well? That would greatly reduce the >> potential for page allocation failures while still letting you use an >> order 6 area. > > In save_image(), that worked. In load_image() it would cause a crash > (something about kernel not being able to satisfy paging request). So, I > just made it __get_free_pages() instead. But, yeah good point. > > Keep in mind that I have absolutely no idea how kernel memory allocation > works. I'm kinda coping and pasting code and hoping it doesn't crash :-) It should be possible to do the allocation at that point. I might see if I can take a look later on. >>> PS. I guess with this, read_sync can simply disappear as well. >> >> I haven't looked at the code for a while, but it might still be needed >> for the header? I know that in TuxOnIce, I need to read the first page >> synchronously when bootstrapping reading the image (can't read the next >> page until you know where it is, and its location is on the first page). >> Since swsusp uses those index pages, I think it would have the same >> issue - they would need to be read before it could read the following >> pages. Of course I'm going off memory :) > > I think it can go, because the header is already read/written with&bio > set to NULL (sync read). See patch to remove read_sync. Okee doke. By the way, please inline your patches. It makes it much easier to read and comment on them. Oh, and I've said it privately but not publicly: great work! Regards, Nigel -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Bojan Smojver on 30 Jul 2010 21:40 On Sat, 2010-07-31 at 11:18 +1000, Nigel Cunningham wrote: > It should be possible to do the allocation at that point. I might see > if I can take a look later on. Maybe it's related to the fact that with vmalloc() you get a different thing then with __get_free_pages() in terms of using it with devices. If found this in Linux Device Drivers: --------------- Addresses allocated by vmalloc can't be used outside of the microprocessor, because they make sense only on top of the processor's MMU. When a driver needs a real physical address (such as a DMA address, used by peripheral hardware to drive the system's bus), you can't easily use vmalloc. The right time to call vmalloc is when you are allocating memory for a large sequential buffer that exists only in software. --------------- I'm guessing the page allocated by vmalloc() eventually gets passed down the I/O chain and is supposed to be read into by some hardware, which then causes a crash. (He, he... kernel according to Bojan - please ignore :-). In the first iteration of the patch, I allocated just one page using __get_free_page() and used that for I/O operations. The only overhead there was memcpy() from the page to vmalloc()-ed cmp. I can go back to that easily. It didn't actually make any significant difference to performance. -- Bojan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Bojan Smojver on 31 Jul 2010 00:50 On Sat, 2010-07-31 at 11:33 +1000, Bojan Smojver wrote: > I can go back to that easily. So, here is that whole enchilada one more time (it includes sync_read removal patch as well). I did 3 hibernate/thaw cycles with it. Images varied from about 850 MB, 1.1 GB to 750 MB. I was getting 156/141 MB/s, 121/118 MBs and 141/130 MBs speeds. Obviously, these things depend on compression ratios achieved etc. I guess the number of pages (i.e. LZO_UNC_PAGES) could be made configurable as well. PS. Inline, as requested. diff --git a/kernel/power/Kconfig b/kernel/power/Kconfig index ca6066a..cb57eb9 100644 --- a/kernel/power/Kconfig +++ b/kernel/power/Kconfig @@ -137,6 +137,8 @@ config SUSPEND_FREEZER config HIBERNATION bool "Hibernation (aka 'suspend to disk')" depends on PM && SWAP && ARCH_HIBERNATION_POSSIBLE + select LZO_COMPRESS + select LZO_DECOMPRESS select SUSPEND_NVS if HAS_IOMEM ---help--- Enable the suspend to disk (STD) functionality, which is usually diff --git a/kernel/power/power.h b/kernel/power/power.h index 006270f..a760cf8 100644 --- a/kernel/power/power.h +++ b/kernel/power/power.h @@ -103,10 +103,6 @@ struct snapshot_handle { void *buffer; /* address of the block to read from * or write to */ - int sync_read; /* Set to one to notify the caller of - * snapshot_write_next() that it may - * need to call wait_on_bio_chain() - */ }; /* This macro returns the address from/to which the caller of diff --git a/kernel/power/snapshot.c b/kernel/power/snapshot.c index 25ce010..f24ee24 100644 --- a/kernel/power/snapshot.c +++ b/kernel/power/snapshot.c @@ -2135,8 +2135,6 @@ int snapshot_write_next(struct snapshot_handle *handle) if (handle->cur > 1 && handle->cur > nr_meta_pages + nr_copy_pages) return 0; - handle->sync_read = 1; - if (!handle->cur) { if (!buffer) /* This makes the buffer be freed by swsusp_free() */ @@ -2169,7 +2167,6 @@ int snapshot_write_next(struct snapshot_handle *handle) memory_bm_position_reset(&orig_bm); restore_pblist = NULL; handle->buffer = get_buffer(&orig_bm, &ca); - handle->sync_read = 0; if (IS_ERR(handle->buffer)) return PTR_ERR(handle->buffer); } @@ -2178,8 +2175,6 @@ int snapshot_write_next(struct snapshot_handle *handle) handle->buffer = get_buffer(&orig_bm, &ca); if (IS_ERR(handle->buffer)) return PTR_ERR(handle->buffer); - if (handle->buffer != buffer) - handle->sync_read = 0; } handle->cur++; return PAGE_SIZE; diff --git a/kernel/power/swap.c b/kernel/power/swap.c index b0bb217..49a9d30 100644 --- a/kernel/power/swap.c +++ b/kernel/power/swap.c @@ -24,6 +24,7 @@ #include <linux/swapops.h> #include <linux/pm.h> #include <linux/slab.h> +#include <linux/lzo.h> #include "power.h" @@ -357,6 +358,13 @@ static int swap_writer_finish(struct swap_map_handle *handle, return error; } +#define LZO_HEADER sizeof(size_t) +#define LZO_UNC_PAGES 64 +#define LZO_UNC_SIZE (LZO_UNC_PAGES * PAGE_SIZE) +#define LZO_CMP_PAGES DIV_ROUND_UP(lzo1x_worst_compress(LZO_UNC_SIZE) + \ + LZO_HEADER, PAGE_SIZE) +#define LZO_CMP_SIZE (LZO_CMP_PAGES * PAGE_SIZE) + /** * save_image - save the suspend image data */ @@ -372,6 +380,38 @@ static int save_image(struct swap_map_handle *handle, struct bio *bio; struct timeval start; struct timeval stop; + size_t ul, cl; + unsigned char *unc, *cmp, *wrk, *page; + + page = (void *)__get_free_page(__GFP_WAIT | __GFP_HIGH); + if (!page) { + printk(KERN_ERR "PM: Failed to allocate LZO page\n"); + return -ENOMEM; + } + + wrk = vmalloc(LZO1X_1_MEM_COMPRESS); + if (!wrk) { + printk(KERN_ERR "PM: Failed to allocate LZO workspace\n"); + free_page((unsigned long)page); + return -ENOMEM; + } + + unc = vmalloc(LZO_UNC_SIZE); + if (!unc) { + printk(KERN_ERR "PM: Failed to allocate LZO uncompressed\n"); + vfree(wrk); + free_page((unsigned long)page); + return -ENOMEM; + } + + cmp = vmalloc(LZO_CMP_SIZE); + if (!cmp) { + printk(KERN_ERR "PM: Failed to allocate LZO compressed\n"); + vfree(unc); + vfree(wrk); + free_page((unsigned long)page); + return -ENOMEM; + } printk(KERN_INFO "PM: Saving image data pages (%u pages) ... ", nr_to_write); @@ -382,16 +422,48 @@ static int save_image(struct swap_map_handle *handle, bio = NULL; do_gettimeofday(&start); while (1) { - ret = snapshot_read_next(snapshot); - if (ret <= 0) + for (ul = 0; ul < LZO_UNC_SIZE; ul += PAGE_SIZE) { + ret = snapshot_read_next(snapshot); + if (ret < 0) + goto out_finish; + + if (ret == 0) + break; + + memcpy(unc + ul, data_of(*snapshot), PAGE_SIZE); + + if (!(nr_pages % m)) + printk(KERN_CONT "\b\b\b\b%3d%%", nr_pages / m); + nr_pages++; + } + + if (ul == 0) + break; + + ret = lzo1x_1_compress(unc, ul, cmp + LZO_HEADER, &cl, wrk); + if (ret < 0) { + printk(KERN_ERR "PM: LZO compression failed\n"); break; - ret = swap_write_page(handle, data_of(*snapshot), &bio); - if (ret) + } + + if (unlikely(cl == 0 || LZO_HEADER + cl > LZO_CMP_SIZE)) { + printk(KERN_ERR "PM: Invalid LZO length\n"); + ret = -1; break; - if (!(nr_pages % m)) - printk(KERN_CONT "\b\b\b\b%3d%%", nr_pages / m); - nr_pages++; + } + + *(size_t *)cmp = cl; + + for (ul = 0; ul < LZO_HEADER + cl; ul += PAGE_SIZE) { + memcpy(page, cmp + ul, PAGE_SIZE); + + ret = swap_write_page(handle, page, &bio); + if (ret) + goto out_finish; + } } + +out_finish: err2 = hib_wait_on_bio_chain(&bio); do_gettimeofday(&stop); if (!ret) @@ -401,6 +473,12 @@ static int save_image(struct swap_map_handle *handle, else printk(KERN_CONT "\n"); swsusp_show_speed(&start, &stop, nr_to_write, "Wrote"); + + vfree(cmp); + vfree(unc); + vfree(wrk); + free_page((unsigned long)page); + return ret; } @@ -416,7 +494,8 @@ static int enough_swap(unsigned int nr_pages) unsigned int free_swap = count_swap_pages(root_swap, 1); pr_debug("PM: Free swap pages: %u\n", free_swap); - return free_swap > nr_pages + PAGES_FOR_IO; + return free_swap > + (nr_pages * LZO_CMP_PAGES) / LZO_UNC_PAGES + PAGES_FOR_IO; } /** @@ -547,9 +626,30 @@ static int load_image(struct swap_map_handle *handle, int error = 0; struct timeval start; struct timeval stop; - struct bio *bio; - int err2; unsigned nr_pages; + size_t ul, cl; + unsigned char *unc, *cmp, *page; + + page = (void *)__get_free_page(__GFP_WAIT | __GFP_HIGH); + if (!page) { + printk(KERN_ERR "PM: Failed to allocate LZO page\n"); + return -ENOMEM; + } + + unc = vmalloc(LZO_UNC_SIZE); + if (!unc) { + printk(KERN_ERR "PM: Failed to allocate LZO uncompressed\n"); + free_page((unsigned long)page); + return -ENOMEM; + } + + cmp = vmalloc(LZO_CMP_SIZE); + if (!cmp) { + printk(KERN_ERR "PM: Failed to allocate LZO compressed\n"); + vfree(unc); + free_page((unsigned long)page); + return -ENOMEM; + } printk(KERN_INFO "PM: Loading image data pages (%u pages) ... ", nr_to_read); @@ -557,27 +657,60 @@ static int load_image(struct swap_map_handle *handle, if (!m) m = 1; nr_pages = 0; - bio = NULL; do_gettimeofday(&start); + + error = snapshot_write_next(snapshot); + if (error <= 0) + goto out_finish; + for ( ; ; ) { - error = snapshot_write_next(snapshot); - if (error <= 0) - break; - error = swap_read_page(handle, data_of(*snapshot), &bio); + error = swap_read_page(handle, page, NULL); /* sync */ if (error) break; - if (snapshot->sync_read) - error = hib_wait_on_bio_chain(&bio); - if (error) + memcpy(cmp, page, PAGE_SIZE); + + cl = *(size_t *)cmp; + if (unlikely(cl == 0 || LZO_HEADER + cl > LZO_CMP_SIZE)) { + printk(KERN_ERR "PM: Invalid LZO length\n"); + error = -1; + break; + } + + for (ul = PAGE_SIZE; ul < LZO_HEADER + cl; ul += PAGE_SIZE) { + error = swap_read_page(handle, page, NULL); /* sync */ + if (error) + goto out_finish; + memcpy(cmp + ul, page, PAGE_SIZE); + } + + ul = LZO_UNC_SIZE; + error = lzo1x_decompress_safe(cmp + LZO_HEADER, cl, unc, &ul); + if (error < 0) { + printk(KERN_ERR "PM: LZO decompression failed\n"); break; - if (!(nr_pages % m)) - printk("\b\b\b\b%3d%%", nr_pages / m); - nr_pages++; + } + + if (unlikely(ul == 0 || ul > LZO_UNC_SIZE)) { + printk(KERN_ERR "PM: Invalid LZO length\n"); + error = -1; + break; + } + + for (cl = 0; cl < ul; cl += PAGE_SIZE) { + memcpy(data_of(*snapshot), unc + cl, PAGE_SIZE); + + if (!(nr_pages % m)) + printk("\b\b\b\b%3d%%", nr_pages / m); + nr_pages++; + + error = snapshot_write_next(snapshot); + if (error <= 0) + goto out_finish; + } } - err2 = hib_wait_on_bio_chain(&bio); + +out_finish: do_gettimeofday(&stop); - if (!error) - error = err2; if (!error) { printk("\b\b\b\bdone\n"); snapshot_write_finalize(snapshot); @@ -586,6 +719,11 @@ static int load_image(struct swap_map_handle *handle, } else printk("\n"); swsusp_show_speed(&start, &stop, nr_to_read, "Read"); + + vfree(cmp); + vfree(unc); + free_page((unsigned long)page); + return error; } -- Bojan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
From: Bojan Smojver on 31 Jul 2010 01:10
On Sat, 2010-07-31 at 14:41 +1000, Bojan Smojver wrote: > I did 3 hibernate/thaw cycles with it. Images varied from about 850 > MB, 1.1 GB to 750 MB. I was getting 156/141 MB/s, 121/118 MBs and > 141/130 MBs speeds. Obviously, these things depend on compression > ratios achieved etc. For the record, my machine usually does around 32 MB/s on hibernate and 54 MB/s on thaw with regular in-kernel swsusp. -- Bojan -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo(a)vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/ |