Accelerating swapless machines
in [UK Linux]
|
From: Theo Markettos on 26 Feb 2010 21:10 Gordon Henderson <gordon+usenet(a)drogon.net> wrote: > More RAM will only help if you can move stuff into it as a ramdisk. > > It's writes to it that will generally cause "clunkyness" - and there is > a multitude of things doing the writes that happen behind the scenes... Hmmm... I wonder if there's anything like 'profiling' for files/syscalls? So I can make a note of which files are being thrashed? > It can be argued that "it's a laptop, no need for syslog or it's > variants"... So it might be worthwhile seeing what's actually running > and kill it off... I'll have a look. Though syslog is handy when the wireless screws up, which it can do in exciting ways... > You may not be doing disk stuff, but applications will be - web browser > will be maintaining a cache and it may well be fsyncing it's index file > - same for cookies and browsing history - firefox tries really hard now > to try to maintain as much state as possible between crashes... True. Though usually the slowness isn't because I'm actively doing anything, everything just grinds to a halt occasionally. It could be a huge file access backlog, I suppose. > And ext3 is going to be writing to disk every 5 seconds too - there are > ways to tune that though - the easiest is to move to ext2, but for a > laptop which runs out of juice every now & then, ext3 is handy... If the writes aren't actually taking CPU cycles, that's not a problem. It's when they hold up other things that it gets problematic. I'll have a look at the output of iostat, when I next have the machine wedged. Theo
From: Tom Anderson on 27 Feb 2010 05:49 On Sat, 27 Feb 2010, Theo Markettos wrote: > Gordon Henderson <gordon+usenet(a)drogon.net> wrote: >> More RAM will only help if you can move stuff into it as a ramdisk. >> >> It's writes to it that will generally cause "clunkyness" - and there is >> a multitude of things doing the writes that happen behind the scenes... > > Hmmm... I wonder if there's anything like 'profiling' for files/syscalls? strace + awk? You could only profile invididual processes, though, not the whole system. Perhaps ps -ef + awk + strace -p + awk could be used to create a sort of monster tracing system, though. top + awk + strace + awk might be a better idea. tom -- taxidermy, high tide marks, sabotage, markets, folklore, subverting, .
From: Nix on 27 Feb 2010 14:19 On 24 Feb 2010, Theo Markettos outgrape: > 3844 atm26 20 0 627m 352m 27m R 66 35.5 618:00.14 epiphany-browse !!!!! > 3893 atm26 20 0 206m 50m 19m S 40 5.1 167:45.10 liferea-bin !!! a bit of a hog, but I note to my horror that my akregator is using more than 350Mb right now. What the *hell* are modern software authors on? > 2911 root 20 0 344m 68m 13m S 23 6.9 107:06.22 Xorg ow. I'm not surprised you're running really slowly. Your fundamental problem is that you're running appalling memory hogs and the system really *wants* to swap, but can't. It's *possible* that the compressed in-memory swapping code in 2.6.33 may help, but I suspect this is simply a job too far for this machine. I suggest finding browsers that are *actually* trim. My aging Konqueror 3.5.10 with 105 tabs open is smaller than that Epiphany, even though it's a 64-bit app with correspondingly higher memory requirements from increased type sizes and alignment constraints. I'd bet that xrestop would show that a lot of the 344Mb in Xorg is also devoted to storing pixmaps dumped into it by Epiphany, so it's using even more. > and I'm just typing into a terminal, the rest are sitting in the background. > I use epiphany as it's lighter than Firefox, but I think Xorg and Flash are > a big eater of CPU cycles (must try flashblock for epiphany). I do try to > avoid bloated apps. Xorg isn't much of an eater of CPU cycles, and your problem is really RAM, not CPU: I'd actually suggest trading off CPU for RAM, hence compressed in- memory swapping. > Ordinarily the answer to speeding a machine up is 'add more RAM'. I can > understand this from the point of view of reducing swapping, which is slow > and laggy. But this machine has no swap - indeed it only has 8GB of fairly > slow SSD so there would be no point. I have never received an 'out of > memory' error regarding RAM. You don't get out of memory errors on most Linux systems. You get slow systems, then really slow systems, then things start to get killed and messages about it land in the *system log*, not on the screen. (But things need to get really bad by then.)
From: Nix on 27 Feb 2010 14:21 On 24 Feb 2010, Jim A. stated: > Theo Markettos wrote: >> My SSD partition is mounted 'relatime', which is like 'noatime' but doesn't >> break some apps. Is there a noticeable difference between the two? > > I confess I hadn't come across relatime before - it seems it will > result in more disk writes than noatime (a quick google will throw up > plenty of discussion on the subject). More, but not many more. It was specifically designed to reduce write volume while not breaking atime semantics so badly that atime's few users (notably mutt) break. It leads to, on average, one extra inode write per file write -- but does *not* lead to a write per read, unlike conventional atime. > There are (at least) two issues with flash drives. One is that write > speeds can be really slow. The other issue is whether excessive > writes will eventually wear out the ssd. IIRC modern high-end Flash disks are starting to render this academic (as in 'you need to write to it at top speed for fifty years before it starts to wear out'). This definitely hasn't got down to the low end, though, and there's some evidence that even at the high end they do this by throttling writes even mor (thus changing the definition of 'top speed' to make their prediction come out true).
From: Theo Markettos on 27 Feb 2010 17:42
Nix <nix-razor-pit(a)esperi.org.uk> wrote: > On 24 Feb 2010, Theo Markettos outgrape: > > 3844 atm26 20 0 627m 352m 27m R 66 35.5 618:00.14 epiphany-browse > > !!!!! The machine ran out of battery on me, but I think in that configuration I had a dozen or so tabs open, possibly a few more. There was no doubt some Flash doing pointless things in invisible windows too. > > 3893 atm26 20 0 206m 50m 19m S 40 5.1 167:45.10 liferea-bin > > !!! a bit of a hog, but I note to my horror that my akregator is using > more than 350Mb right now. What the *hell* are modern software authors > on? Well, quite. It's just a bunch of RSS feeds. It might be a hundred or so, but it's only XML. I think I had about 4 tabs actually showing webpages. > > 2911 root 20 0 344m 68m 13m S 23 6.9 107:06.22 Xorg > > ow. I think that's Flash pointlessly animating things. Oh for the day HTML5 is useful. > I'm not surprised you're running really slowly. Your fundamental problem > is that you're running appalling memory hogs and the system really > *wants* to swap, but can't. It's *possible* that the compressed > in-memory swapping code in 2.6.33 may help, but I suspect this is simply > a job too far for this machine. That's why I was wondering about adding RAM. But folks seemed to suggest it was disc that's the bottleneck, which is why I'm confused. > I suggest finding browsers that are *actually* trim. My aging Konqueror > 3.5.10 with 105 tabs open is smaller than that Epiphany, even though it's > a 64-bit app with correspondingly higher memory requirements from increased > type sizes and alignment constraints. I'd bet that xrestop would show that a > lot of the 344Mb in Xorg is also devoted to storing pixmaps dumped into it by > Epiphany, so it's using even more. I do use Netsurf as a trim browser, but it's not that useful for interactive sites (it has no Javascript). I use Konqueror elsewhere and find it a bit primitive. Opera after about version 7 seems to be a CPU hog (on the 'bigger' Centrino machine). I suppose there's Chrome(ium) and Webkit-based browsers to try. Any particularly worth looking at? > > and I'm just typing into a terminal, the rest are sitting in the background. > > I use epiphany as it's lighter than Firefox, but I think Xorg and Flash are > > a big eater of CPU cycles (must try flashblock for epiphany). I do try to > > avoid bloated apps. > > Xorg isn't much of an eater of CPU cycles, and your problem is really RAM, > not CPU: I'd actually suggest trading off CPU for RAM, hence compressed in- > memory swapping. Hmm... that's not my experience. On the 'other' machine (1.6GHz Centrino. 1.5GB RAM, 2GB swap): top - 22:34:25 up 24 days, 12:25, 7 users, load average: 0.87, 0.94, 0.95 Tasks: 198 total, 2 running, 195 sleeping, 1 stopped, 0 zombie Cpu(s): 60.5%us, 7.7%sy, 0.0%ni, 31.2%id, 0.0%wa, 0.2%hi, 0.4%si, 0.0%st Mem: 1544104k total, 1486452k used, 57652k free, 20920k buffers Swap: 1991504k total, 735416k used, 1256088k free, 185372k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 15932 atm26 20 0 922m 732m 24m R 61.1 48.6 1004:37 firefox 3929 root 20 0 276m 101m 8032 S 2.1 6.7 374:39.55 Xorg 5084 atm26 20 0 25100 4944 3992 S 0.7 0.3 75:49.17 multiload-apple 17597 atm26 20 0 124m 20m 6488 S 0.5 1.3 21:56.19 gnome-terminal That's a ratio of 1:3 Firefox to Xorg execution time. And yes, Firefox is taking that much CPU just sitting in the background. Probably Flash again, despite having Flashblock active (did you notice I'm not Flash's greatest fan?) > You don't get out of memory errors on most Linux systems. You get slow > systems, then really slow systems, then things start to get killed and > messages about it land in the *system log*, not on the screen. (But > things need to get really bad by then.) Even if there's no swap to resort to? I can understand thrashing swap to death, but if there's nothing to swap to and you try to run a zillion applications at once, what happens? One option is to leave all the binaries on disc and only mmap() in small chunks of code. This will make things very slow. But that doesn't help for data. If I do: char i[2*1024*1024*1024]; that will surely fail, full stop? Theo |