Prev: What to backup on a Linux system an what not?
Next: vi - hide some line
From: liuchangjohn on 4 May 2008 10:48
Hi,

I'm trying to diagnose the memory usage of a VPS server that I just
signed up for, and I just can't make sense of what is actually using
all that memory.

`Free` reports that 400+ MB of memory is used. It also shows that no
cache/buffer is used whatsoever. `top`'s summary line agrees with
`free`, but the sum of the memory of the individual processed account
for at most 25% of the physical memory. A script that I found on the
net lists the memory usage by process by inspecting /proc, and it too
reports a total memory much less than 400+ MB.

So, what is using all the 400+ MB of memory? What other diagnostics
tools can be helpful? Thanks in advance!

Regards,
Liu Chang

================================================

[liuchang(a)vps lib]$ free -m
total used free shared buffers
cached
Mem: 512 411 100 0
0 0
-/+ buffers/cache: 411 100
Swap: 0 0 0

=================================================

top - 07:44:00 up 50 min, 2 users, load average: 0.00, 0.02, 0.00
Tasks: 28 total, 1 running, 27 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.0% us, 0.1% sy, 0.0% ni, 99.9% id, 0.0% wa, 0.0% hi,
0.0% si
Mem: 524288k total, 422656k used, 101632k free, 0k
buffers
Swap: 0k total, 0k used, 0k free, 0k
cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
30313 liuchang 16 0 89632 35m 13m S 0 6.9 0:04.12 firefox-
bin
30030 liuchang 15 0 87164 33m 13m S 0 6.5 0:03.12 firefox-
bin
28453 liuchang 18 0 136m 20m 4904 S 0 4.0 0:01.21 java
25924 mysql 16 0 121m 18m 4080 S 0 3.7 0:00.11 mysqld
28382 liuchang 15 0 11624 9020 1832 S 0 1.7 0:02.19 Xvnc
28388 liuchang 16 0 5412 3204 2500 S 0 0.6 0:00.05 blackbox
26040 root 15 0 6908 2212 1788 S 0 0.4 0:00.01 sshd
29741 root 15 0 6908 2212 1788 S 0 0.4 0:00.01 sshd
30022 liuchang 16 0 3436 2096 1632 S 0 0.4 0:00.02 gconfd-2
25808 root 16 0 5856 1792 1364 S 0 0.3 0:00.00 cupsd
28386 liuchang 16 0 3752 1536 1296 S 0 0.3 0:00.00 vncconfig
25970 xfs 16 0 3252 1524 728 S 0 0.3 0:00.00 xfs
29746 liuchang 16 0 6908 1508 1060 S 0 0.3 0:00.05 sshd
26044 liuchang 15 0 6908 1488 1056 S 0 0.3 0:00.10 sshd
26045 liuchang 16 0 2160 1320 1076 S 0 0.3 0:00.04 bash
29747 liuchang 16 0 2160 1304 1064 S 0 0.2 0:00.01 bash
30599 root 16 0 2164 1304 1048 S 0 0.2 0:00.00 bash
25830 root 15 0 4028 1136 840 S 0 0.2 0:00.02 sshd
25891 root 17 0 2152 1116 956 S 0 0.2 0:00.00
mysqld_safe
30591 root 17 0 2304 1076 852 S 0 0.2 0:00.00 su
26010 root 19 0 2724 1048 924 S 0 0.2 0:00.00 cups-
config-dae
26035 root 18 0 3728 992 768 S 0 0.2 0:00.00 vsftpd
30616 liuchang 16 0 1896 972 788 R 0 0.2 0:00.04 top
26000 dbus 16 0 2400 932 784 S 0 0.2 0:00.00 dbus-
daemon-1
25948 root 16 0 2484 912 524 S 0 0.2 0:00.00 crond
25990 root 22 0 4060 796 568 S 0 0.2 0:00.00 saslauthd
1 root 16 0 1628 600 520 S 0 0.1 0:00.02 init
25772 root 16 0 1532 524 440 S 0 0.1 0:00.00 syslogd

==========================================

[root(a)vps bin]# ./ps_mem.py
Private + Shared = RAM used Program

84.0 KiB + 440.0 KiB = 524.0 KiB syslogd
80.0 KiB + 520.0 KiB = 600.0 KiB init
672.0 KiB + 0.0 KiB = 672.0 KiB ps
228.0 KiB + 568.0 KiB = 796.0 KiB saslauthd
388.0 KiB + 524.0 KiB = 912.0 KiB crond
148.0 KiB + 784.0 KiB = 932.0 KiB dbus-daemon-1
224.0 KiB + 768.0 KiB = 992.0 KiB vsftpd
124.0 KiB + 924.0 KiB = 1.0 MiB cups-config-dae
224.0 KiB + 852.0 KiB = 1.1 MiB su
160.0 KiB + 956.0 KiB = 1.1 MiB mysqld_safe
796.0 KiB + 728.0 KiB = 1.5 MiB xfs
240.0 KiB + 1.3 MiB = 1.5 MiB vncconfig
428.0 KiB + 1.3 MiB = 1.8 MiB cupsd
748.0 KiB + 1.1 MiB = 1.8 MiB bash (3)
464.0 KiB + 1.6 MiB = 2.0 MiB gconfd-2
1.1 MiB + 2.0 MiB = 3.1 MiB sshd (5)
704.0 KiB + 2.4 MiB = 3.1 MiB blackbox
7.0 MiB + 1.8 MiB = 8.8 MiB Xvnc
14.8 MiB + 4.0 MiB = 18.8 MiB mysqld
15.7 MiB + 4.8 MiB = 20.5 MiB java
40.6 MiB + 14.0 MiB = 54.6 MiB firefox-bin (2)

Private + Shared = RAM used Program
From: Robert Heller on 4 May 2008 11:34
At Sun, 4 May 2008 07:48:29 -0700 (PDT) "liuchangjohn(a)gmail.com" <liuchangjohn(a)gmail.com> wrote:

>
> Hi,
>
> I'm trying to diagnose the memory usage of a VPS server that I just
> signed up for, and I just can't make sense of what is actually using
> all that memory.
>
> `Free` reports that 400+ MB of memory is used. It also shows that no
> cache/buffer is used whatsoever. `top`'s summary line agrees with
> `free`, but the sum of the memory of the individual processed account
> for at most 25% of the physical memory. A script that I found on the
> net lists the memory usage by process by inspecting /proc, and it too
> reports a total memory much less than 400+ MB.

A VPS server will never show the cache usage, since you cannot 'see'
physical memory. The 'parent' server is doing all of the 'real' disk
I/O and doing the caching. All you will see is you virtual memory.

>
> So, what is using all the 400+ MB of memory? What other diagnostics
> tools can be helpful? Thanks in advance!

ps and top should show you what is going on.

>
> Regards,
> Liu Chang
>
> ================================================
>
> [liuchang(a)vps lib]$ free -m
> total used free shared buffers
> cached
> Mem: 512 411 100 0
> 0 0
> -/+ buffers/cache: 411 100
> Swap: 0 0 0
>
> =================================================
>
> top - 07:44:00 up 50 min, 2 users, load average: 0.00, 0.02, 0.00
> Tasks: 28 total, 1 running, 27 sleeping, 0 stopped, 0 zombie
> Cpu(s): 0.0% us, 0.1% sy, 0.0% ni, 99.9% id, 0.0% wa, 0.0% hi,
> 0.0% si
> Mem: 524288k total, 422656k used, 101632k free, 0k
> buffers
> Swap: 0k total, 0k used, 0k free, 0k
> cached
>
> PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> 30313 liuchang 16 0 89632 35m 13m S 0 6.9 0:04.12 firefox-
> bin
> 30030 liuchang 15 0 87164 33m 13m S 0 6.5 0:03.12 firefox-
> bin
89632+87164 = almost 173meg
> 28453 liuchang 18 0 136m 20m 4904 S 0 4.0 0:01.21 java
> 25924 mysql 16 0 121m 18m 4080 S 0 3.7 0:00.11 mysqld

136m+121m == 257meg

257+173 = 430meg

So, between two firefox's, Java, and MySQL, there pretty much goes all
of your virtual memory. Why are you running firefox on a server? Why
is firefox even installed on the server?

MySQL seems to be a real memory hog: 121meg virtual! Compare to
PostgreSQL (only 16meg virtual):

postgres 21666 0.0 0.1 16760 1852 ? S Apr30 0:00 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data -i -h 127.0.0.1

(I just checked: starting MySQL cold grabs 121meg virtual!)


> 28382 liuchang 15 0 11624 9020 1832 S 0 1.7 0:02.19 Xvnc
> 28388 liuchang 16 0 5412 3204 2500 S 0 0.6 0:00.05 blackbox
> 26040 root 15 0 6908 2212 1788 S 0 0.4 0:00.01 sshd
> 29741 root 15 0 6908 2212 1788 S 0 0.4 0:00.01 sshd
> 30022 liuchang 16 0 3436 2096 1632 S 0 0.4 0:00.02 gconfd-2
> 25808 root 16 0 5856 1792 1364 S 0 0.3 0:00.00 cupsd
> 28386 liuchang 16 0 3752 1536 1296 S 0 0.3 0:00.00 vncconfig
> 25970 xfs 16 0 3252 1524 728 S 0 0.3 0:00.00 xfs
> 29746 liuchang 16 0 6908 1508 1060 S 0 0.3 0:00.05 sshd
> 26044 liuchang 15 0 6908 1488 1056 S 0 0.3 0:00.10 sshd
> 26045 liuchang 16 0 2160 1320 1076 S 0 0.3 0:00.04 bash
> 29747 liuchang 16 0 2160 1304 1064 S 0 0.2 0:00.01 bash
> 30599 root 16 0 2164 1304 1048 S 0 0.2 0:00.00 bash
> 25830 root 15 0 4028 1136 840 S 0 0.2 0:00.02 sshd
> 25891 root 17 0 2152 1116 956 S 0 0.2 0:00.00
> mysqld_safe

Then you've got the whole rest of your desktop...

> 30591 root 17 0 2304 1076 852 S 0 0.2 0:00.00 su
> 26010 root 19 0 2724 1048 924 S 0 0.2 0:00.00 cups-
> config-dae
> 26035 root 18 0 3728 992 768 S 0 0.2 0:00.00 vsftpd
> 30616 liuchang 16 0 1896 972 788 R 0 0.2 0:00.04 top
> 26000 dbus 16 0 2400 932 784 S 0 0.2 0:00.00 dbus-
> daemon-1
> 25948 root 16 0 2484 912 524 S 0 0.2 0:00.00 crond
> 25990 root 22 0 4060 796 568 S 0 0.2 0:00.00 saslauthd
> 1 root 16 0 1628 600 520 S 0 0.1 0:00.02 init
> 25772 root 16 0 1532 524 440 S 0 0.1 0:00.00 syslogd
>
> ==========================================
>
> [root(a)vps bin]# ./ps_mem.py
> Private + Shared = RAM used Program
>
> 84.0 KiB + 440.0 KiB = 524.0 KiB syslogd
> 80.0 KiB + 520.0 KiB = 600.0 KiB init
> 672.0 KiB + 0.0 KiB = 672.0 KiB ps
> 228.0 KiB + 568.0 KiB = 796.0 KiB saslauthd
> 388.0 KiB + 524.0 KiB = 912.0 KiB crond
> 148.0 KiB + 784.0 KiB = 932.0 KiB dbus-daemon-1
> 224.0 KiB + 768.0 KiB = 992.0 KiB vsftpd
> 124.0 KiB + 924.0 KiB = 1.0 MiB cups-config-dae
> 224.0 KiB + 852.0 KiB = 1.1 MiB su
> 160.0 KiB + 956.0 KiB = 1.1 MiB mysqld_safe
> 796.0 KiB + 728.0 KiB = 1.5 MiB xfs
> 240.0 KiB + 1.3 MiB = 1.5 MiB vncconfig
> 428.0 KiB + 1.3 MiB = 1.8 MiB cupsd
> 748.0 KiB + 1.1 MiB = 1.8 MiB bash (3)
> 464.0 KiB + 1.6 MiB = 2.0 MiB gconfd-2
> 1.1 MiB + 2.0 MiB = 3.1 MiB sshd (5)
> 704.0 KiB + 2.4 MiB = 3.1 MiB blackbox
> 7.0 MiB + 1.8 MiB = 8.8 MiB Xvnc
> 14.8 MiB + 4.0 MiB = 18.8 MiB mysqld
> 15.7 MiB + 4.8 MiB = 20.5 MiB java
> 40.6 MiB + 14.0 MiB = 54.6 MiB firefox-bin (2)
>
> Private + Shared = RAM used Program
>

--
Robert Heller -- Get the Deepwoods Software FireFox Toolbar!
Deepwoods Software -- Linux Installation and Administration
http://www.deepsoft.com/ -- Web Hosting, with CGI and Database
heller(a)deepsoft.com -- Contract Programming: C/C++, Tcl/Tk

From: liuchangjohn on 4 May 2008 13:17
On May 4, 11:34 pm, Robert Heller <hel...(a)deepsoft.com> wrote:
> At Sun, 4 May 2008 07:48:29 -0700 (PDT) "liuchangj...(a)gmail.com" <liuchangj...(a)gmail.com> wrote:
>
>
>
> > Hi,
>
> > I'm trying to diagnose thememoryusage of a VPS server that I just
> > signed up for, and I just can't make sense of what is actually using
> > all thatmemory.
>
> > `Free` reports that 400+ MB ofmemoryis used. It also shows that no
> > cache/buffer is used whatsoever. `top`'s summary line agrees with
> > `free`, but the sum of thememoryof the individual processed account
> > for at most 25% of the physicalmemory. A script that I found on the
> > net lists thememoryusage by process by inspecting /proc, and it too
> > reports a totalmemorymuch less than 400+ MB.
>
> A VPS server will never show the cache usage, since you cannot 'see'
> physicalmemory. The 'parent' server is doing all of the 'real' disk
> I/O and doing the caching. All you will see is you virtualmemory.
>
>
>
> > So, what is using all the 400+ MB ofmemory? What other diagnostics
> > tools can be helpful? Thanks in advance!
>
> ps andtopshould show you what is going on.
>
>
>
>
>
> > Regards,
> > Liu Chang
>
> > ================================================
>
> > [liuchang(a)vps lib]$ free -m
> > total used free shared buffers
> > cached
> > Mem: 512 411 100 0
> > 0 0
> > -/+ buffers/cache: 411 100
> > Swap: 0 0 0
>
> > =================================================
>
> >top- 07:44:00 up 50 min, 2 users, load average: 0.00, 0.02, 0.00
> > Tasks: 28 total, 1 running, 27 sleeping, 0 stopped, 0 zombie
> > Cpu(s): 0.0% us, 0.1% sy, 0.0% ni, 99.9% id, 0.0% wa, 0.0% hi,
> > 0.0% si
> > Mem: 524288k total, 422656k used, 101632k free, 0k
> > buffers
> > Swap: 0k total, 0k used, 0k free, 0k
> > cached
>
> > PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
> > 30313 liuchang 16 0 89632 35m 13m S 0 6.9 0:04.12 firefox-
> > bin
> > 30030 liuchang 15 0 87164 33m 13m S 0 6.5 0:03.12 firefox-
> > bin
>
> 89632+87164 = almost 173meg
>
> > 28453 liuchang 18 0 136m 20m 4904 S 0 4.0 0:01.21 java
> > 25924 mysql 16 0 121m 18m 4080 S 0 3.7 0:00.11 mysqld
>
> 136m+121m == 257meg
>
> 257+173 = 430meg
>
> So, between two firefox's, Java, and MySQL, there pretty much goes all
> of your virtualmemory. Why are you running firefox on a server? Why
> is firefox even installed on the server?
>
> MySQL seems to be a realmemoryhog: 121meg virtual! Compare to
> PostgreSQL (only 16meg virtual):
>
> postgres 21666 0.0 0.1 16760 1852 ? S Apr30 0:00 /usr/bin/postmaster -p 5432 -D /var/lib/pgsql/data -i -h 127.0.0.1
>
> (I just checked: starting MySQL cold grabs 121meg virtual!)
>
>
>
> > 28382 liuchang 15 0 11624 9020 1832 S 0 1.7 0:02.19 Xvnc
> > 28388 liuchang 16 0 5412 3204 2500 S 0 0.6 0:00.05 blackbox
> > 26040 root 15 0 6908 2212 1788 S 0 0.4 0:00.01 sshd
> > 29741 root 15 0 6908 2212 1788 S 0 0.4 0:00.01 sshd
> > 30022 liuchang 16 0 3436 2096 1632 S 0 0.4 0:00.02 gconfd-2
> > 25808 root 16 0 5856 1792 1364 S 0 0.3 0:00.00 cupsd
> > 28386 liuchang 16 0 3752 1536 1296 S 0 0.3 0:00.00 vncconfig
> > 25970 xfs 16 0 3252 1524 728 S 0 0.3 0:00.00 xfs
> > 29746 liuchang 16 0 6908 1508 1060 S 0 0.3 0:00.05 sshd
> > 26044 liuchang 15 0 6908 1488 1056 S 0 0.3 0:00.10 sshd
> > 26045 liuchang 16 0 2160 1320 1076 S 0 0.3 0:00.04 bash
> > 29747 liuchang 16 0 2160 1304 1064 S 0 0.2 0:00.01 bash
> > 30599 root 16 0 2164 1304 1048 S 0 0.2 0:00.00 bash
> > 25830 root 15 0 4028 1136 840 S 0 0.2 0:00.02 sshd
> > 25891 root 17 0 2152 1116 956 S 0 0.2 0:00.00
> > mysqld_safe
>
> Then you've got the whole rest of your desktop...
>
>
>
> > 30591 root 17 0 2304 1076 852 S 0 0.2 0:00.00 su
> > 26010 root 19 0 2724 1048 924 S 0 0.2 0:00.00 cups-
> > config-dae
> > 26035 root 18 0 3728 992 768 S 0 0.2 0:00.00 vsftpd
> > 30616 liuchang 16 0 1896 972 788 R 0 0.2 0:00.04top
> > 26000 dbus 16 0 2400 932 784 S 0 0.2 0:00.00 dbus-
> > daemon-1
> > 25948 root 16 0 2484 912 524 S 0 0.2 0:00.00 crond
> > 25990 root 22 0 4060 796 568 S 0 0.2 0:00.00 saslauthd
> > 1 root 16 0 1628 600 520 S 0 0.1 0:00.02 init
> > 25772 root 16 0 1532 524 440 S 0 0.1 0:00.00 syslogd
>
> > ==========================================
>
> > [root(a)vps bin]# ./ps_mem.py
> > Private + Shared = RAM used Program
>
> > 84.0 KiB + 440.0 KiB = 524.0 KiB syslogd
> > 80.0 KiB + 520.0 KiB = 600.0 KiB init
> > 672.0 KiB + 0.0 KiB = 672.0 KiB ps
> > 228.0 KiB + 568.0 KiB = 796.0 KiB saslauthd
> > 388.0 KiB + 524.0 KiB = 912.0 KiB crond
> > 148.0 KiB + 784.0 KiB = 932.0 KiB dbus-daemon-1
> > 224.0 KiB + 768.0 KiB = 992.0 KiB vsftpd
> > 124.0 KiB + 924.0 KiB = 1.0 MiB cups-config-dae
> > 224.0 KiB + 852.0 KiB = 1.1 MiB su
> > 160.0 KiB + 956.0 KiB = 1.1 MiB mysqld_safe
> > 796.0 KiB + 728.0 KiB = 1.5 MiB xfs
> > 240.0 KiB + 1.3 MiB = 1.5 MiB vncconfig
> > 428.0 KiB + 1.3 MiB = 1.8 MiB cupsd
> > 748.0 KiB + 1.1 MiB = 1.8 MiB bash (3)
> > 464.0 KiB + 1.6 MiB = 2.0 MiB gconfd-2
> > 1.1 MiB + 2.0 MiB = 3.1 MiB sshd (5)
> > 704.0 KiB + 2.4 MiB = 3.1 MiB blackbox
> > 7.0 MiB + 1.8 MiB = 8.8 MiB Xvnc
> > 14.8 MiB + 4.0 MiB = 18.8 MiB mysqld
> > 15.7 MiB + 4.8 MiB = 20.5 MiB java
> > 40.6 MiB + 14.0 MiB = 54.6 MiB firefox-bin (2)
>
> > Private + Shared = RAM used Program
>
> --
> Robert Heller -- Get the Deepwoods Software FireFox Toolbar!
> Deepwoods Software -- Linux Installation and Administrationhttp://www.deepsoft.com/ -- Web Hosting, with CGI and Database
> hel...(a)deepsoft.com -- Contract Programming: C/C++, Tcl/Tk

Thanks! I didn't really pay attention to the VIRT field of `ps`, the
MEM field apparently isn't the whole story. My understanding is
usually VIRT = MEM + SWAP, but in the VPS all the swaps are reported
to be zero, so why is there such a big difference between them?

Not that it matters, but the firefox is for long running automated
testing of dynamic websites. That's what this server is for :)

Cheers,
Liu Chang
 | 
Pages: 1
Prev: What to backup on a Linux system an what not?
Next: vi - hide some line