How to calculate memory used/free in Linux

Formula for used% in linux

free -o
             total       used       free     shared    buffers     cached
Mem:       8167004    8115292      51712          0      87680    1444676
Swap:      8388584    3252996    5135588

((memUsed-mebBuffers-memCached)*100)/memTotal => totalMemoryUsed%

((8115292-87680-1444676)*100/8167004) => 81%

 Display free memory in Linux / Ubuntu
Short Answer:
free -m | awk 'NR==3 {print $4 " MB"}'

The command 'free' displays some formatted information from /proc/meminfo.
The throw '-m' displays these numbers in rounded megabytes.


# free -m
         total      used     free   shared     buffers     cached
Mem:      4049      3982       67        0          16       3530
-/+ buffers/cache:   435     3614
Swap:     6142        53     6088

It's strange, cause I always figured the "first row, third column" - right under 'FREE' - would be the amount of free memory on a Linux system. But that number was always low after I'd been running awhile.


And it turns out that it is NOT really what I was looking for. A more accurate representation of the memory being used by your applications and available for new processes is displayed in the SECOND line.

In addition to the memory that is actually being USED by the kernel and processes resident in memory - Linux also reserves memory to allocate to processes as 'buffers' AND uses pretty much any left over memory to hold "cached" files.

Looking only at the top line...
total = all memory in the system (4GB on this server)
used = all memory currently in use/reserved by running processes and the OS
free = total - used
shared = memory being shared by multiple processes (deprecated?)
buffers = memory reserved by the OS to alloc as buffers when process need them (aka the 'heap')
cached = recently used files being stored in ram (THANK YOU LINUX!)

Here's a simple example I found to show off the power of 'caching':

for i in 1 2 ; do free -o; time grep -r foo /usr/bin >/dev/null 2>/dev/null; done

So really the buffers would be allocated to a running process if it asked for them anyway, and the memory being used to cache copies of recently used files would be released immediately if it makes sense to allocate the RAM elsewhere. So all that memory is 'available'.

Using these definitions:

When thinking about 'how much memory is really being used' - I want to calculate:
'used' - ('buffers' + 'cached')

When thinking about 'how much memory is really free' - I want to calculate:
'free' + ('buffers' + 'cached')

With this in mind, the meaning of the second row header form the output of the Linux command "free" (-/+ buffers/cache:) makes more sense...

Free is doing some light lifting for us, using the formula's above to display:
"minus buffers and cache" for the used column
and
"plus buffers and cache" for the free colum

So when you run free on Linux - the amount of free memory is always displayed right there in the second row, third column. Hence the 'Short Answer'...

Using Stunnel to Encrypt Unsecure Connections

So you’re a fully signed up ‘tin foil hatter’ and you have an old app that communicates over the internet but doesn’t use SSL?
Not to worry, there is a solution for you and its called stunnel.
Stunnel is available for both Linux and Windows, and simply put creates an SSL tunnel from one machine or server to another.
So why use stunnel? Well stunnel…
… provides an extra layer of security to an internet based application.
… secures data previously transmitted in plain text.
… protects your exploitable services by hiding them behind stunnel.
So what else can it be used for? Well…
… you could use it to fool certain websites into thinking you are in a different location.
… it can be used as a simple ‘bouncer’ for pretty much anything!

In my example I wanted to provide a little security to my simple ‘Munin’ monitoring solution (I’ll write something about that at a later date). Munin checks the servers status by ‘telnet’ to the munin-node running on the target server/pc, so if you are monitoring a remote server you have data about your system flying about everywhere! Also it means that there is a naked telnet port open on your server, which although you can lock it down by only allowing connections from a specific IP etc. doesn’t work to well if your on a dynamic IP and still leaves unencrypted data flying about the ether. Ok so in this case security isn’t that vital and none of the data is particularly sensitive. But if it uses TCP and you want to provide an extra layer of protection stunnel can do it.
As an added bonus stunnel is extremely easy to setup.
The rest of this post is based around a CentOS installation. But its pretty much the same on Debian/Ubuntu… just replace ‘yum’ with ‘apt-get’. The Windows install is even easier and the configs are in the same format as the Linux confs so you can easily adapt the instructions bellow!
Firstly lets install stunnel on the client machine (In the case of my example this is actually my ‘server’ machine in Munin, but for example if you wanted to secure SMTP communications this would be the machine sending the emails).
Although in this post I’ve used Munin as an example it really is easy to modify these instructions to fit any service. Simply change the ports!

Install using Yum:
yum install stunnel
(Now I cant remember exactly but I think stunnel is not on the default CentOS repos… so you can add the RPMforge repos like I tend to do: http://www.rpmrepo.org/RPMforge/Using)
Install from source:
wget http://mirror.hudecof.net/stunnel/stunnel-4.22.tar.gz tar zxf stunnel-4.22.tar.gz cd stunnel-4.22 ./configure make make install

Configuration:
On CentOS the default location for the stunnel.conf is /etc/stunnel/ so open this file in your editor of choice:
vi /etc/stunnel/stunnel.conf

Lets set the following options:
chroot = /var/run/stunnel setuid = nobody setgid = nobody pid = /stunnel.pid client = yes

And create a service:
[munin] accept = 127.0.0.1:4948 connect = myserver.hostname.com:4948
Now as you can see here I’ve set the listener to 127.0.0.1 but you could set this to all interfaces or a specific one by omitting or replacing the 127.0.0.1: The ‘connect’ setting is the servers hostname or IP address and the port that the stunnel ‘server’ is listening on. Another example of using stunnel could be to direct all web requests onto another server using a secure layer:
[www] accept = 80 connect = myserver.hotname.com:8080
You would then setup the ‘server’ stunnel to listen on 8080 and connect to the local (or even a remote!) web server.

Ok thats it for the client side for now. Lets look at the server:
Install stunnel as per the installation instructions above.

On CentOS the default location for the stunnel.conf is /etc/stunnel/ so open this file in your editor of choice:
vi /etc/stunnel/stunnel.conf

Set the following options:
cert = /etc/stunnel/stunnel.pem chroot = /var/run/stunnel setuid = nobody setgid = nobody pid = /stunnel.pid
And create a service:
[munin] accept = 4948 connect = 127.0.0.1:4949
Now you can see here I have set the connect to 127.0.0.1… this is because I can now set the vulnerable service to only bind to a local port rather than a external accessible IP of my server.

Or to continue from our secondary www example:
[www] accept = 8080 connect = myserver.hostname.com:80
With a bit of messing around with DNS or hosts files we could use stunnel to bounce our connection to any server for example www.bbc.co.uk! But that’s not really what we’re covering in this article and I’m guessing if you have your reasons for doing something like that you can figure it out on your own.
Ok we’re nearly ready to start stunnel. But the observant amongst you will have noticed that stunnel.pem certificate file that we set in the server options doesn’t exist! So lets create one now!
openssl req -new -x509 -days 3650 -nodes -out /etc/stunnel/stunnel.pem -keyout /etc/stunnel/stunnel.pem

Right now lets start stunnel, on both machines simply run the following command:
/usr/sbin/stunnel /etc/stunnel/stunnel.conf
Now lets test it! If the service we’ve setup can be talked to with telnet (eg. Munin or SMTP) then we can test this very simply from the client machine:
telnet localhost:4948
You should get the following back:
Trying 127.0.0.1…
Connected to localhost (127.0.0.1).
Escape character is ‘^]’.
# munin node at mysever.hostname.com
(Now its a little confusing because ‘Connected to localhost’ is actually the response from the munin-node on the remote server!)
As you can see your telnet session has gone into stunnel locally, been transmitted from stunnel on the local machine to stunnel on the server and then from stunnel into the Munin node on the server! Magic!

Stunnel startup script:
Now you can either start stunnel every time your machine starts up manually, add it to the crontab (if you try and start stunnel again and its already running the second instance will just close, but it leaves a mess in your /var/log/secure so don’t do it to often) or use a simple startup script like this one I ( used ) to use:
#!/bin/bash
    if [ -f /var/run/stunnel/stunnel.pid ]; then
        ps aux |grep -v grep |grep $pid |grep stunnel > /dev/null
        if [ $? = 0 ]; then
            echo “Server is already running !!”
        else
            echo “Pid file exists but process not found … trying to start stunnel”
            /usr/sbin/stunnel /etc/stunnel/stunnel.conf
        fi
        rm -f /tmp/stunnelrun > /dev/null
    else
        echo “Pid file not found. Starting stunnel.”
        /usr/sbin/stunnel /etc/stunnel/stunnel.conf
    fi
This is a very simple script I knocked up in a few mins. I later replaced it with an init script that I wrote when I had a little more time. But I’m going to post that as another article as I have a bit more to say about that (including some adaptationto make general init scripts for various programs)

To use the above script to start stunnel do the following:
cd /usr/local/sbin
vi stunnel-run
Paste in the above code.
Save and exit.
chmod +x stunnel-run

Test by doing the following:
./stunnel-run

Now add it to the crontab:
crontab -e

Insert the following line:
*/15 * * * * /usr/local/sbin/stunnel-run 2>&1 > /dev/null
Now your system will run the stunnel-run script every 15mins, checking if stunnel is running and starting stunnel if it is not running. You could simply start stunnel every 15mins as it will exit if it finds it can not use the ports its been assigned, but that leaves a mess in your secure log.
Some of you I’m sure are thinking ‘why use stunnel you could use a vpn or xyz’. Well yes, there are many ways of skinning this cat, stunnel is a simple and versatile one but not the ultimate jack of all trades. Use it where you see fit!