How to mount directory in memory on GNU / Linux
and FreeBSD / Mount directory in RAM memory to increase performance
on Linux and BSD
One of the websites hosted on a server I currently manage has a
cache directory which doesn't take much space but tens of thousands
of tiny files. Each second a dozen of files are created in the
cache dir. Hence using a hard disk directory puts some serious load
on the server consequence of the many
fopen and
fclose HDD I/O operations.
To get through the problem, the solution was obvious use a
directory which stores its information in memory.
There are of course other benefits of using a memory to store data
in as we all know as access to RAM is so many times faster.
GNU/Linux is equipped with a
tmpfs since
kernel version
2.4.x, primary usage of
tmpfs file system across many G
/ Linux distributious is the
/tmp directory.
Some general
tmpfs information about tmpfs is explained in
mount's manual e.g.:
man mount, a good other reading is the
tmpfs kernel
documentation file
An implementation of
tmpfs is
/dev/shm .
/dev/shm is a standard memory device used among Linuces, its
actually an ordinary directory one can list with
ls .
/dev/shm is used as a "virtual directory" memory space.
Below is an output of
/dev/shm from my notebook, one can see
few files stored in memory which belong to the
pulse audio
linux architecture:
linux:~$ ls -al /dev/shm
ls -al /dev/shm/ total 7608 drwxrwxrwt 2 root root 160 Oct 10 18:05
.
drwxr-xr-x 16 root root 3500 Oct 10 10:57 ..
p-w------- 1 root root 0 Oct 10 10:57 acpi_fakekey
-r-------- 1 hipo hipo 67108904 Oct 10 17:20
pulse-shm-2067018443
-r-------- 1 hipo hipo 67108904 Oct 10 10:59
pulse-shm-2840042043
-r-------- 1 hipo hipo 67108904 Oct 10 10:59
pulse-shm-3215031142
-r-------- 1 hipo hipo 67108904 Oct 10 18:05
pulse-shm-4157723670
-r-------- 1 hipo hipo 67108904 Oct 10 18:06
pulse-shm-702872358
To measure the size of
/dev/shm across different Linux
distriubtions one can use the usual
df cmd, e.g.:
[root@centos: ~]$ df -h /dev/shm
Filesystem Size Used Avail Use% Mounted on
tmpfs 16G 0 16G 0% /dev/shm
Above I show a
df -h /dev/shm output from a CentOS server
equipped with 32 GB of memory, as you can see CentOS has reserved
half of the size of the system memory (16GB) for the purposes of
creating files in memory through /dev/shm. The memory is
dynamically assigned, so if its not use the assigned memory by it
can still be used for the purposes of the services running (which
by the way is very nice).
Accoring to what, I've read in
wikipedia about tmpfs,
tmpfs defaults in Linux to half of the system physical
memory.
However I've noticed Debian Linux hosts usually reserve less memory
for /dev/shm, on my personal notebook Debian
/dev/shm is
only 1 Giga, where on a Debian running server, Debian automatically
has set it to the humble 2GB. Setting less by the way as with the
Debian example, is a rather good idea since not many desktop or
server applications are written to get actively advantage of the
virtual /dev/shm directory.
One can directly drop files in
/dev/shm which will
immediately be stored in memory, and after a system reboot the
files will disappear.
Let's say you zip archive file,
testing.zip and you like to
store the file in memory to do so, just copy the file in
/dev/shm.
linux:~$ cp -rpf testing.zip /dev/shm
You don't even need to be root to copy files in the "virtual memory
directory". This is a reason many crackers (script kiddies), are
storing their cracking tools in
/dev/shm ;)
A rather funny scenario, I've witness before is when you list
/dev/shm on some Linux server and suddenly you see a tons of brute
forcing tools and all kind of "hack" related stuff belonging to
some system user. Sometimes even this malicious script tools belong
to the root user...
Now as I've said a few words on how linux's tmpfs works here is how
to mount a directory which cache content will be stored in volatile
memory:
linux:~# mount -t tmpfs -o size=3G,mode=0755 tmpfs
/var/www/site/cache
As you can see the above command will dynamically assign a tmpfs
directory taking up from the system RAM mem which could expand up
to 3GB within the system memory.
Of course before mounting, its necessery to create the
/var/www/site/cache and set proper permissions in the above
example I use
/var/www/site/cache with a default permissions
of 755 which is owned by the use with which the Apache server is
running, e.g.:
linux:~# mkdir -p /var/www/site/cache
linux:~# chown -R www-data:www-data /var/www/site/cache
linux:~# chmod -R 755 /var/www/site/cache
Using a tmpfs is very handy and have many advantages, however one
should be very careful with the data stored inside a tmpfs dir
resource, all data will be lost in case of sudden system restart as
the data is stored in the memory.
One other problems one might expect with
tmpfs would be if
the assigned virtual disk space gets filled with data. It never
happened to me but, I've red online some stories that in the past
this led to system crashes, today as the dox I've checked
prescribed overfilling it will start swapping make the system
terribly sluggish and eventually afred depleting the reserver swap
space will start killing processes.
Using
tmpfs as a cache directory is very useful on servers
running Apache+PHP/Perl/Python/Ruby etc. as it can be used for
stroring script generated temorary data.
Using a
tmpfs can signifantly decrease server i/o created
disk overheads.
Some other application I can think of though, I haven't tested it
would be if
tmpfs mounted directory is used to store
scripting executable files, copied after restart. Executing the
script reading it directly from the "virtual directory" could for
sure have very good impact especially on huge websites.
One common service which takes advantage of the elegancy of
tmpfs nowdays almost all modern GNU/Linux has is
udevd - The Linux dynamic device management. By the way (man
udev) is a very good and
must read manual especially for
Linux novices to get a good basic idea on how /dev/ mamagement
occurs via udev.
To
make permenant directory contained in memory on Linux the
/etc/fstab file should be used.
In order to mount permanently a directory as a memory device of a
size of 3GB with 0755 permissions in /var/www/site/cache, as shown
in the earlier example, one can use the command:
linux:~# echo 'tmpfs /var/www/site/cache/ tmpfs
size=3G,mode=0755 0 0' >> /etc/fstab
This will assure the directory stored in memory would be recreated
on next boot.
Nowdays the use of
tmpfs is constantly growing, I've seen it
to be used as a way to substitute ordinary disk based
/tmp
with a tmpfs directory contained in memory in
Cloud Linux
OS.
The applications of tmpfs is pretty much to the imagination of the
one who wants to get advantage of it. For sure using tmpfs will be
seen by the Linux GUI programs.
Going to FreeBSD and the BSD world,
tmpfs is also available,
however it is still considered a bit experimental. To get use of
tmpfs to gain some performance, one should first enable it
via bsd's /etc/rc.conf:
freebsd# echo 'tmpfs_load="YES"' >>
/etc/rc.conf
Mounting a directory permanently using tmpfs persmanently it again
is doable via
/etc/fstab to add a new directory inside
memory with tmpfs: is done with adding:
freebsd# echo 'tmpfs /var/www/site/cache tmpfs rw 0 0'
>> /etc/fstab
The native equivallent of
tmpfs in FreeBSD is called
mdmfs.
As I said it is slower than
tmpfs but rock solid.
To mount a 4gigabyte size mdmfs "ram directory" on BSD from
csh:
freebsd# mdmfs -s 4g md
/var/www/site/cache
Mounting a directory permanently using tmpfs persmanently it again
is doable via
/etc/fstab to add a new directory inside
memory with tmpfs: is done with adding:
freebsd# echo 'tmpfs /var/www/site/cache tmpfs rw 0 0'
>> /etc/fstab
The native equivallent of
tmpfs in FreeBSD is called
mdmfs.
As I said it is slower than
tmpfs but rock solid.
To mount a 4gigabyte size mdmfs "ram directory" on BSD from
csh:
freebsd# mdmfs -s 4g md
/var/www/site/cache
Mounting a directory permanently using tmpfs persmanently it again
is doable via
/etc/fstab to add a new directory inside
memory with tmpfs: is done with adding:
freebsd# echo 'md /var/www/site/cache mfs rw,-s4G 2 0'
>> /etc/fstab
There are some reports of users
who presumable use it to increase the ports / kernel compile times,
but I haven't tried it yet so I don't know for sure
In huge corporations like Google and Yahoo tmpfs is certanly used a
lot as this technology can dramatically can improve access times to
information. I'm curious to know for some good ways to get use of
tmpfs to improve efficiency.
If someone has some readings or has some idea please shar with me
;)