Reboot Safely a frozen (hanged up) Linux Server
with Magic SysRQ Key combination
Being a remote system administrator for many years from now, I've
experienced many times sudden Linux crashes, mostly caused by
system hardware overloads.
This is a common thing especially on busy Web Servers with
(Apache/MySQL). Situations like this are really messy and many
times in order to bringt the server back online, one has to either
physically go to the Data Center or contact the Technical support
to request a server reboot.
In this terrible times, data loss might occur if the Server reset
button is used to cold reboot it.
Happily in 99% of the cases the data loss which might occur could
be prevented with Linux's kernel capability to support the
Magic
SysRQ key !
Having the
Magic SysRQ key functionality being supported in
Linux in past times wasn't that common, thanksfully these days this
has changed andlmost every Linux distrubution supports this handy
feature.
Now you might wonder what is the magic with those
Magic SysRQ
key ?
Let me explain,
Magic SysRQ is a kernel level functionality
which supports even completely crashed Linux systems with the
horrifying:
Kernel Panic
message to be properly shutdown.
Using
Magic SysRQ instead of the mostly used indiced cold
reboots is really advantageous, as all the opened files by programs
on the crashed server which hanged will be properly saved and
closed and thus possible data loss caused by the sudden server
crash will be minimized.
One other good thing about the
Magic SysRQ Key keyboard
combination is that initiating the
Magic SysRQ could be made
blindly (no need for server monitor or display to present any error
messages on the failed server).
Now to check if the
magic sysrq is enabled on a server one
should issue the command:
Here are some Magic SysRQ keyboard combinations one can use in case
of server failure:
ALT+SYSRQ+M to dump memory info;
ALT+SYSRQ+P to dump processes states;
ALT+SYSRQ+S to sync disks;
ALT+SYSRQ+U to unmount all mounted filesystems;
ALT+SYSRQ+E to terminate processes;
ALT+SYSRQ+I to kill all processes
ALT+SYSRQ+U to try to unmount once again;
ALT+SYSRQ+B to reboot.
I would skip to explain what each of the keyboard combinations will
do as I believe the above description explains it well.
One classics of combinations one might want to issue on a failed
Linux server supporting the
Magic SysRQ would be:
ALT+SYSRQ+R
ALT+SYSRQ+E
ALT+SYSRQ+I
ALT+SYSRQ+S
ALT+SYSRQ+U
ALT+SYSRQ+B
The
ALT+SYSRQ+REISUB key combination is very popular among
system administrators.
Also
Magic SysRQ is heavily used by kernel developers as it
includes many handy debugging options.
If you try the
Magic SysRQ key on older servers and you
realize it doesn't react you will have to recompile the linux
kernel and enable the
CONFIG_MAGIC_SYSRQ kernel time
compilation option.
The
Magic SysRQ can also be initiated remotely on a properly
running server
by initiating:
server:~# echo b > /proc/sysrq-trigger
This command would be useful if you want to just have fun and test
that
magic sysrq works on your system for sure ;)
To sum it up using the
Magic SysRQ will guarantee your
locked up, server a
Safe Reboot and will hopefully save you
a lot of time for backu ps recovery.