How to Split files on Linux FreeBSD, NetBSD and
OpenBSD
Did you have the need to sometimes split an SQL extra large files
to few pieces in order to be able to later upload it via
phpmyadmin?
Did you needed an extra large video or data file to be cut in few
pieces in order to transfer it in few pieces over an USB
stick?
Or just to give you an another scenario where I sometimes need to
have an enormous file let's say 3G split in few pieces, in order to
later read it in
vim or
mcedit .
I sometimes need to achieve this on FreeBSD and Linux hosts thus I
thought it will be helpful to somebody to give a very quick
tutorial on the way large files can be cut in pieces on Linux and
BSD hosts.
GNU/Linux and FreeBSD are equipped with the
split command.
The purpose of this command is exactly the cutting of a file to a
number of pieces.
On Linux the
split command comes by default install to the
system with the
coreutils package on most Debian (deb) based
and Redhat based (rpm) distributions, theerefore Linux's version of
split is GNU/split since it's part of the GNU Coreutils package. An
interesting fact about Linux split is that one of the two
programmers who has coded it is Richard Stallman ;)
On BSD Unix split is the
AT&T UNIX (BSD) split
In the past splitting files in pieces was much more needed than
today, as people used floppy drives to transfer data, though today
with the bloom of Internet and the improve of the data carriers
transferring even an extra large files from one place to another is
a way more trivial task still at many occasions splitting it in
pieces is needed.
Even though today splitting file is very rarely required, still
there are times when being able to split a file in
X number
of parts is very much needed.
Maybe the most common use of splitting a file today is necessery
when a large SQL file dumps, like let's say 200 MBytes of info
database needs to be moved from ane hosting provider to another
one.
Many hosting providers does disallow direct access with standard
mySQL client programs to the database directly and only allow a
user to connect only via
phpMyAdmin or some other web
interface like Cpanel to improve data into the SQL or PostgreSQL
server.
In such times, having knowledge on the Unix
split command is
a priceless asset.
Even though on Linux and BSD the code for the split command is not
identical and GNU/split and BSD/split has some basic differences,
the use of
split on both of these Unices is identical.
The way to split a file in few pieces using on both Linux and BSD
OSes is being done with one and the same command, here is
how:
1. Splitting file in size of 40 mb On Linux
linux:~# split -b 40m SQL-Backup-Data.sql
SQL-Backup-Data_split
2. Splitting file in size of 40mb on BSD (FreeBSD, OpenBSD,
NetBSD)
freebsd# split -b 40m SQL-Backup-Data.sql
SQL-Backup-Data_split
The Second argument the split command takes is actually called a
prefix, the prefix is used as a basis name for the creation of the
newly generated files cut in pieces file based on
SQL-Backup-Data.sql.
As I said identical command will split the
SQL-Backup-Data.sql files in a couple of parts which of it
will be sized 40 megas.
These command will generate few files output like:
freebsd# ls -1 SQL-Backup-Dat* SQL-Backup-Data.sql
SQL-Backup-Dataa
SQL-Backup-Dataab
SQL-Backup-Dataac
SQL-Backup-Dataad
SQL-Backup-Dataae
As you see the
SQL-Backup-Data.sql with size 200MB is being
split in four files each of which is sized 40mbytes.
After the files are transfered to another Linux or BSD host, they
can easily be again united in the original file with the
command:
linux:~# for i in $(ls -1 SQL-Backup-Data_split*); echo $i
>> SQL-Backup-Data.sql
Alternatively in most Unices also using cat should be enough to
collect back the pieces into the original file, like so:
freebsd# cat SQL-Backup-Data_split* >>
SQL-Backup-Data.sql
Enjoy splitting