Posts Tagged ‘archive’

How to use zip command to archive directory and files in GNU / Linux

Monday, November 6th, 2017

how-to-use-zip-command-to-archive-directory-and-files-in-gnu-linux-and-freebsd

How to zip directory or files with ZIP command in LInux or any other Unix like OS?

Why would you want to ZIP files in Linux if you have already gzip and bzip archive algorithms? Well for historical reasons .ZIP is much supported across virtually all major operating systems like Unix, Linux, VMS, MSDOS, OS/2, Windows NT, Minix, Atari and Macintosh, FreeBSD, OpenBSD, NetBSD, Amiga and Acorn RISC and many other operating systems.

Assuming that zip command line tool is available across most GNU / Linux and WinZIP is available across almost all Windowses, the reason you might need to create .zip archive might be to just transfer the files from your Linux / FreeBSD desktop system or a friend with M$ Windows.

So below is how to archive recursively files inside a directory using zip command:
 

 $ zip -r myvacationpics.zip /home/your-directory/your-files-pictures-text/

 


or you can write it shorter with omitting .zip as by default zip command would create .zip files

 

$ zip -r whatever-zip-file-name /home/your-directory/your-files-pictures-text/

 


The -r tells zip to recurse into directories (e.g. archive all files and directories inside your-files-pictures-text/)

If you need to archive just a files recursively with a file extension such as .txt inside current directory

 

$ zip -R my-zip-archive.zip '*.txt'


Above command would archive any .txt found inside your current directory if the zip command is for example issued from /home/hipo all found files such as /home/hipo/directory1, /home/hipo/directory2, /home/hipo/directory2/directory3/directory4 and all the contained subdirs that contain any .txt extension files will be added to the archive.

For the Linux desktop users that are lazy and want to zip files without much typing take a look at PeaZip for Linux 7Z / ZIP GUI interface tool

 

Resume sftp / scp cancelled (interrupted) network transfer – Continue (large) partially downloaded files on Linux / Windows

Thursday, April 23rd, 2015

resume-sftp-scp-cancelled-interrupted-file-transfer-download-upload-network-transfer-continue-large-partially-downloaded-file-howto-linux-windows
I've recentely have a task to transfer some huge Application server long time stored data (about 70GB) of data after being archived between an old Linux host server and a new one to where the new Tomcat Application (Linux) server will be installed to fit the increased sites accessibility (server hardware overload).

The two systems are into a a paranoid DMZ network and does not have access between each other via SSH / FTP / FTPs and even no Web Access on port (80 or SSL – 443) between the two hosts, so in order to move the data I had to use a third HOP station Windows (server) which have a huge SAN network attached storage of 150 TB (as a Mapped drive I:/).

On the Windows HOP station which is giving me access via Citrix Receiver to the DMZ-ed network I'm using mobaxterm so I have the basic UNIX commands such as sftp / scp already existing on the Windows system via it.
Thus to transfer the Chronos Tomcat application stored files .tar.gz archived I've sftp-ed into the Linux host and used get command to retrieve it, e.g.:

 

sftp UserName@Linux-server.net
Password:
Connected to Linux-server.
sftp> get Chronos_Application_23_04_2015.tar.gz

….


The Secured DMZ Network seemed to have a network shaper limiting my get / Secured SCP download to be at 2.5MBytes / sec, thus the overall file transfer seemed to require a lot of time about 08:30 hours to complete. As it was the middle of day about 13:00 and my work day ends at 18:00 (this meant I would be able to keep the file retrieval session for a maximum of 5 hrs) and thus file transfer would cancel when I logout of the HOP station (after 18:00). However I've already left the file transfer to continue for 2hrs and thus about 23% of file were retrieved, thus I wondered whether SCP / SFTP Protocol file downloads could be resumed. I've checked thoroughfully all the options within sftp (interactive SCP client) and the scp command manual itself however none of it doesn't have a way to do a resume option. Then I thought for a while what I can use to continue the interrupted download and I remembered good old rsync (versatile remote and local file copying tool) which I often use to create customer backup stragies has the ability to resume partially downloaded files I wondered whether this partially downloaded file resume could be done only if file transfer was only initiated through rsync itself and luckily rsync is able to continue interrupted file transfers no matter what kind of HTTP / HTTPS / SCP / FTP program was used to start file retrievalrsync is able to continue cancelled / failed transfer due to network problems or user interaction activity), that turned even pretty easy to continue failed file transfer download from where it was interrupted I had to change to directory where file is located:
 

cd /path/to/interrupted_file/


and issue command:
 

rsync -av –partial username@Linux-server.net:/path/to/file .


the –partial option is the one that does the file resume trick, -a option stands for –archive and turns on the archive mode; equals -rlptgoD (no -H,-A,-X) arguments and -v option shows a file transfer percantage status line and an avarage estimated time for transfer to complete, an easier to remember rsync resume is like so:
 

rsync -avP username@Linux-server.net:/path/to/file .
Password:
receiving incremental file list
chronos_application_23_04_2015.tar.gz
  4364009472   8%    2.41MB/s    5:37:34

To continue a failed file upload with rsync (e.g. if you used sftp put command and the upload transfer failed or have been cancalled:
 

rsync -avP chronos_application_23_04_2015.tar.gz username@Linux-server.net:/path/where_to/upload


Of course for the rsync resume to work remote Linux system had installed rsync (package), if rsync was not available on remote system this would have not work, so before using this method make sure remote Linux / Windows server has rsync installed. There is an rsync port also for Windows so to resume large Giga or Terabyte file archive downloads easily between two Windows hosts use cwRsync.

Linux convert and read .mht (Microsoft html) file format. MHT format explained

Thursday, June 5th, 2014

linux-open-and-convert-mht-file-format-to-html-howto
If you're using Linux as a Desktop system sooner or later you will receive an email with instructions or an html page stored in .mht file format.
So what is mht? MHT is an webpage archive format (short for MIME HTML document). MHTML saves the Web page content and incorporates external resources, such as images, applets, Flash animations and so on, into HTML documents. Usually those .mht files were produced with Microsoft Internet Explorer – saving pages through:

File -> Save As (Save WebPage) dialog saves pages in .MHT.

To open those .mht files on Linux, where Firefox is available add the UNMHT FF Extension to browser. Besides allowing you to view MHT on Linux, whether some customer is requiring a copy of an HTML page in MHT, UNMHT allows you to also save complete web pages, including text and graphics, into a MHT file.
There is also support for Google Chrome browser for MHT opening and saving via a plugin called IETAB. But unfortunately IETAB is not supported in Linux.
Anyways IETAB is worthy to mention here as if your'e a Windows users and you want to browse pages compatible only with Internet Explorer, IETAB will emulates exactly IE by using IE rendering engine in Chrome  and supports Active X Controls. IETAB is a great extension for QA (web testers) using Windows for desktop who prefer to not use IE for security reasons. IETab supports IE6, IE7, IE8 and IE9.

Another way to convert .MHT content file into HTML is to use Linux KDE's mhttohtml tool.

linux-kde-converter-mhttohtml

Another approach to open .MHT files in Linux is to use Opera browser for Linux which has support for .MHT

Note that because MHT files could be storing potentially malicious content (like embedded Malware) it is always wise when opening MHT on Windows to assure you have scanned the file with Antivirus program. Often mails containing .MHT from unknown recipients are containing viruses or malware. Also links embedded into MHT file could easily expose you to spoof attacks. MHT files are encoded in combination of plain text MIMEs and BASE64 encoding scheme, MHT's mimetype is:

MIME type: message/rfc822
 

Archive Outlook mail in Outlook 2010 to free space in your mailbox

Thursday, May 15th, 2014

outlook-archive-old-mail-to-prevent-out-of-space-problems-outlook-logo
If you're working in a middle or big sized IT company or corporation like IBM or HP, you're already sucked into the Outlook "mail whirlwind of corporate world" and daily flooded with tons of corporate spam emails with fuzzy business random terms like taken from Corporate Bullshit Generator

Many corporations, because probably of historic reasons still provide employees with small sized mailboxes half a gigabyte, a gigabyte or even in those with bigger user Mailboxes like in Hewlett Packard, this is usually no more than 2 Gigabytes.

This creates a lot of issues in the long term because usually mail communication in Inbox, Sent Items, Drafts Conversation History, Junk Email and Outbox grows up quickly and for a year or a year and a half, available Mail space fills up and you stop receiving email communication from customers. This is usually not too big problem if your Mailbox gets filled when you're in the Office (in office hours). However it is quite unpleasent and makes very bad impression to customers when you're in a few weeks Summar Holiday with no access to your mailbox and your Mailbox free space  depletes, then you don't get any mail from the customer and all the time the customer starts receiving emails disrupting your personal or company image with bouncing messages saying the "INBOX" is full.

To prevent this worst case scenario it is always a good idea to archive old mail communication (Items) to free up space in Outlook 2010 mailbox.
Old Outlook Archived mail is (Saved) exported in .PST outlook data file format. Later exported Mail Content and Contacts could be easily (attached) from those .pst file to Outlook Express, leaving you possibility to still have access to your old archived mail keeping the content on your hard drive instead on the Outlook Exchange Mailserver (freeing up space from your Inbox).

Here is how to archive your Outlook mail Calendar and contacts:

Archive-outlook-mail-in-microsoft-outlook-2010-free-space-in-your-mailbox

1. Click on the "File" tab on the top horizontal bar.Select "Cleanup Tools" from the options.

2. Click "Cleanup Tools" from the options.

3. Click on the "Archive this folder and all subfolders" option.

4. Select what to archive (e.g. Inbox, Drafts, Sent Items, Calendar whatever …)

5. Choose archive items older than (this is quite self-explanatory)

6. Select the location of your archive file (make sure you palce the .PST file into directory you will not forget later)

That's all now you have old mails freed up from Outlook Exchange server. Now make sure you create regular backups ot old-archived-mail.pst file you just created, it is a very good idea to upload this folder to encrypted file system on USB stick or use something like TrueCrypt to encrypt the file and store it to external hard drive, if you already don't have a complete backup corporate solution backuping up all your Laptop content.

Later Attaching or detaching exported .PST file in Outlook is done from:

File -> Open -> Open Outlook Data File

outlook-open-backupped-pst-datafile-archive-importing-to-outlook-2010


Once .PST file is opened and attached in Left Inbox pane you will have the Archived old mail folder appear.

 

outlook-archived-mail-pannel-screenshot-windows-7
You can change Archived name (like I did to some meaningful name) like I've change it to Archives-2013 by right clicking on it (Data File properties -> Advanced)

how to archive with Windows default zip (compress) algorithm

Tuesday, April 1st, 2014

windows-sent-to-compressed-file
I'm working on a decomissioning project (for those who hear decomissioning for a first time – in corporate world this means removal of service/s or assigned resources of a server or a physical server hardware that is not to be used in future or is obsolete). The decomissioning includes removal of Apache Tomcat (Software Configuration Item CI – in HP terms) from Microsoft Windows 2007 – Service Pack 2.
Part of decomissioning is of course to create backup of Tomcat Application server and for that I needed to create compressed archive of Tomcat instances. Usually I do archives on Windows using Winrar or Winzip but this time as the server productive server has the minimum installed there was no any external vendor produced archiving software available.

My memories from past were that there is a native compressing program embedded into Windows as I've unzipped compressed archives on Win hosts with no need for external WinZip. However until so far I never did .ZIP archive with no available external uncompress software.

Using Winzip or Winrar so far to make archive from a number of files I used to select files to enter Archive press right mouse button and select Create Archive (Add To Archive) so I expected this will work whenever no Winrar, however there was no obvious button like this, so I googled a bit to find out how is that possible ending up on Article from Microsoft titled "Compress and uncompress files (zip files)", there is a dumb proof video teaching Compressing files with Microsoft default ZIP program is done by the the weird "Send To" menu 🙂 

Selecting files to enter Archive;
> (Click Right Mouse Button) -> (Send To Compressed Zipped Folder)

compress_zipped_folder_with_windows_default_archive_algorithm how to archive with windows default compress archive

Honestly If I didn't checked the net probably I will never think of looking it there.

My PHP/ MySQL Restaurant Reservation Form

Tuesday, December 14th, 2010

Two months ago, I’ve developed a contact reservation form in PHP. The form is really easily customizable and is straight forward to integrate.
I’ve developed the form for a small restaurant which was missing the feature on it’s joomla based website in order to be able for restaurant clients to reserve tables.

Here is how my restaurant reservation form looks like:

hip0's plain php restaurant reservation form

Later on I found there are plenty of possibilities to easily make a reservation form in Joomla but at that time I had no idea that custom contact forms can be prepared with Joomla, so I developed my own one from scratch in plain PHP and MySQL.

The form’s fields are in Dutch, because the form I’ve developed for a Dutch restaurant.
However changing the form text is really easy,to do so open the php file and modify it, according to your needs.
I decided to share here the reservation form in hope that it might be helpful to somebody out there.
The Reservation form is licensed under GPLv2 so you’re very welcome to distribute and modify it freely.

The form installation is described in the README file you will find bundled with the reservation form archive.

You can d Download my PHP restaurant reservation form here

Feedback on the form is very welcome!
 

Create Routine mirror copy of milw0rm & packetstorm exploits database

Wednesday, January 13th, 2010

Few weeks ago, I’ve built a small script and put it to
execute in cron in order to have an up2date local copy of
milw0rm. Ofcourse that’s pretty handy for several reasons.
For example milw0rm may go down and the exploit database tobe lost forever. This once happened with hack.co.za which ceasedto exist several years ago, even though it has one of thegreatest exploits database for it’s time.
Luckily I did a copy of hack.co.za, knowing that it’s gone
day might come here is the mirror archive of hack.co.za database
Anyways back to the main topic which was creating routine mirror
copy of milw0rm exploits database.
Here is the small script that needs to be setup on cron in order tohave periodic copy of milw0rm exploits database.
#!/usr/local/bin/bash# Download milw0rm exploitsdownload_to='/home/hipo/exploits';milw0rm_archive_url='http://milw0rm.com/sploits/milw0rm.tar.bz2';milw0rm_archive_name='milw0rm.tar.bz2';if [ ! -d '/home/hipo/exploits' ]; thenmkdir -p $download_to;ficd $download_to;wget -q $milw0rm_archive_url;tar -jxvvf $milw0rm_archive_name;rm -f $milw0rm_archive_name;exit 0The script is available as wellfor download via milw0rm_exploits_download.sh
To make the script operational I’ve set it up to execute via cron with
the following cron record:
00 1 * * * /path_to_script/milw0rm_exploits_download.sh >/dev/null 2>&1Here is another shell code I used to download all packetstormsecurity exploits frompacketstormsecurity’s website:
#!/usr/local/bin/bash# Download packetstormsecurity exploits# uses jot in order to run in freebsdpacketstorm_download_dir='/home/hipo/exploits';if [ ! -d "$packetstorm_download" ]; thenmkdir -p "$packetstorm_download";for i in $(jot 12); do wget http://www.packetstormsecurity.org/0"$i"11-exploits/0"$i"11-exploits.tgz; done
The script can be obtained also via following link (packetstormsecurity_expl_db_download.sh)

Another interesting tutorial that relates to the topic of building local
mirrors (local exploit database) is an article I found on darkc0de.com’s
website calledHow to build a local exploit database
The article explains thoroughly
howto prepare packetstormsecurity exploits database mirrorand
how to mirror milw0rm through python scripts.
Herein I include links to the 2 mirror scripts:
PacketStorm Security Mirror Script
milw0rm archive mirror script
Basicly the milw0rm archive script is identical to the small shellscript
I’ve written and posted above in the article. However I believe there is
one advantage of the shellscript it doesn’t require you
to have python installed 🙂

Run Apache with SSL Self Signed SSL Certificate

Friday, August 14th, 2009

Recently I had to run apache on Debian 4.0 (Edge) with Self Signed certificate.To make it happen I had to Google around and try out stuff. I've red that Debiancomes with a command (apache2-ssl-certificate) that generates a self signed openssl certificate.However on my Debian systems this cmd wasn't available. So I had to google around about it,and I came along the following website which provided mewith the script itself and some instructions how to use it. I've modified a bit the archive mentionedon the above website to make the install instructions of the website through a script. I've built a newarchive based on the archive apache2-ssl.tar.gz that includes an extra file runme.sh which does the properinstallation for you. The new archive itself could be found here .

In the mean time I recommend you read my article explaining how to quickly and efficiently generate self-signed certificate with openssl command on GNU / Linux and BSD

END—–