Posts Tagged ‘Deleting’

Run 2 and more Skypes simultaneously on Mac OS X – Run multiple Skype acccounts on same Mac

Saturday, June 21st, 2014

run-2-and-more-skypes-simultaneously-on-mac-os-x-multiple-skype-account-login-on-mac
For people running Mac OS X, the question of 
how is it possible to use 2 skype accounts in parallel on Mac probably makes good sense?

I don't own a Mac notebook and thefore I'm a Mac newbie, however, I'm into situation where I and my wife Svetlana went (for 3 days) to my hometown Dobrich and we have with us only her Mac OS X powered Mac Book air.

 

One user is already logged in Skype, (my wife) is expecting some relatives and friends to contact us and  same time I had to login to check few servers via ssh and discuss some server downtime issues from yesterday in Skype .
Thus we
need 2 skype instances to run separately on her Macbook air powered PC with Mac OS X Leopard
 

Earlier I've blogged how to make 2 and more Skype accounts work simultaneously on one Windows PC because I had to set it up for a company, in this short article I will explain how is possible to run many skype clients on Mac OS X.

 

1. Open Mac Terminal from Finder

finder-terminal-screenshot-mac-os-x-leopard-run-many-skypes-mac-os

2. In Terminal run the first Skype Instance

Type in Terminal:

open /Applications/Skype.app/Contents/MacOS/Skype

3. Run Second Skype instance

In older Skype Mac OS versions, I read the

/secondary

Skype command option was there and could be used to run a second parallel skype instance on Mac, however in newer releases this option was removed and if you try to invoke it warning window pops up saying an instance is already running.

mac-os-x-you-have-another-copy-of-skype-running-screenshot

To get around the issue and run the second Skype, quickest way is to run another Skype client under privileged user through sudo command (this is unsecure – but anyways as Mac OS is proprietary and we don't have access to code and probably there are tons of spy and report software integrated into the OS, it doesn't really matter.)

mac-os-x-skype-run-screenshot-pic

To get around the issue and run the second Skype, quickest way is to run another Skype client under privileged user through sudo command (this is unsecure – but anyways as Mac OS is proprietary and we don't have access to code and probably there are tons of spy and report software integrated into the OS, it doesn't really matter.)

4. Script it into 2nd_skype.sh for later use

To run and use two parallel skypes regularly it might be useful to make shell script out of it and place it somewhere, 2nd_skype.sh script should be something like:


#!/bin/bash
open /Applications/Skype.app/Contents/MacOS/Skype
sudo /Applications/Skype.app/Contents/MacOS/Skype

Then make the script executable with:

chmod a+x 2nd_skype.sh

5. Run more than 2 Skypes (Run multiple Skypes on same Mac PC hack)

There is another "hack" method with deleting the Skype.pid (Process ID). Skype recognize where it is running by checking its Skype.pid on start up.

Deleting the pid after each next Skype client launch,  allow the user to run as many Skypes as you want on Mac OS X but it is not clear for how long it time it will work.

rm -f ~/Library/Application Support/Skype/Skype.pid

Then launch again Skype in background from Mac Terminal

open -nW '/Application/Skype.app' &

In case if you wonder why the open command is used, since above line could be run also directly and Skype will pop-up, by using open command you instruct the program to detach itself from Terminal from which it run, so later if Terminal is closed Skype app. will not terminate.

Another approach is to create, a many users lets say 5 users and use the Skype sudo run method each client with a separate user.

sudo user1 /Applications/Skype.app/Contents/MacOS/Skype
sudo user2 /Applications/Skype.app/Contents/MacOS/Skype
sudo user3 /Applications/Skype.app/Contents/MacOS/Skype
sudo user4 /Applications/Skype.app/Contents/MacOS/Skype

sudo user5 /Applications/Skype.app/Contents/MacOS/Skype

I enclose the script with the custom icon (Skype) ready to be launched and Voila, on script launch Skype multiple login prompts pops up.

For the lazy ones who don't want to tamper with writting scripts or doing hacks to run Skype multiple times on Mac there is even a Multi Skype Launcher app for Mac.

 


 

How to delete million of files on busy Linux servers (Work out Argument list too long)

Tuesday, March 20th, 2012

How to Delete million or many thousands of files in the same directory on GNU / Linux and FreeBSD

If you try to delete more than 131072 of files on Linux with rm -f *, where the files are all stored in the same directory, you will get an error:

/bin/rm: Argument list too long.

I've earlier blogged on deleting multiple files on Linux and FreeBSD and this is not my first time facing this error.
Anyways, as time passed, I've found few other new ways to delete large multitudes of files from a server.

In this article, I will explain shortly few approaches to delete few million of obsolete files to clean some space on your server.
Here are 3 methods to use to clean your tons of junk files.

1. Using Linux find command to wipe out millions of files

a.) Finding and deleting files using find's -exec switch:

# find . -type f -exec rm -fv {} \;

This method works fine but it has 1 downside, file deletion is too slow as for each found file external rm command is invoked.

For half a million of files or more, using this method will take "long". However from a server hard disk stressing point of view it is not so bad as, the files deletion is not putting too much strain on the server hard disk.
b.) Finding and deleting big number of files with find's -delete argument:

Luckily, there is a better way to delete the files, by using find's command embedded -delete argument:

# find . -type f -print -delete

c.) Deleting and printing out deleted files with find's -print arg

If you would like to output on your terminal, what files find is deleting in "real time" add -print:

# find . -type f -print -delete

To prevent your server hard disk from being stressed and hence save your self from server normal operation "outages", it is good to combine find command with ionice, e.g.:

# ionice -c 3 find . -type f -print -delete

Just note, that ionice cannot guarantee find's opeartions will not affect severely hard disk i/o requests. On  heavily busy servers with high amounts of disk i/o writes still applying the ionice will not prevent the server from being hanged! Be sure to always keep an eye on the server, while deleting the files nomatter with or without ionice. if throughout find execution, the server gets lagged in serving its ordinary client requests or whatever, stop the execution of the cmd immediately by killing it from another ssh session or tty (if physically on the server).

2. Using a simple bash loop with rm command to delete "tons" of files

An alternative way is to use a bash loop, to print each of the files in the directory and issue /bin/rm on each of the loop elements (files) like so:

for i in *; do
rm -f $i;
done

If you'd like to print what you will be deleting add an echo to the loop:

# for i in $(echo *); do \
echo "Deleting : $i"; rm -f $i; \

The bash loop, worked like a charm in my case so I really warmly recommend this method, whenever you need to delete more than 500 000+ files in a directory.

3. Deleting multiple files with perl

Deleting multiple files with perl is not a bad idea at all.
Here is a perl one liner, to delete all files contained within a directory:

# perl -e 'for(<*>){((stat)[9]<(unlink))}'

If you prefer to use more human readable perl script to delete a multitide of files use delete_multple_files_in_dir_perl.pl

Using perl interpreter to delete thousand of files is quick, really, really quick.
I did not benchmark it on the server, how quick exactly is it, but I guess the delete rate should be similar to find command. Its possible even in some cases the perl loop is  quicker …

4. Using PHP script to delete a multiple files

Using a short php script to delete files file by file in a loop similar to above bash script is another option.
To do deletion  with PHP, use this little PHP script:

<?php
$dir = "/path/to/dir/with/files";
$dh = opendir( $dir);
$i = 0;
while (($file = readdir($dh)) !== false) {
$file = "$dir/$file";
if (is_file( $file)) {
unlink( $file);
if (!(++$i % 1000)) {
echo "$i files removed\n";
}
}
}
?>

As you see the script reads the $dir defined directory and loops through it, opening file by file and doing a delete for each of its loop elements.
You should already know PHP is slow, so this method is only useful if you have to delete many thousands of files on a shared hosting server with no (ssh) shell access.

This php script is taken from Steve Kamerman's blog . I would like also to express my big gratitude to Steve for writting such a wonderful post. His post actually become  inspiration for this article to become reality.

You can also download the php delete million of files script sample here

To use it rename delete_millioon_of_files_in_a_dir.php.txt to delete_millioon_of_files_in_a_dir.php and run it through a browser .

Note that you might need to run it multiple times, cause many shared hosting servers are configured to exit a php script which keeps running for too long.
Alternatively the script can be run through shell with PHP cli:

php -l delete_millioon_of_files_in_a_dir.php.txt.

5. So What is the "best" way to delete million of files on Linux?

In order to find out which method is quicker in terms of execution time I did a home brew benchmarking on my thinkpad notebook.

a) Creating 509072 of sample files.

Again, I used bash loop to create many thousands of files in order to benchmark.
I didn't wanted to put this load on a productive server and hence I used my own notebook to conduct the benchmarks. As my notebook is not a server the benchmarks might be partially incorrect, however I believe still .they're pretty good indicator on which deletion method would be better.

hipo@noah:~$ mkdir /tmp/test
hipo@noah:~$ cd /tmp/test;
hiponoah:/tmp/test$ for i in $(seq 1 509072); do echo aaaa >> $i.txt; done

I had to wait few minutes until I have at hand 509072  of files created. Each of the files as you can read is containing the sample "aaaa" string.

b) Calculating the number of files in the directory

Once the command was completed to make sure all the 509072 were existing, I used a find + wc cmd to calculate the directory contained number of files:

hipo@noah:/tmp/test$ find . -maxdepth 1 -type f |wc -l
509072

real 0m1.886s
user 0m0.440s
sys 0m1.332s

Its intesrsting, using an ls command to calculate the files is less efficient than using find:

hipo@noah:/tmp/test$ time ls -1 |wc -l
509072

real 0m3.355s
user 0m2.696s
sys 0m0.528s

c) benchmarking the different file deleting methods with time

– Testing delete speed of find

hipo@noah:/tmp/test$ time find . -maxdepth 1 -type f -delete
real 15m40.853s
user 0m0.908s
sys 0m22.357s

You see, using find to delete the files is not either too slow nor light quick.

– How fast is perl loop in multitude file deletion ?

hipo@noah:/tmp/test$ time perl -e 'for(<*>){((stat)[9]<(unlink))}'real 6m24.669suser 0m2.980ssys 0m22.673s

Deleting my sample 509072 took 6 mins and 24 secs. This is about 3 times faster than find! GO-GO perl 🙂
As you can see from the results, perl is a great and time saving, way to delete 500 000 files.

– The approximate speed deletion rate of of for + rm bash loop

hipo@noah:/tmp/test$ time for i in *; do rm -f $i; done

real 206m15.081s
user 2m38.954s
sys 195m38.182s

You see the execution took 195m en 38 secs = 3 HOURS and 43 MINUTES!!!! This is extremely slow ! But works like a charm as the running of deletion didn't impacted my normal laptop browsing. While the script was running I was mostly browsing through few not so heavy (non flash) websites and doing some other stuff in gnome-terminal) 🙂

As you can imagine running a bash loop is a bit CPU intensive, but puts less stress on the hard disk read/write operations. Therefore its clear using it is always a good practice when deletion of many files on a dedi servers is required.

b) my production server file deleting experience

On a production server I only tested two of all the listed methods to delete my files. The production server, where I tested is running Debian GNU / Linux Squeeze 6.0.3. There I had a task to delete few million of files.
The tested methods tried on the server were:

– The find . type -f -delete method.

– for i in *; do rm -f $i; done

The results from using find -delete method was quite sad, as the server almost hanged under the heavy hard disk load the command produced.

With the for script all went smoothly. The files were deleted for a long long time (like few hours), but while it was running, the server continued with no interruptions..

While the bash loop was running, the server load avarage kept at steady 4
Taking my experience in mind, If you're running a production, server and you're still wondering which delete method to use to wipe some multitude of files, I would recommend you go  the bash for loop + /bin/rm way. Yes, it is extremely slow, expect it run for some half an hour or so but puts not too much extra load on the server..

Using the PHP script will probably be slow and inefficient, if compared to both find and the a bash loop.. I didn't give it a try yet, but suppose it will be either equal in time or at least few times slower than bash.

If you have tried the php script and you have some observations, please drop some comment to tell me how it performs.

To sum it up;

Even though there are "hacks" to clean up some messy parsing directory full of few million of junk files, having such a directory should never exist on the first place.

Frankly, keeping millions of files within the same directory is very stupid idea.
Doing so will have a severe negative impact on a directory listing performance of your filesystem in the long term.

If you know better (more efficient) ways to delete a multitude of files in a dir please share in comments.

How to block IP address with pf on FreeBSD, NetBSD and OpenBSD

Wednesday, July 27th, 2011

Pf Firewall BSD logo

I’ve noticed some IPs which had a kind of too agressive behaviour towards my Apache webserver and thus decided to filter them out with the Firewall.
As the server is running FreeBSD and my firewall choise is bsd’s pf I added the following lines to my /etc/pf.conf to filter up the abiser IP:

table persist file "/etc/pf.blocked.ip.conf"
EXT_NIC="ml0" # interface connected to internet
block drop in log (all) quick on $EXT_NIC from to any
echo '123.123.123.123' >> /etc/pf.blocked.ip.conf

As you see I’m adding the malicious IP to /etc/pf.blocked.ip.conf, if I later decide to filter some other IPs I can add them up there and they will be loaded and filtered by pf on next pf restart.

Next I restarted my pf firewall definitions to make the newly added rules in pf.conf to load up.

freebsd# pfctl -d
freebsd# pfctl -e -f /etc/pf.conf

To show all IPs which will be inside the blockips filtering tables, later on I used:

pfctl -t blockips -T show

I can also later use pf to add later on new IPs to be blocked without bothering to restart the firewall with cmd:

freebsd# pfctl -t blockedips -T add 111.222.333.444

Deleting an IP is analogous and can be achieved with:

freebsd# pfctl -t blockedips -T delete 111.222.333.444

There are also logs stored about pf IP blocking as well as the other configured firewall rules in /var/log/pflog file.
Hope this is helpful to somebody.