Posts Tagged ‘cake’

find text strings recursively in Linux and UNIX – find grep in sub-directories command examples

Tuesday, May 13th, 2014

unix_Linux_recursive_file_search_string_grep
GNU Grep
is equipped with a special option "-r" to grep recursively. Looking for string in a file in a sub-directories tree with the -r option is a piece of cake. You just do:

grep -r 'string' /directory/

or if you want to search recursively non-case sensitive for text

grep -ri 'string' .
 

Another classic GNU grep use (I use almost daily) is whether you want to match all files containing (case insensitive) string  among all files:

grep -rli 'string' directory-name
 

Now if you want to grep whether a string is contained in a file or group of files in directory recursively on some other UNIX like HP-UX or Sun OS / Solaris where there is no GNU grep installed by default here is how to it:

find /directory -exec grep 'searched string' {} dev/null ;

Note that this approach to look for files containing string on UNIX is very slowThus on not too archaic UNIX systems for some better search performance it is better to use xargs;

find . | xargs grep searched-string


A small note to open here is by using xargs there might be weird results when run on filesystems with filenames starting with "-".

Thus comes the classical (ultimate) way to grep for files containing string with find + grep, e.g.

find / -exec grep grepped-string {} dev/null ;

Another way to search a string recursively in files is by using UNIX OS '*' (star) expression:

grep pattern * */* */*/* 2>/dev/null

Talking about recursive directory text search in UNIX, should mention  another good GNU GREP alternative ACK – check it on betterthangrep.com 🙂 . Ack is perfect for programmers who have to dig through large directory trees of code for certain variables, functions, objects etc.

 

How to convert Ogg Video (.ogv) to Flash video (.flv) on Linux and FreeBSD

Thursday, September 29th, 2011

ffmpeg is the de-facto standard for Video conversion on Linux and BSD platforms. I was more than happy to find out that ffmpeg is capable of converting an .ogv file format to .flv (Flash compressed Video).
Ogg Vorbis Video to Flash’s conversion on Linux is a real piece of cake with ffmpeg .
Here is how to convert .ogv to .flv:

debian:~# ffmpeg -i ogg_vorbis_video_to_convert_.ogv converted_ogg_vorbis_video_to_flash_video.flv
...

Conversion of a 14MB ogg vorbis video to flv took 28 seconds, the newly produced converted_ogg_vorbis_video_to_flash_video.flv has been reduced to a size of 9MB. This is on a system with 2 GB of memory and dual core 1.8 Ghz intel CPU.

Scanning shared hosting servers to catch abusers, unwanted files, phishers, spammers and script kiddies with clamav

Friday, August 12th, 2011

Clamav scanning shared hosting servers to catch abusers, phishers, spammers, script kiddies etc.  logo

I’m responsible for some GNU/Linux servers which are shared hosting and therefore contain plenty of user accounts.
Every now and then our company servers gets suspended because of a Phishing websites, Spammers script kiddies and all the kind of abusers one can think of.

To mitigate the impact of the server existing unwanted users activities, I decided to use the Clamav Antivirus – open source virus scanner to look up for potentially dangerous files, stored Viruses, Spammer mailer scripts, kernel exploits etc.

The Hosting servers are running latest CentOS 5.5. Linux and fortunately CentOS is equipped with an RPM pre-packaged latest Clamav release which of the time of writting is ver. (0.97.2).

Installing Clamav on CentOS is a piece of cake and it comes to issuing:

[root@centos:/root]# yum -y install clamav
...

After the install is completed, I’ve used freshclam to update clamav virus definitions

[root@centos:/root]# freshclam
ClamAV update process started at Fri Aug 12 13:19:32 2011
main.cvd is up to date (version: 53, sigs: 846214, f-level: 53, builder: sven)
WARNING: getfile: daily-13357.cdiff not found on remote server (IP: 81.91.100.173)
WARNING: getpatch: Can't download daily-13357.cdiff from db.gb.clamav.net
WARNING: getfile: daily-13357.cdiff not found on remote server (IP: 163.1.3.8)
WARNING: getpatch: Can't download daily-13357.cdiff from db.gb.clamav.net
WARNING: getfile: daily-13357.cdiff not found on remote server (IP: 193.1.193.64)
WARNING: getpatch: Can't download daily-13357.cdiff from db.gb.clamav.net
WARNING: Incremental update failed, trying to download daily.cvd
Downloading daily.cvd [100%]
daily.cvd updated (version: 13431, sigs: 173670, f-level: 60, builder: arnaud)
Downloading bytecode.cvd [100%]
bytecode.cvd updated (version: 144, sigs: 41, f-level: 60, builder: edwin)
Database updated (1019925 signatures) from db.gb.clamav.net (IP: 217.135.32.99)

In my case the shared hosting hosted websites and FTP user files are stored in /home directory thus I further used clamscan in the following way to check report and log into file the scan results for our company hosted user content.

[root@centos:/root]# screen clamscan -r -i --heuristic-scan-precedence=yes --phishing-scan-urls=yes --phishing-cloak=yes --phishing-ssl=yes --scan-archive=no /home/ -l /var/log/clamscan.log
home/user1/mail/new/1313103706.H805502P12513.hosting,S=14295: Heuristics.Phishing.Email.SpoofedDomain FOUND/home/user1/mail/new/1313111001.H714629P29084.hosting,S=14260: Heuristics.Phishing.Email.SpoofedDomain FOUND/home/user1/mail/new/1305115464.H192447P14802.hosting,S=22663: Heuristics.Phishing.Email.SpoofedDomain FOUND/home/user1/mail/new/1311076363.H967421P17372.hosting,S=13114: Heuristics.Phishing.Email.SpoofedDomain FOUND/home/user1/mail/domain.com/infos/cur/859.hosting,S=8283:2,S: Heuristics.Phishing.Email.SSL-Spoof FOUND/home/user1/mail/domain.com/infos/cur/131.hosting,S=6935:2,S: Heuristics.Phishing.Email.SSL-Spoof FOUND

I prefer running the clamscan in a screen session, because it’s handier, if for example my ssh connection dies the screen session will preserve the clamscan cmd execution and I can attach later on to see how scan went.

clamscan of course is slower as it does not use Clamav antivirus daemon clamd , however I prefer running it without running the daemon, as having a permanently running clamd on the servers sometimes creates problems or hangs and it’s not really worthy to have it running since I’m intending to do a clamscan no more than once per month to see some potential users which might need to be suspended.

Also later on, after it finishes all possible problems are logged to /var/log/clamscan.log , so I can read the file report any time.

A good idea might also be to implement the above clamscan to be conducted, once per month via a cron job, though I’m still in doubt if it’s better to run it manually once per month to search for the malicious users content or it’s better to run it via cron schedule.

One possible pitfall with automating the above clamscan /home virus check up, might be the increased load it puts to the system. In some cases the Webserver and SQL server might be under a heavy load at the exactly same time the clamscan cron work is running, this might possible create severe issues for users websites, if it’s not monitored.
Thus I would probably go with running above clamscan manually each month and monitor the server performance.
However for people, who have “iron” system hardware and clamscan file scan is less likely to cause any issues, probably a cronjob would be fine. Here is sample cron job to run clamscan:

10 05 01 * * clamscan -r -i --heuristic-scan-precedence=yes --phishing-scan-urls=yes --phishing-cloak=yes --phishing-ssl=yes --scan-archive=no /home/ -l /var/log/clamscan.log >/dev/null 2>&1

I’m interested to hear if somebody already is using a clamscan to run on cron without issues, once I’m sure that running it on cron would not lead to server down-times, i’ll implement it via cron job.

Anyone having experience with running clamscan directory scan through crond? 🙂

Send message to GNOME on Linux over plain text ssh terminal session

Wednesday, July 27th, 2011

I just had some fun as, my work day is over. I was curious if I can send message directly to a logged in Gnome user on Ubuntu / Debian environment.

After quick investigation I’ve found gxmessage which is available as deb package for both Debian and Ubuntu.

I installed it and use in the following way:

root@linux:~# apt-get install gxmessage

hipo@linux:~$ ssh root@remotehost.com
hipo@remote-desktop:~$ DISPLAY=:0; XAUTHORITY=~owner_of:0/.Xauthority; export DISPLAY XAUTHORITY
hipo@remote-desktop:~$ gxmessage 'Heya what's up ;) Wanna have a cake?"

Below you see what pops up on the screen on the remote-desktop Gnome Desktop 😉
GxMessage Gnome message Screenshot

20th of July St. Elijah (Elias) feast in Bulgarian Orthodox Church / st. prophet Elias short Living

Thursday, July 21st, 2011

Saint Prophet Elijah Bulgarian Orthodox Christian icon

It’s again 20th of July and we in the Bulgarian Orthodox Church, celebrate the Holy memory of st. Prophet Elijah (Elias

St. Elijah is among the greatest Old Testamental prophets. He lived before the birth of our saviour Jesus Christ in a time when apostacy from true faith was severe.

In Elias’s time (few centuries B.C.), there used to be also a severe hunger on the land.
Elias was among the little number of Jews who did continued to honour the True God creator of Heaven and of Earth instead of turning to false religious teachings inspired by demons, like it’s the Baal’s cult which was widely spread among Jews back in the time.

Because of his faith in the true God, st. Prophet Elijah was hunted by the king’s wife Jezabel and he was forced to escape from Bersheeba and went alone in the wilderness and sat down in despondency under a juniper tree.

As he slept, an angel touched him, and said unto him, “Arise and eat; because the journey is too great for thee.” He arose and found a cake and a cruse of water. Having partaken of the provision, he went forward on his way for forty days to Horeb, where he took residence in a cave. Here God appeared to him and said, “What dost thou here, Elijah?” In answer to Elijah’s despondent words God manifests to him his glory, and then directs him to return to Damascus and anoint Hazael king over the Arameans (Syria), Jehu king over Israel, and Elisha to be prophet in his room (1 Kings 19:13-21; compare 2 Kings 8:7-15; 9:1-10).

Some six years after this he prophecised and warned by God’s commandment Ahab and Jezebel of the violent deaths they would die (1 Kings 21:19-24; 22:38)

The reason for the great hunger at this times was the fact that no rain has fallen down to the land for two years, as Elias prayed to God that no rain will be given to the people of Israel until they turn back to their true God.

One of the most notable moments of st. Elias’s earhtly living was the great miracle he performed by God’s providence which proofed that only the God of Israel is the true God of heaven and earth.

This happened while he was still in desolation on mount Carmel, it came to pass that Elijah met Obadiah, one of king Ahab’s officers (the king at that time of Israel).
Obadiah was he sent out at this time to seek for pasturage for the cattle (as the hunger on the land was so severe that even pastures for animals was obscure).
Elias’s by God’s word bade him go and tell his master that Elijah was there. King Ahab came forth and met Elias and accused him to be the “Trouble of Israel” as, he thought that his lands troubles are because Elias did not turns to Baal as the the king himself and most of the Jews at that time.
It was then proposed by Elias, that sacrifices should be publicly offered (to determine whose God is in control of taking out the hunger – e.g. Elias’s God of Israel or Baal to whom has many falsely turned.
, for the purpose of determining whether Baal or the Israelite’s God is the one’s master of the Universe.
The competition was accepted by Ahab and many of the prophets of Baal gathered together to “beg” his God for a miracle.

Baal’s false manu prophets gathered around the sacrifice place and started Begging Baal to send fire and burn the sacrificial altar.
Nomatter how hard they screamed, nothing happened so eventually they started cutting their bodies pouring blood in a hope to catch baal’s attention and make him have mercy on them.

As long time passed on and no miracle happened Elias told baal’s worshippers to step back from the sacrificial calf on the built altar and ordered some of the king’s servants to spill water over the calf (three times).
The water brought was so much that the whole altar trench was over-filled with water.

Then saint Elias begged to God: “Oh Lord, God of Abraham, Isaac and Jacob, please hear me! Let oh God this people recognize that You Lord is God and please oh Lord turn the people’s hearts towards you.”, he was still praying when fire came down from heaven and burnt the stones, the water in and the trench itself.

When all the gathered people around saw this great miracle of God, they bowed down and confessed the One and only God.
By Elias’s order the Baal’s false priests has been killed.
After this great miracle, Elias said to Ahab to came back home, before the rain has followed him.
The saint prophet climbed to the mountain top and prayed for a very long time.

After the holy man of God completed his prayers a powerful wind started blowing and the sky has filled with big clouds and a pouring rain come down to the overdried earth.
The king’s wife Jezabel, still kept her stubborness and continued severely hunting for st. Elias and even gave a vow to kill him because all her prophets were killed.

But God again was with st. Elias and saved him by warning him to run away in the wilderness.

Elias was in a big sorrow, here because many of the people still continued to honour the false god baal even though all the great heavenly signs.

God however by a great miracles, has once again shown the prophet he should be of a good spirit always and always to trust in God as he even send his Angel to give Elijah food in the desert.

Elias was among the only 7000 of Israelish man who does rejected to venerate the false god Baal.

By God’s providence st. Elias had a pupil prophet called Elijah.
The time now drew near when he was to be taken up into heaven (2 Kings 2:1-12). He went down to Gilgal, where there was a school of prophets, and where his successor Elisha, whom he had anointed some years before, resided.
Elisha was distraught by the thought of his master’s leaving him, and refused to be parted from him. The two went on and came to Bethel and Jericho, and crossed the Jordan, the waters of which were “divided hither and thither” when smitten with Elijah’s mantle.
Upon arriving at the borders of Gilead, which Elijah had left many years before, it “came to pass as they still went on and talked” they were suddenly separated by a chariot and horses of fire; and “Elijah went up by a whirlwind into heaven,” Elisha receiving his mantle, which fell from Elijah as he ascended.
Elijah’s chosen successor was the prophet Elisha; Elijah designated Elisha as such by leaving his mantle with him (2 Kings 2:13-15), so that his wish for “a double portion” of the older prophet’s spirit (2:9)

Later on in New Testamential time it was again st. prophet Elias, who appearance in glory on Mount Tabor and talked with our Saviour Christ at the Son of God’s Glorious Transfiguration.
An interesting fact is that Elias and Enoch are the only two people who according to the Holy writtings were taken by God straight to heaven. No other mortal man has been taken that way in flesh.

According to Orthodox Church’s tradition, Elias and Enoch will be brought back to earth again by God in the last days of the anti-christ, when they will again testify about our Lord Jesus Christ’s being the truth God and will also expose the anti-christ who will falsely pretend as being the true god of Israel and a true saviour of mankind.

Again according to this church history this two prophets at this last days, will be given an enormous power from God, even a power to stop the rain to come down to earth and the power to command the rivers to stop.

Oh Holy Prophet Elias, let God have mercy on us the sinners by your Holy prayers and save us, give us grace and always show us His great mercies.
Amen!

How to delete entries from routing table on Linux

Thursday, July 7th, 2011

Every now and then I had to tamper routing tables in Linux and every time I had to do it I forgot how I did it last time so finally I decided to put it on my blog and find how I can delete from Linux routing table easier

Deleting a record from a wrong routing table on Linux is a piece of cake basicly, here is an example:

linux:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.0.1 0.0.0.0 255.255.255.0 U 0 0 0 eth0
169.254.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth0

Let’s say this is the routing table and it’s incorrect as the routing through the node 169.254.0.0 should not be there.
Here is how the routing through 169.254.0.0 can be deleted:

linux:~# /sbin/route del -net 169.254.0.0 netmask 255.255.0.0

Now here is the difference after deletion:

linux:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.0.1 0.0.0.0 255.255.255.0 U 0 0 0 eth0
0.0.0.0 192.168.0.1 0.0.0.0 UG 0 0 0 eth0

If you want to delete the default gateway just use 0.0.0.0, e.g. -net 0.0.0.0 .. in above cmd example.
This would delete default gateway record from routing table which by the way in routing tables is marked with the UG flag.
Cheers 😉

How to extract Audio Sound and Music from Flash Videos (.flv) files and convert it to (.mp3) on Linux and BSD

Friday, April 15th, 2011

In my quest to know Linux better and use it capabilities to fulfill a multimedia tasks I came across the question:

How can I extract audio sound and music from Flash Videos .flv file format?

After a bit of investigation online I’ve found out in order to achieve this task the quickest way is via the handy ffmpeg conversion tool .

It’s rather easy actually, all necessery to do the conversion is to have the ffmpeg installed.
FFMpeg is part of Debian and Ubuntu repositories, so if you haven’t installed it yet, go straigh and install it with:

debian:~# apt-get install ffmpeg
...

Many modern day Linux distributions already have the ffmpeg pre-installed by default, ffmpeg even have a Windows version so this little tutorial should be directly applied on a Windows host with installed ffmpeg.

Convertion of a .flv file to .mp3 file for example is a real piece of cake to so do issue the command:

debian:~# ffmpeg -i input_file.flv -ab 128 -ar 44100 output_file.mp3

The few mmpeg options meaning is as follows:

-i (specifies input file)
-ab (Set the audio bitrate in bit/s 64k by default)
-ar (Set the audio sampling frequency (default = 44100 Hz).)

For more options checkout the ffmpeg help.

I found ffmpeg to be a bit slower than I expected. A 17 minutes .flv video file is converted to .mp3 for 38 seconds time.

Here is the textual output I got on my Debian Linux while extracting the flash video’s sound and converting it to mp3:

debian:~# time ffmpeg -i g7tvI6JCXD0.flv -ab 128 -ar 44100 output.mp3
FFmpeg version SVN-r25838, Copyright (c) 2000-2010 the FFmpeg developers
built on Jan 21 2011 08:21:58 with gcc 4.4.5
configuration: –enable-libdc1394 –prefix=/usr –extra-cflags=’-Wall -g ‘ –cc=’ccache cc’ –enable-shared –enable-libmp3lame –enable-gpl –enable-libvorbis –enable-pthreads –enable-libfaac –enable-libxvid –enable-postproc –enable-x11grab –enable-libgsm –enable-libtheora –enable-libopencore-amrnb –enable-libopencore-amrwb –enable-libx264 –enable-libspeex –enable-nonfree –disable-stripping –enable-avfilter –enable-libdirac –disable-decoder=libdirac –enable-libschroedinger –disable-encoder=libschroedinger –enable-version3 –enable-libopenjpeg –enable-libvpx –enable-librtmp –extra-libs=-lgcrypt –disable-altivec –disable-armv5te –disable-armv6 –disable-vis
libavutil 50.33. 0 / 50.39. 0
libavcore 0.14. 0 / 0.14. 0
libavcodec 52.97. 2 / 52.97. 2
libavformat 52.87. 1 / 52.87. 1
libavdevice 52. 2. 2 / 52. 2. 2
libavfilter 1.65. 0 / 1.65. 0
libswscale 0.12. 0 / 0.12. 0
libpostproc 51. 2. 0 / 51. 2. 0
[flv @ 0x1336760] Estimating duration from bitrate, this may be inaccurate

Seems stream 0 codec frame rate differs from container frame rate: 2000.00 (2000/1) -> 29.92 (359/12)
Input #0, flv, from ‘g7tvI6JCXD0.flv’:
Metadata:
duration : 1060
starttime : 0
totalduration : 1060
width : 480
height : 360
videodatarate : 76
audiodatarate : 94
totaldatarate : 179
framerate : 30
bytelength : 23723246
canseekontime : true
sourcedata : B5F9E82C6HH1302704673918653
purl :
pmsg :
Duration: 00:17:40.35, start: 0.000000, bitrate: 174 kb/s
Stream #0.0: Video: h264, yuv420p, 480×360 [PAR 1:1 DAR 4:3], 77 kb/s, 29.92 tbr, 1k tbn, 2k tbc
Stream #0.1: Audio: aac, 44100 Hz, stereo, s16, 96 kb/s
WARNING: The bitrate parameter is set too low. It takes bits/s as argument, not kbits/s
Output #0, mp3, to ‘output.mp3’:
Metadata:
duration : 1060
starttime : 0
totalduration : 1060
width : 480
height : 360
videodatarate : 76
audiodatarate : 94
totaldatarate : 179
framerate : 30
bytelength : 23723246
canseekontime : true
sourcedata : B5F9E82C6HH1302704673918653
purl :
pmsg :
TSSE : Lavf52.87.1
Stream #0.0: Audio: libmp3lame, 44100 Hz, stereo, s16, 0 kb/s
Stream mapping:
Stream #0.1 -> #0.0
Press [q] to stop encoding
size= 16576kB time=1060.81 bitrate= 128.0kbits/s
video:0kB audio:16575kB global headers:0kB muxing overhead 0.002404%

real 0m38.489s
user 0m37.126s
sys 0m0.764s

When talking about conversions, another very useful application of ffmpeg is in case if you want to:

Extract Audio from online streams

Let’s say you have a favourite radio, you often listen and there are a podcast you want to capture for later listening, or just catch a few nice songs, using ffmpeg it’s a piece of cake by using the command like:

debian:~# ffmpeg -i http:///xxx.xxx.xxx.xxx/some -ab 128 -ar 44100 captured-radio-sound.mp3

The possible ways of use of ffmpeg is truly versatily, you can use it for instance if you have to convert some kind of audio or video format to another one I have given a very simple example of converting a .flv file to .avi and vice versa in my previous post

How to Add sub-menus to Joomla main navigation buttons (Making dropdown menus) in Joomla

Tuesday, December 7th, 2010

I’m using a template youbizz just recently for a website. The website is already configured to use Joomla as a CMS.
This is a website related to the university so joomla was the perfect choice for a quick and easy to configure Content Management System

The youbizz template really makes difference and make the website layout look & feel quite a business looking.

However I needed a way to make my general configured menu buttons on the website to have a dropdown sub-menus in it

I even didn’t know if Joomla is supporting this, but by a chance I’ve made a submenu to the website Home buttom menu and thus I learned it how I can make submenus.

It actually appears creating the submenus to a menu in Joomla is a piece of cake, all you have to do in joomla administrator is go to to:

Menus -> Main Menu

From there you can configure as website buttons and link them to the ones already prepared in Article Manager

Just in case if you don’t know to add a buttons to a new joomla installation it’s necessary from admin to first go to:

Content -> Article Manager

Next press the New button (a green button with an image of a plus sign)

Thereon put a Title , Alias and in the input box put on everything you want this button pressed to show up in ur website.

Completing that simply press the Apply button and it should be done.
Next step to make the article appear in Menus -> Main Menu is to go this section and respectively from there use the green button with the plus sign to add up a new element to the main menu.

Right after you will have the Menu Item: [ New ] to appear it looks like in the pic below:

Joomla menu add new element screenshot

From the list of items to select you need to select the Article menu element. A submenu will appear in your browser to the Articles reading:

Article Layout
Article Submission Layout
etc.

From this menu you will have to select the Article Layout

The next screen to appear will look like the pic below:

Joomla main menu add new button through article, article layout

Here in article layout few things needs to be adjusted, again you will need to place the Title and the Alias, further after from the Parameters (Basic) located on the right you will have to select an article to link the new menu you’re just creating in Joomla’s new menu.
This is achieved via pressing the Select button located nearby the Select Article
Here it’s important to note the existence of Parent Item scrolldown field. This field will have all the created menu buttons in Main Menu . In case if this is the first one to create in main menu then it could be also empty.

The Parent Item field is really an important field and through it the menu’s submenus are created in Joomla.

For instance let’s say you want the current creating article to be listing as a sub-benu button to another already existing category, instead of listing as a separate button to the Joomla’s main menu navigation.

Well it’s pretty easy just choose from the Parent Item the name of an already existing menu button in Joomla main menu to be the parent of the sub-menu button you want to have.
That’s it now you will have the sub-menu button to appear as a drop down button (if of course your template supports dropdown menus).
There are few more options to choose in between which I found to be quite self explanatory, so I want explain them

But I hope I was able to explain at least the basics how menu subbuttons can be created in Joomla 1.5I find it a bit harder to explain in a bit plain way, but anyways if some of my dear readers is not understanding how to achieve the sub-menus I’m more than willing to help out further via the comments.