Posts Tagged ‘line’

Howto create Linux Music Audio CD from MP3 files / Create playable WAV format Audio CD Albums from MP3s

Tuesday, July 16th, 2019

cdburning-audio-music-cd-from-mp3-on-linuxcomapct-disc-tux-linux-logo

Recently my Mother asked me to prepare a Music Audio CD for her from a popular musician accordionist Stefan Georgiev from Dobrudja who has a unique folklore Bulgarian music.

As some of older people who still remember the age of the CD and who had most likely been into the CD burning Copy / Piracy business so popular in the countries of the ex-USSR so popular in the years 1995-2000 audio ,  Old CD Player Devices were not able to play the MP3 file format due to missing codecs (as MP3 was a proprietary compression that can't be installed on every device without paying the patent to the MP3 compression rights holder.

The revolutionary MP3 compression used to be booming standard for transferring Music data due to its high compression which made an ordinary MP3 of 5 minutes of 5MB (10+ times more compression than an ordinary classic WAV Audio the CPU intensiveness of MP3 files that puts on the reading device, requiring the CD Player to have a more powerful CPU.

Hence  due to high licensing cost and requirement for more powerful CPU enabled Audio Player many procuders of Audio Players never introduced MP3 to their devices and MP3 Neve become a standard for the Audio CD that was the standard for music listening inside almost every car out there.

Nowdays it is very rare need to create a Audio CD as audio CDs seems to be almost dead (As I heard from a Richard Stallman lecture In USA nowadays there is only 1 shop in the country where you can still buy CD or DVD drives) and only in third world as Africa Audio CDs perhaps are still in circulation.

Nomatter that as we have an old Stereo CD player on my village and perhaps many others, still have some old retired CD reading devices being able to burn out a CD is a useful thing.

Thus to make mother happy and as a learning excercise, I decided to prepare the CD for her on my Linux notebook.
Here I'll shortly describe the takes I took to make it happen which hopefully will be useful for other people that need to Convert and burn Audio CD from MP3 Album.

 

1. First I downloaded the Album in Mp3 format from Torrent tracker

My homeland Bulgaria and specific birth place place the city of Dobrich has been famous its folklore:  Galina Durmushlijska and Stefan Georgiev are just 2 of the many names along with Оркестър Кристал (Orchestra Crystal) and the multitude of gifted singers. My mother has a santiment for Stefan Georgiev, as she listened to this gifted accordinist on her Uncle's marriage.

Thus In my case this was (Стефан Георгиев Хора и ръченици от Добруджа) the album full song list here If you're interested to listen the Album and Enjoy unique Folklore from Dobrudja (Dobrich) my home city, Stefan Georgiev's album Hora and Rachenica Dances is available here

 


Stefan_Georgiev-old-audio-Music-CD-Hora-i-Rychenici-ot-Dobrudja-Horos-and-Ruchenitsas-from-Dobrudja-CD_Cover
I've downloaded them from Bulgarian famous torrent tracker zamunda.net in MP3 format.
Of course you need to have a CD / DVD readed and write device on the PC which nowdays is not present on most modern notebooks and PCs but as a last resort you can buy some cheap External Optical CD / DVD drive for 25 to 30$ from Amazon / Ebay etc.

 

2. You will need to install a couple of programs on Linux host (if you don't have it already)


To be able to convert from command line from MP3 to WAV you will need as minimum ffmpeg and normalize-audio packages as well as some kind of command line burning tool like cdrskin  wodim which is
the fork of old good known cdrecord, so in case if you you're wondering what happened with it just
use instead wodim.

Below is a good list of tools (assuming you have enough HDD space) to install:

 

root@jeremiah:/ # apt-get install –yes dvd+rw-tools cdw cdrdao audiotools growisofs cdlabelgen dvd+rw-tools k3b brasero wodim ffmpeg lame normalize-audio libavcodec58

 

Note that some of above packages I've installed just for other Write / Read operations for DVD drives and you might not need that but it is good to have it as some day in future you will perhaps need to write out a DVD or something.
Also the k3b here is specific to KDE and if you're a GNOME user you could use Native GNOME Desktop app such brasero or if you're in a more minimalistic Linux desktop due to hardware contrains use XFCE's native xfburn program.

If you're a console / terminal geek like me you will definitely enjoy to use cdw
 

root@jeremiah:/ # apt-cache show cdw|grep -i description -A 1
Description-en: Tool for burning CD's – console version
 Ncurses-based frontend for wodim and genisoimage. It can handle audio and

Description-md5: 77dacb1e6c00dada63762b78b9a605d5
Homepage: http://cdw.sourceforge.net/

 

3. Selecting preferred CD / DVD / BD program to use to write out the CD from Linux console


cdw uses wodim (which is a successor of good old known console cdrecord command most of use used on Linux in the past to burn out new Redhat / Debian / different Linux OS distro versions for upgrade purposes on Desktop and Server machines.

To check whether your CD / DVD drive is detected and ready to burn on your old PC issue:

 

root@jeremiah:/# wodim -checkdrive
Device was not specified. Trying to find an appropriate drive…
Detected CD-R drive: /dev/cdrw
Using /dev/cdrom of unknown capabilities
Device type    : Removable CD-ROM
Version        : 5
Response Format: 2
Capabilities   :
Vendor_info    : 'HL-DT-ST'
Identification : 'DVDRAM GT50N    '
Revision       : 'LT20'
Device seems to be: Generic mmc2 DVD-R/DVD-RW.
Using generic SCSI-3/mmc   CD-R/CD-RW driver (mmc_cdr).
Driver flags   : MMC-3 SWABAUDIO BURNFREE
Supported modes: TAO PACKET SAO SAO/R96P SAO/R96R RAW/R16 RAW/R96P RAW/R96R

You can also use xorriso (whose added value compared to other console burn cd tools is is not using external program for ISO9660 formatting neither it use an external or an external burn program for CD, DVD or BD (Blue Ray) drive but it has its own libraries incorporated from libburnia-project.org libs.

Below output is from my Thinkpad T420 notebook. If the old computer CD drive is there and still functional in most cases you should not get issues to detect it.

cdw ncurses text based CD burner tool's interface is super intuitive as you can see from below screenshot:

cdw-burn-cds-from-console-terminal-on-GNU-Linux-and-FreeBSD-old-PC-computer

CDW has many advanced abilities such as “blanking” a disk or ripping an audio CD on a selected folder. To overcome the possible problem of CDW not automatically detecting the disk you have inserted you can go to the “Configuration” menu, press F5 to enter the Hardware options and then on the first entry press enter and choose your device (by pressing enter again). Save the setting with F9.
 

4. Convert MP3 / MP4 Files or whatever format to .WAV to be ready to burn to CD


Collect all the files you want to have collected from the CD album in .MP3 a certain directory and use a small one liner loop to convert files to WAV with ffmpeg:
 

cd /disk/Music/Mp3s/Singer-Album-directory-with-MP3/

for i in $( ls *.mp3); do ffmpeg -i $i $i.wav; done


If you don't have ffmpeg installed and have mpg123 you can also do the Mp3 to WAV conversion with mpg123 cmd like so:

 

for i in $( ls ); do mpg123 -w $i.wav $i.mp3; done


Another alternative for conversion is to use good old lame (used to create Mp3 audio files but abling to also) decode
mp3 to wav.

 

lame –decode somefile.mp3 somefile.wav


In the past there was a burn command tool that was able to easily convert MP3s to WAV but in up2date Linux modern releases it is no longer available most likely due to licensing issues, for those on older Debian Linux 7 / 8 / 9 / Ubuntu 8 to 12.XX / old Fedoras etc. if you have the command you can install burn and use it (and not bother with shell loops):

apt-get install burn

or

yum install burn


Once you have it to convert

 

$ burn -A -a *.mp3
 

 

5. Fix file naming to remove empty spaces such as " " and substitute to underscores as some Old CD Players are
unable to understand spaces in file naming with another short loop.

 

for f in *; do mv "$f" `echo $f | tr ' ' '_'`; done

 

6. Normalize audio produced .WAV files (set the music volume to a certain level)


In case if wondering why normalize audio is needed here is short extract from normalize-audio man page command description to shed some light.

"normalize-audio  is  used  to  adjust  the volume of WAV or MP3 audio files to a standard volume level.  This is useful for things like creating mp3 mixes, where different recording levels on different albums can cause the volume to  vary  greatly from song to song."
 

cd /disk/Music/Mp3s/Singer-Album-directory-with-MP3/

normalize-audio -m *.wav

 

7. Burn the produced normalized Audio WAV files to the the CD

 

wodim -v -fix -eject dev='/dev/sr0' -audio -pad *.wav


Alternatively you can conver all your MP3 files to .WAV with anything be it audacity
or another program or even use 
GNOME's CDBurn tool brasero (if gnome user) or KDE's CDBurn which in my opinion is
the best CD / DVD burning application for Linux K3B.

Burning Audio CD with K3b is up to few clicks and super easy and even k3b is going to handle the MP3 to WAV file Conversion itself. To burn audio with K3B just run it and click over 'New Audio CD Project'.

k3b-on-debian-gnu-linux-burn-audio-cd-screenshot

For those who want to learn a bit more on CD / DVD / Blue-Ray burning on GNU / Linux good readings are:
Linux CD Burning Mini Howto, is Linux's CD Writing Howto on ibiblio (though a bit obsolete) or Debian's official documentation on BurnCD.
 

8. What we learned here


Though the accent of this tutorial was how to Create Audio Music CD from MP3 on GNU / Linux, the same commands are available in most FreeBSD / NetBSD / OpenBSD ports tree so you can use the same method to build prepare Audio Music CD on *BSDs.

In this article, we went through few basic ways on how to prepare WAV files from MP3 normalize the new created WAV files on Linux, to prepare files for creation of Audio Music CD for the old mom or grandma's player or even just for fun to rewind some memories. For GUI users this is easily done with  k3b,  brasero or xfburn.

I've pointed you to cdw a super useful text ncurses tool that makes CD Burninng from plain text console (on servers) without a Xorg / WayLand  GUI installed super easy. It was shortly reviewed what has changed over the last few years and why and why cdrecord was substituted for wodim. A few examples were given on how to handle conversion through bash shell loops and you were pointed to some extra reading resources to learn a bit more on the topic.
There are plenty of custom scripts around for doing the same CD Burn / Covnersion tasks, so pointing me to any external / Shell / Perl scripts is mostly welcome.

Hope this learned you something new, Enjoy ! 🙂

Windows missing volume control on Windows 7, 8 Fix / How to run volume control from command line

Thursday, March 9th, 2017

windows-7-missing-volume-controller-bring-back-volume-control-windows-7-command-to-show-volume-control-on-windows-os

 

Windows 7/8 Volume Icon disappear from Taskbar?

If you are using  Windows 7 or  Windows 8 Operating System inside a corporate network and your notebook PC is inside domain controller controlled by some crazy administrators who for some reason decided to remove the Taskbar from your Taskbar tray you have come over to exactly same situation like I do here.

Actually some might have experienced an icon "combined" feature which gives the opportunity of some of the standard Tray icons we know since Windows 98 / XP onwards to not show full time in order to save you space. No doubt this feature is great one to use as it is distracting sometimes to have a tons of applications constantly keeping in the Taskbar (right down corner) however if the Active Domain admin did it without any notification and you're a kind of victim you might dislike especially since this behaviour is making you impossible to easily control your phone / headspeakers and mic.

 

windows-7-8-grouped-taskbar-icons-screenshot-volume-dialog-bar

If you check in the Control Panel and click on Sounds  menu in Windows 7/8, you don't see any checkbox for adding the icon back as I have assumed, , but instead all the audio there you can only see the inputs and outputs on your system general settings.

windows-7-8-control-panel-sounds-dialog-box-properties

This behavior was made on purpose and makes sense cauze the taskbar icons since Win XP (if not mistaken) has to be controlled by the taskbar settings pan.

Thus in order to bring back the disappeared icon on  Winblows 7 / Win 8 there is a taskbar properties feature enabling to to hide or view the various taskbar running apps in that number the Volume icon, hence to bring back your Volme Control speaker icon to taskbar you need to customize it.

To do so do a mouse Right-click anywhere on the taskbar and choose Properties.

taskbar-properties-windows-7-8-dialog

Now, click on the Customize button under Notification area.

taskbar-and-startmenu-menu-properties-customize-taskbar-dialog-bar

In  Notification Area Icons dialog box, there is 2 ticks to check. Assure yourself the volume icon default behavior is set to

Show icon and notifications like in below screenshot

show-icon-notifications-volume-bar-windows-7-8-dialog

To make the new behaviour active click on Turn system icons on or off.

turn-system-icons-on-off-windows-7-8-notification-restore-default-icon-behavior-dialog

One thing to note here is the volume icon shoukld be set to On like in below| shot:

 

turn-system-icons-on-off-volume-icon-on-windows-7-8
If the reason for the disapperance of the Volume controller in task is not due to Domain Controllear policty it could happen due to late updates pushed by Microsoft if the PC needs a restart or after computer Log off operation.
Another reason for the casual disappearance of sound box could be also a buggy driver, so if the icon keeps disappearing over and over again, you better try to update the driver for your sound card.

However if you end up in a Windows Domain Controller (AD) Policy that is prohibiting the Sound Voulme to appear on your taskbar like in my case all the above won't help you solve it, but luckily there is an easy way to invoke the Volume Control dialog box via

 

sndvol.exe

 

the command will bring up the Volume Control as in upper left corner of screen like in below screenshot:

windows-volume-mixer-taskbar-windows-7-8-shot

 

If you to show it with a silder use -f flag

sndvol.exe -f

Running just

sndvol.exe


opens the volume mixer, as you noted.

 

On windows XP the respective command to open a missing Volume Control dialog in taskbar, use instead:

sndvol32

command from Windows Command Prompt:

 

Start -> Run -> cmd.exe

 

sndvol32

no params to display master volume window

 

 

sndvol32 -x

to display small master volume window

sndvol32 -t

to display volume control only (as per sound icon)

If you have the Volume Controller behavior to be hidden or you need to view any other taskbar hidden application icon  it will be useful for you to use:

AutoHotKey Win+B to focus on the system tray, Left (arrow) to highlight the Volume Control icon icon, and then Enter to bring up the popup.

 

A good tip you might be interestted to use occasionally is  how to show the current Wireless networks via a command (if that's prohibited otherwise via GUI) so you can easily see the  Connected Networks on Windows using cmd:

rundll32 van.dll,RunVAN

Windows XP multicore not detected after CPU update – XP Enable multicore after singlecore install

Friday, April 8th, 2016

windows-xp-add-multicore-with-command-after-multiple-cpu-not-detected
These days it is not common to install Windows XP however for some old unsupproted applications that still work on XP in many countries  in Africa, Asia, Europe and even America. Custom patched Windows XP is still heaveily used for some corporate businesses in accounting and on airports and other government institutions even to these day, I'm aware of Windows still heavily used especially in  Russia, Belarus,Ukraine, Kazakhstan, Armenia, Bulgaria etc.

Hence still there is plenty of softwares designed to work XP the good old Win XP and thus often XP needs to be emulated on VMs though officially not supported any longer  by Microsoft (its Support lifecycle End was for a last time on April 14, 2009).

Now I guess these days I guess nobody doesn't install and use Windows XP on a normal hardware PC Desktop / laptop but XP is continually installed on Virtual machine servers VMWare / VirtualBox.

Hence if you happen to have already migrated or installed some old Windows XP operating systems under VMWare for a corporate clients single core machine (no matter virtual or physical) and the client requires an update of hardware of the Virtual Machine you will be surprised that even though you add a second / third etc. core (new CPUs) the virtual machine hardware and restart the Windows XP installation.

It seems XP is designed to remember the install time CPU model hardware so once the VM and doesn't have a way to update its HAL (Hardware Abstraction Layer) definitions if you install it in Virtualbox thus to make XP recognize the extra added CPU cores it is necessery to do a small hack with a devcon.exe utility downloadable from Microsoft site to do the trick

1. Download the command line devicemanager utility (devcon.exe) from Microsoft Development Network MSDN here.

Note that it will work only if you use the correct version depending whether XP is  (x86/x64) bit install so check it out from My Computer -> Properties.

windows-xp-multicore-not-detected-enable-add-multicore-after-singlecore-windows-xp-install-with-devcon-exe

2  Next. Execute the following 2 commands:

    devcon sethwid @ROOT\ACPI_HAL\0000 := +acpiapic_mp !acpiapic_up
    devcon update c:\windows\inf\hal.inf acpiapic_mp

devcon.exe will  let the automatic hardware detection find out the extra CPU (multicores) added.
Wait 'till you get prompted for a reboot.
Be brave Reboot! 🙂

There is pretty much more fun useful things you can do with devcon.exe such as disabling USBs from command line,

DEVCON-command-DisableUSB_on-windows-xp-7-8-howto

listing your PCI devices and so on:

devcon-windows-command-to-list-pci-devices-on-xp7-win8-win10

You should now see all cores, hooray cores will appear in Task Manager / System Information.

Improve Apache Load Balancing with mod_cluster – Apaches to Tomcats Application servers Get Better Load Balancing

Thursday, March 31st, 2016

improve-apache-load-balancing-with-mod_cluster-apaches-to-tomcats-application-servers-get-better-load-balancing-mod_cluster-logo


Earlier I've blogged on How to set up Apache to to serve as a Load Balancer for 2, 3, 4  etc. Tomcat / other backend application servers with mod_proxy and mod_proxy_balancer, however though default Apache provided mod_proxy_balancer works fine most of the time, If you want a more precise and sophisticated balancing with better load distribuion you will probably want to install and use mod_cluster instead.

 

So what is Mod_Cluster and why use it instead of Apache proxy_balancer ?
 

Mod_cluster is an innovative Apache module for HTTP load balancing and proxying. It implements a communication channel between the load balancer and back-end nodes to make better load-balancing decisions and redistribute loads more evenly.

Why use mod_cluster instead of a traditional load balancer such as Apache's mod_balancer and mod_proxy or even a high-performance hardware balancer?

Thanks to its unique back-end communication channel, mod_cluster takes into account back-end servers' loads, and thus provides better and more precise load balancing tailored for JBoss and Tomcat servers. Mod_cluster also knows when an application is undeployed, and does not forward requests for its context (URL path) until its redeployment. And mod_cluster is easy to implement, use, and configure, requiring minimal configuration on the front-end Apache server and on the back-end servers.
 


So what is the advantage of mod_cluster vs mod proxy_balancer ?

Well here is few things that turns the scales  in favour for mod_cluster:

 

  •     advertises its presence via multicast so as workers can join without any configuration
     
  •     workers will report their available contexts
     
  •     mod_cluster will create proxies for these contexts automatically
     
  •     if you want to, you can still fine-tune this behaviour, e.g. so as .gif images are served from httpd and not from workers…
     
  •     most importantly: unlike pure mod_proxy or mod_jk, mod_cluster knows exactly how much load there is on each node because nodes are reporting their load back to the balancer via special messages
     
  •     default communication goes over AJP, you can use HTTP and HTTPS

 

1. How to install mod_cluster on Linux ?


You can use mod_cluster either with JBoss or Tomcat back-end servers. We'll install and configure mod_cluster with Tomcat under CentOS; using it with JBoss or on other Linux distributions is a similar process. I'll assume you already have at least one front-end Apache server and a few back-end Tomcat servers installed.

To install mod_cluster, first download the latest mod_cluster httpd binaries. Make sure to select the correct package for your hardware architecture – 32- or 64-bit.
Unpack the archive to create four new Apache module files: mod_advertise.so, mod_manager.so, mod_proxy_cluster.so, and mod_slotmem.so. We won't need mod_advertise.so; it advertises the location of the load balancer through multicast packets, but we will use a static address on each back-end server.

Copy the other three .so files to the default Apache modules directory (/etc/httpd/modules/ for CentOS).
Before loading the new modules in Apache you have to remove the default proxy balancer module (mod_proxy_balancer.so) because it is not compatible with mod_cluster.

Edit the Apache configuration file (/etc/httpd/conf/httpd.conf) and remove the line

 

LoadModule proxy_balancer_module modules/mod_proxy_balancer.so

 


Create a new configuration file and give it a name such as /etc/httpd/conf.d/mod_cluster.conf. Use it to load mod_cluster's modules:

 

 

 

LoadModule slotmem_module modules/mod_slotmem.so
LoadModule manager_module modules/mod_manager.so
LoadModule proxy_cluster_module modules/mod_proxy_cluster.so

In the same file add the rest of the settings you'll need for mod_cluster something like:

And for permissions and Virtualhost section

Listen 192.168.180.150:9999

<virtualhost  192.168.180.150:9999="">

    <directory>
        Order deny,allow
        Allow from all 192.168
    </directory>

    ManagerBalancerName mymodcluster
    EnableMCPMReceive
</virtualhost>

ProxyPass / balancer://mymodcluster/


The above directives create a new virtual host listening on port 9999 on the Apache server you want to use for load balancing, on which the load balancer will receive information from the back-end application servers. In this example, the virtual host is listening on IP address 192.168.204.203, and for security reasons it allows connections only from the 192.168.0.0/16 network.
The directive ManagerBalancerName defines the name of the cluster – mymodcluster in this example. The directive EnableMCPMReceive allows the back-end servers to send updates to the load balancer. The standard ProxyPass and ProxyPassReverse directives instruct Apache to proxy all requests to the mymodcluster balancer.
That's all you need for a minimal configuration of mod_cluster on the Apache load balancer. At next server restart Apache will automatically load the file mod_cluster.conf from the /etc/httpd/conf.d directory. To learn about more options that might be useful in specific scenarios, check mod_cluster's documentation.

While you're changing Apache configuration, you should probably set the log level in Apache to debug when you're getting started with mod_cluster, so that you can trace the communication between the front- and the back-end servers and troubleshoot problems more easily. To do so, edit Apache's configuration file and add the line LogLevel debug, then restart Apache.
 

2. How to set up Tomcat appserver for mod_cluster ?
 

Mod_cluster works with Tomcat version 6, 7 and 8, to set up the Tomcat back ends you have to deploy a few JAR files and make a change in Tomcat's server.xml configuration file.
The necessary JAR files extend Tomcat's default functionality so that it can communicate with the proxy load balancer. You can download the JAR file archive by clicking on "Java bundles" on the mod_cluster download page. It will be saved under the name mod_cluster-parent-1.2.6.Final-bin.tar.gz.

Create a new directory such as /root/java_bundles and extract the files from mod_cluster-parent-1.2.6.Final-bin.tar.gz there. Inside the directory /root/java_bundlesJBossWeb-Tomcat/lib/*.jar you will find all the necessary JAR files for Tomcat, including two Tomcat version-specific JAR files – mod_cluster-container-tomcat6-1.2.6.Final.jar for Tomcat 6 and mod_cluster-container-tomcat7-1.2.6.Final.jar for Tomcat 7. Delete the one that does not correspond to your Tomcat version.

Copy all the files from /root/java_bundlesJBossWeb-Tomcat/lib/ to your Tomcat lib directory – thus if you have installed Tomcat in

/srv/tomcat

run the command:

 

cp -rpf /root/java_bundles/JBossWeb-Tomcat/lib/* /srv/tomcat/lib/.

 

Then edit your Tomcat's server.xml file

/srv/tomcat/conf/server.xml.


After the default listeners add the following line:

 

<listener classname="org.jboss.modcluster.container.catalina.standalone.ModClusterListener" proxylist="192.168.204.203:9999"> </listener>



This instructs Tomcat to send its mod_cluster-related information to IP 192.168.180.150 on TCP port 9999, which is what we set up as Apache's dedicated vhost for mod_cluster.
While that's enough for a basic mod_cluster setup, you should also configure a unique, intuitive JVM route value on each Tomcat instance so that you can easily differentiate the nodes later. To do so, edit the server.xml file and extend the Engine property to contain a jvmRoute, like this:
 

.

 

<engine defaulthost="localhost" jvmroute="node2" name="Catalina"></engine>


Assign a different value, such as node2, to each Tomcat instance. Then restart Tomcat so that these settings take effect.

To confirm that everything is working as expected and that the Tomcat instance connects to the load balancer, grep Tomcat's log for the string "modcluster" (case-insensitive). You should see output similar to:

Mar 29, 2016 10:05:00 AM org.jboss.modcluster.ModClusterService init
INFO: MODCLUSTER000001: Initializing mod_cluster ${project.version}
Mar 29, 2016 10:05:17 AM org.jboss.modcluster.ModClusterService connectionEstablished
INFO: MODCLUSTER000012: Catalina connector will use /192.168.180.150


This shows that mod_cluster has been successfully initialized and that it will use the connector for 192.168.204.204, the configured IP address for the main listener.
Also check Apache's error log. You should see confirmation about the properly working back-end server:

[Tue Mar 29 10:05:00 2013] [debug] proxy_util.c(2026): proxy: ajp: has acquired connection for (192.168.204.204)
[Tue Mar 29 10:05:00 2013] [debug] proxy_util.c(2082): proxy: connecting ajp://192.168.180.150:8009/ to  192.168.180.150:8009
[Tue Mar 29 10:05:00 2013] [debug] proxy_util.c(2209): proxy: connected / to  192.168.180.150:8009
[Tue Mar 29 10:05:00 2013] [debug] mod_proxy_cluster.c(1366): proxy_cluster_try_pingpong: connected to backend
[Tue Mar 29 10:05:00 2013] [debug] mod_proxy_cluster.c(1089): ajp_cping_cpong: Done
[Tue Mar 29 10:05:00 2013] [debug] proxy_util.c(2044): proxy: ajp: has released connection for (192.168.180.150)


This Apache error log shows that an AJP connection with 192.168.204.204 was successfully established and confirms the working state of the node, then shows that the load balancer closed the connection after the successful attempt.

You can start testing by opening in a browser the example servlet SessionExample, which is available in a default installation of Tomcat.
Access this servlet through a browser at the URL http://balancer_address/examples/servlets/servlet/SessionExample. In your browser you should see first a session ID that contains the name of the back-end node that is serving your request – for instance, Session ID: 5D90CB3C0AA05CB5FE13121E4B23E670.node2.

Next, through the servlet's web form, create different session attributes. If you have a properly working load balancer with sticky sessions you should always (that is, until your current browser session expires) access the same node, with the previously created session attributes still available.

To test further to confirm load balancing is in place, at the same time open the same servlet from another browser. You should be redirected to another back-end server where you can conduct a similar session test.
As you can see, mod_cluster is easy to use and configure. Give it a try to address sporadic single-back-end overloads that cause overall application slowdowns.

Check Windows Operating System install date, Full list of installed and uninstalled programs from command line / Check how old is your Windows installation?

Tuesday, March 29th, 2016

when-was-windows-installed-check-howto-from-command-line
Sometimes when you have some inherited Windows / Linux OS servers or Desktops, it is useful to be aware what is the Operating System install date. Usually the install date of the OS is closely to the date of purchase of the system this is especially true for Windows but not necessery true for Liunx based installs.

Knowing the install date is useful especially if you're not sure how outdated is a certain operating system. Knowing how long ago a current installation was performed could give you some hints on whether to create a re-install plans in order to keep system security up2date and could give you an idea whether the system is prone to some common errors of the time of installation or security flaws.

 

1. Check out how old is Windows install?

Finding out the age of WIndows installation can be performed across almost all NT 4.0 based Windowses and onwards, getting Winblows install date is obtained same way on both Windows XP / Vista/  7  and 8.

Besides many useful things such as detailed information about the configuration of your PC / notebook systeminfo could also provide you with install date, to do so just run from command line (cmd.exe).
 

C:\Users\hipo> systeminfo | find /i "install date"
Original Install Date:     09/18/13, 15:23:18 PM


check-windows-os-install-date-from-command-line-howto-screenshot

If you need to get the initial Windows system install date however it might be much better to use WMIC command to get the info:

 

 

C:\Users\hipo>WMIC OS GET installdate
InstallDate
20130918152318.000000+180


The only downside of using WMIC as you can see is it provides the Windows OS install date in a raw unparsed format, but for scripters that's great.

2. Check WIndows Installed and Uinstalled software and uptime from command line

One common other thing next to Windows install date is what is the Windows uptime, the easiest way to get that is to run Task Manager in command line run taskmgr

windows-task-manager-how-to-check-windows-operating-system-uptime-easily

For those who want to get the uptime from windows command line for scripting purposes, this can be done again with systeminfo cmd, i.e.:

 

C:\> systeminfo | find "System Boot Time:"
System Boot Time:          03/29/16, 08:48:59 AM


windows-os-command-to-get-system-uptime-screenshot

Other helpful Windows command liners you might want to find out about is getting all the Uninstalled and Installed programs from command line this again is done with WMIC

 

C:\> wmic /OUTPUT:my_software.txt product get name

 


get-a-full-list-of-installed-software-programs-on-windows-xp-vista-7-8-command-howto-screenshot

Alternative way to get a full list of installed software on Windows OS is to use Microsoft/SysInternals psinfo command:

 

C:\> psinfo -s > software.txt
C:\> psinfo -s -c > software.csv


If you need to get a complete list of Uinstalled Software using command line (e.g. for batch scripting) purposes, you can query that from Windows registry, like so:

 

C:\>reg query HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Uninstall


Command Output will be something like on below shot:

windows-OS-show-get-full-list-of-uninstalled-programs-using-a-command-line-screenshot

Well that's all folks 🙂

 

Remove string line from file on Linux and BSD – Delete entire line with string from file

Tuesday, March 15th, 2016

linux-remove-lines-containing-string-with-sed

If you're already used too using grep -v "sometring" filename to print everything from a file without the certain grepped string output and you want to do the same to delete lines based on strings without having to output the grepped string to a file and then overwritting the original file:
 

grep -v 'whatever' filename > filename1
mv filename1 filename


A much better way to delete an whole line containing a string match from a file is to use sed
sed
should be the tool of choice especially if you're scripting because sed is especially made for such batch edittings.

Here is how to do delete an entire line based on a given string:

 

sed –in-place '/some string to search and delete/d' myfilename


It might be a good idea to also create backups just to make sure something doesn't get deleted incidently to do use:

sed –in-place=.bak '/some string to search and delete/d' myfilename

If you need to wipe out an exact string from all files within a folder you might use a for loop or perl (some good examples check my previous article here)

In short to use bash's for loop here is how to backup and remove all lines with a string match within all files within a Linux directory:

 

for f in *.txt; do sed –in-place '/some string/d'
"$f"; done
find -name '*.txt' -exec sed –in-place=.bak '/some
string/d' "{}" ';'

 

BTW SED is really rich editor and some people got so much into it that there is even a sed written text (console) version of arkanoid 🙂

sed-text-editor-written-arkanoid-game-linux-bsd

If you want to break the ice and get some fun in your boring sysadmin life get sed arkanoid code from here.
I have it installed under pc-freak.net free ASCII Games entertainment service, so if you want to give it a try just login and give a try.

Enjoy 🙂

How to SSH client Login to server with password provided from command line as a script argument – Running same commands to many Linux servers

Friday, March 6th, 2015

ssh-how-to-login-with-password-provided-from-command-line-use-sshpass-to-run-same-command-to-forest-of-linux-servers

Usually admins like me who casuanlly need to administer "forests" (thousands of identicallyconfigured services Linux servers) are generating and using RSA / DSA key authentication for passwordless login, however this is not always possible as some client environments does prohibit the use of RSA / DSA non-pass authentication, thus in such environments to make routine server basic package rpm / deb upgrades or do other maintanance patching its necessery to use normal ssh user / pass login but as ssh client doesn't allow password to be provided from prompt for security reasons and therefore using some custom bash loop to issue single command to many servers (such as explained in my previous article) requires you to copy / paste password on password prompt multiple times. This works its pretty annoying so if you want to run single command on all your 500 servers with specifying the password from password prompt use sshpass tool (for non-interactive ssh password auth).

SSHPASS official site description:
 

sshpass is a utility designed for running ssh using the mode referred to as "keyboard-interactive" password authentication, but in non-interactive mode.

 

Install sshpass on Debian / Ubuntu (deb based) Linux

sshpass is installable right out of regular repositories so to install run:
 

apt-get install —yes sshpass


Install sshpass on CentOS / Fedora (RPM based) Linux


sshpass is available also across most RPM based distros too so just use yum package manager

 

yum -y install sshpass


If its not available across standard RPM distro provided repositories, there should be RPM on the net for distro just download latest one and use wget and rpm to install:

 wget -q http://dl.fedoraproject.org/pub/epel/6/x86_64/sshpass-1.05-1.el6.x86_64.rpm

 rpm -ivh sshpass-1.05-1.el6.x86_64.rpm

 

How Does SshPass Works?

 

Normally openssh (ssh) client binary uses direct TTY (/dev/tty)= an abbreviation for PhyTeleTYpewriter or (the admin jargon call Physical Console access)  instead of standard remotely defined /dev/ptsVirtual PTY.
To get around this Sshpass runs ssh in a dedicated TTY to emulate the password is indeed issues by interactive keyboard user thus  fooling remote sshd server to thinking password
is provided by interactive user.


SSHPass use

Very basic standard use which allows you to pass the password from command line is like this:
 

sshpass -p 'Your_Password_Goes_here123' ssh username@server.your-server.com


Note that the server you're working is shared with other developers they might be able to steal your username / password by using a simple process list command such as:
 

 ps auxwwef


In my case security is not a hot issue, as I'm the only user on the server (and only concern might be if someone hacks into the server 🙂 

 

Then assuming that you have a plain text file with all your administered servers, you can easily use sshpass in a Bash Script loop in order to run, lets say a package upgrade across all identical Linux version machines:
 

while read line; do
sshpass -p 'Your_Password_Goes_here123' ssh username@$line "apt-get update && apt-get upgrade && apt-get dist-upgrade" < /dev/null;
done < all_servers_list.txt

Change the command you like to issue across all machines with the string "apt-get …"
Above command can be used to keep up2date all Debian stable server packages. What you will do on servers is up to your imaginations, very common use of above line would be if you want to see uptime /netstat command output across all your network servers.

 

while read line; do
sshpass -p 'Your_Password_Goes_here123' ssh username@$line "uptime; who; netstat -tunlp; " < /dev/null;
done < all_servers_list.txt

 


As you can guess SshPass is swiss army knife tool for admins whoneed to automate things with scripts simultaneously across number of servers.
 

Happy SSH-ing 🙂

 

 

 

How to deb upgrade PHP 5.3.3-7 / MySQL Server 5.1 to PHP 5.4.37 MySQL 5.5 Server on Debian 6.0 / 7.0 Squeeze / Wheezy GNU / Linux

Thursday, February 12th, 2015

how-to-deb-upgrade-mysql-server-5.1-to-mysql-5.5-php-5.3-to-php-5.4-5.5-upgrade-howto-on-old-stable-debian-squeeze-wheezy

I've been still running Debian Squeeze 6.0 GNU / Linux on few of the Linux / Apache / MySQL servers I'm administrating and those servers are running few Wordperss / Joomla websites which lately face severe MySQL performance issues. I tried to optimize using various mysql performance optimization scripts such as mysql-tuner.pl, Tuning-primer.sh and Percona Toolkit – a collection of advanced command-line tools for system administrators and tech / support staff to perform a variety of MySQL and system tasks that are too difficult or complex to perform manually. Though with above tools and some my.cnf tunizations I managed to achieve positive performance improvement results with above optimizations, still I didn't like how MyQSL served queries and since the SQL server is already about 5 years old (running version 5.1) and the PHP on sever is still at 5.3 branch, I was advised by my dear colleague Anatoliy to try version update as a mean to improve SQLserver performance. I took seriously the suggestion to try upgrade as a mean to resolve performance issues in this article I will explain in short what I had to do to make MySQL upgrade a success

Of course to try keep deb installed software versions as fresh as possible possible deb packagse, I'm already using Debian Back Ports (for those who hear it a first time Debian Backports is a special repository for Stable versioned Debian Desktop and Servers  – supporting stable releases of Debian Linux) which allows you to keep install packages versions less outdated (than default installable software which usually are way behind latest stable package versions with 2-5 years).

If you happen to administer Stable Debian servers and you never used BackPorts I warmly recommend it as it often includes security patches of packages part of Debian stable releases that reached End Of Support (EOS) and already too old even for security updates to be issued by respective Debian Long Term Suport (LTS) repositories.

If you're like me and still in situation to manage remotely Debian 6.0 Squeeze and its the first time you hear about BackPorts and Debian LTShttps://wiki.debian.org/LTS/ to start using those two add to your /etc/apt/sources.list below 3 lines

Open with vim editor and press shift+G to go to last line of file and then press I to enter INSERT mode, once you're done to save, press (ESC) then press : and type x! in short key combination for exit and save setting in vim is 
 

Esc + :x! 

 

debian-server:~# vim /etc/apt/sources.list
deb http://http.debian.net/debian squeeze-lts main contrib non-free
deb-src http://http.debian.net/debian squeeze-lts main contrib non-free
deb http://http.debian.net/debian-backports squeeze-backports main

If you haven't been added a security updates line in /etc/apt/sources.list make sure you add also:

 

deb http://security.debian.org/ squeeze/updates main contrib non-free
deb-src http://security.debian.org/ squeeze/updates main contrib non-free


Then to apply latest security updates and packages from LTS / Backports repository run the usual:

 

debian-server:~# apt-get update && apt-get –yes upgrade
….

If you need to search a package or install something from just added backports repository use:

 

debian-server:~# apt-cache -t squeeze-backports search "mysql-server"
auth2db – Powerful and eye-candy IDS logger, log viewer and alert generator
torrentflux – web based, feature-rich BitTorrent download manager
cacti – Frontend to rrdtool for monitoring systems and services
mysql-server-5.1 – MySQL database server binaries and system database setup
mysql-server-core-5.1 – MySQL database server binaries
mysql-server – MySQL database server (metapackage depending on the latest version)

 

To install specific packages only with all their dependencies from Backports while keeping rest of packages from Debian Stable:

 

debian-server:~# apt-get install -t squeeze-backports "package_name"

In same way you can also search or install specific packages from LTS repo:

 

debian-server:~# apt-get search -t squeeze-lts "package_name"

debian-server:~# apt-get install -t squeeze-lts "package_name"

Latest mysql available from Debian BackPorts and LTS is still quite old 5.1.73-1+deb6u1 therefore I made an extensive research online on how can I easily update MySQL 5.1 to MySQL 5.5 / 5.6 on Debian Stable Linux.
 

Luckily there were already DotDeb deb repositories for Debian LAMP (Linux / Apache  / MySQL / PHP / Nginx ) running servers prepared in order to keep the essential Webserver services up2date even long after distro official support is over. I learned about existence of this repo thanks to a Ryan Tate's post who updates his LAMP stack on TurnKey Linux which by the way is based on slightly modified official stable Debian Linux releases packages

To start using DotDeb repos add in /etc/apt/sources.list (depending whereh you're on Squeeze or Wheeze Debian):

 

deb http://packages.dotdeb.org squeeze all
deb-src http://packages.dotdeb.org squeeze all

or for Debian Wheezy add repos:

 

deb http://packages.dotdeb.org wheezy all
deb-src http://packages.dotdeb.org wheezy all

 

I was updating my DebianLatest MySQL / PHP / Apache release to Latest ones on (6.0.4) Squeeze so added above squeeze repos:

Before refreshing list of package repositories, to authenticate repos issue:

 

debian-server:~# wget -q http://www.dotdeb.org/dotdeb.gpg
debian-server:~# apt-key add dotdeb.gpg

Once again to update my packages from newly added DodDeb repository

 

debian-server:~# apt-get update

Before running the SQL upgrade to insure myself, I dumped all databases with:

 

debian-server:~# mysqldump -u root -p -A > /root/dump.sql

Finally I was brave enough to run apt-get dist-upgrade to update with latest LAMP packages

 

debian-server:~# apt-get dist-upgrade
Reading package lists… Done
Building dependency tree
Reading state information… Done
Calculating upgrade… Done
The following packages will be REMOVED:
  mysql-client-5.1 mysql-server mysql-server-5.1
The following NEW packages will be installed:
  libaio1 libmysqlclient18 mysql-client-5.5 mysql-client-core-5.5 python-chardet python-debian
The following packages will be upgraded:
  curl krb5-multidev libapache2-mod-php5 libc-bin libc-dev-bin libc6 libc6-dev libc6-i386 libcurl3 libcurl3-gnutls libcurl4-openssl-dev libevent-1.4-2
  libgssapi-krb5-2 libgssrpc4 libjasper1 libk5crypto3 libkadm5clnt-mit7 libkadm5srv-mit7 libkdb5-4 libkrb5-3 libkrb5-dev libkrb53 libkrb5support0 libmysqlclient-dev
  libxml2 libxml2-dev locales mysql-client mysql-common ntp ntpdate php-pear php5 php5-cgi php5-cli php5-common php5-curl php5-dev php5-gd php5-imagick php5-mcrypt
  php5-mysql php5-odbc php5-recode php5-sybase php5-xmlrpc php5-xsl python-reportbug reportbug unzip

50 upgraded, 6 newly installed, 3 to remove and 0 not upgraded.
Need to get 51.7 MB of archives.
After this operation, 1,926 kB of additional disk space will be used.
Do you want to continue [Y/n]? Y

As you see from above output above command updates Apache webservers / PHP and PHP related modules, however it doesn't update MySQL installed version, to update also MySQL server 5.1 to MySQL server 5.5

 

debian-server:~#  apt-get install –yes mysql-server mysql-server-5.5

You will be prompted with the usual Debian ncurses text blue interface to set a root password to mysql server, just set it the same as it used to be on old upgraded MySQL 5.1 server.

Well now see whether mysql has properly restarted with ps auxwwf

 

debian-server:~#  ps axuwwf|grep -i sql
root     22971  0.0  0.0 112360   884 pts/11   S+   15:50   0:00  |                   \_ grep -i sql
root     19436  0.0  0.0 115464  1556 pts/1    S    12:53   0:00 /bin/sh /usr/bin/mysqld_safe
mysql    19837  4.0  2.3 728192 194552 pts/1   Sl   12:53   7:12  \_ /usr/sbin/mysqld –basedir=/usr –datadir=/var/lib/mysql –plugin-dir=/usr/lib/mysql/plugin –user=mysql –pid-file=/var/run/mysqld/mysqld.pid –socket=/var/run/mysqld/mysqld.sock –port=3306
root     19838  0.0  0.0 110112   700 pts/1    S    12:53   0:00  \_ logger -t mysqld -p daemon.error

In my case it was running, however if it fails to run try to debug what is going wrong on initialization by manually executing init script /etc/init.d/mysql stop; /etc/init.d/mysql start and look for errors. You can also manually try to run mysqld_safe from console if it is not running run:

 

debian-server:~# /usr/bin/mysqld_safe &

This should give you a good hint on why it is failing to run
 

One more thing left is to check whether php modules load correctly to do so issue:

 

debian-server:~# php -v
Failed loading /usr/lib/php5/20090626/xcache.so:  /usr/lib/php5/20090626/xcache.so: cannot open shared object file: No such file or directory

Failed loading /usr/lib/php5/20090626/xdebug.so:  /usr/lib/php5/20090626/xdebug.so: cannot open shared object file: No such file or directory


You will likely get an exception (error) like above.
To solve the error, reinstall xcache and xcache-debug debs

 

debian-server:~# apt-get purge php5-xcache php5-xdebug

Now PHP + MySQL + Apache environment should be running much smootly.

debian-squeeze-wheeze-update-install-mysql-sevver5.55-620x344-howto

Upgrading the MySQL server / PHP library to MySQL server 5.6 / PHP 5.5 on Wheeze Linux is done in very much analogous ways all you have to do is change the repositories with above wheeze 7.0 ones and to follow the process as described in this article. I haven't tested update on Wheezy yet, so if you happen to try my article with wheezy reports and got a positive upgrade result please drop a comment.

Automatic restart Tomcat on Windows script via TaskScheduler daily – A command line to add / remove new Windows “Cron” like job

Thursday, January 22nd, 2015

automatic-restart-Tomcat-on-Windows-via-TaskShcheduler-daily-weekly-monthly-a-command-line-to-add-remove-new-windows-cron-job
I'm responsbile for a project environment made up of 3 components which is occasionally dying. Here is a short raw overview of environment

  • Apache Reverse Proxy (entry door to app server)
  • Tomcat Server with an Application enabling web access
  • A Java Standalone application using SQLite database

 The Tomcat and Java Standalone application is running on top of Windows 2008 RC2 Standard, the overall environment is becoming inacessible periodically and in order to solve that the customer decided to implement a daily Windows server reboot in my opinion this is very bad approach as it is much better to just set an auto reboot of each of components using few tiny batch scripts and Windows Taskmgr, however as the customer is king and decided to implement the reboot its their own thing. 
However even fter the daily server reboot was set once a week or so the application was becoming inaccessible and a Tomcat server restart was necessery as a fix.

Finally as a work-around to the issue, I've proposed the logical thing to automatically restart Tomcat once a day early in morning, here is how Tomcat auto Restart was implemented on the Win server:

1. Check out the name of running Tomcat service

First thing is to use the sc command to find out the Tomcat application name:

 

how-to-show-tomcat-service-name-command-windows-screenshot

C:UsersGeorgi>sc query state= all| findstr "Tomcat"
SERVICE_NAME: Tomcat7_r2c
DISPLAY_NAME: Apache Tomcat Tomcat7_r2c

C:UsersGeorgi>

 

2. Create bat script to stop and start Tomcat service

Press keyboard Win-button + R, start notepad type inside:
 

@echo off
sc stop Tomcat7_r2c && sc start Tomcat7_r2c

(MyApp-Tomact-Restart-bat-file-ms-windows-screenshot

Don't be confused from screenshot that I have Tomcat7_MyApp instead of Tomcat7_r2c, but I made screenshot in hurry for another app.
Save the file, somewhere (preferrably) in application folder/bin/  it is best to save it once with bat extension MyApp-Tomcat_Restart.bat and once as MyApp-Tomcat_Restart.xml (XML format file is later needed for import to Task Scheduler which understands .XMLs). The .bat file is good to have because it is useful to somtimes restart Tomcat manually by running it (in case of some sudden Tomcat Appserver occurs even though the auto-restart script).
 

3. Create new Task using command line (cmd.exe)


Task can be created also from command line using following syntax:
 

schtasks /Create [/S [/U [/P [  ]]]]
/XML <xmlfile> /TN <taskname>

Simple way to create a new Windows task is shown in below command, it will set my Tomcat Restart script to run everyday in 05:00 early morning when no employees are using the system:

schTasks /Create /SC DAILY /TN "My Task" /TR "C:UsersGeorgiDesktopmyApp-Tomcat_Restart.bat" /ST 05:00
SUCCESS: The scheduled task "Tomcat Restart Task" has successfully been created.

 

import-new-windows-task-scheduler-task-from-command-line-windows-add-new-cronjob-command-screenshot


4. Create / Import new Windows "Cron" job 

Alternative way is to use Task Scheduler GUI frontend and create new (Basic Task) or  import just created script

To run Windows Task Scheduler from comamnd line :
 

Taskschd.msc

taskschd_windows-run-from-command-line-screenshot

To import already existing .XML formatted file for Task scheduler, right click on the Task Scheduler (Local) and select Import task

task-scheduler-local-task-import-microsoft-2008-r2-windows-screenshot

Import the myApp-Tomcat_Restart.XML previously created file

task-scheduler-import-tomcat-restart-xml-file-windows-server-2008-r2-screenshot

Adjust settings to suit your needs, but what change atleast:

  •         the path to the myApp-Tomcat_Restart.bat file in Actions tab
  •         the Local User account with which script will be running (administrator) in General tab

Task-Scheduler-windows-general-local-user-account-with-which-task-will-be-running

After making all changes you will be prompted for server Administrator account password 

5. check existing Win Cron job from command line

To see the configured (Scheduled Tasks) in command line mode with a command:

Schtasks.exe

schtasks-windows-equivalent-command-to-linux-unix-crontab-screenshot

The command is Windows equivalent to UNIX / Linux's crontab, e.g.:

crontab -u root -l


6. Delete existing Windows Task Job from Command line

If you happen to need to delete just created task or any other task from command line (Assuming that you know the previously created task name), use cmd:

C:>schtasks /Delete /TN "Tomcat Restart Task"
WARNING: Are you sure you want to remove the task "Tomcat Restart Task" (Y/N)? y

SUCCESS: The scheduled task "Tomcat Restart Task" was successfully deleted.


Task completed, Tomcat will auto-restart on Windows host at your scheduled time. Feedback is mostly welcome 🙂
Enjoy