Its not very common, but sometimes it happens you have to crack some downloaded file from thepiratebay.com or some other big torrent tracker. An example scenario would be downloading a huge words dictionary (a rainbow tables) dictionary etc., which was protected by the author with a password and zipped.
Fortunately Mark Lehmann developed a software called fcrackzip which is capable of brute forcing zip protected file passwords straight on UNIX like operating systems (GNU/Linux, FreeBSD).
fcrackzip is available from package repositories on Debian and Ubuntu Linuces to install via apt:
linux:~# apt-get install frackzip
...
fcrackzip is also available on FreeBSD via the ports tree and can be installed with:
freebsd# cd /usr/ports/security/fcrackzip
freebsd# make install cleam
On Debian it's worthy to have a quick look on the README file:
linux:~# cat /usr/share/doc/fcrackzip/READMESee fcrackzip.txt (which is derived from the manpage), or fcrackzip.html
There is a web page with more information at http://lehmann.home.ml.org/fcrackzip.html or http://www.goof.com/pcg/marc/fcrackzip.html
A sample password-protected .zip file is included as "noradi.zip". It's password has 6 lower case characters, and fcrackzip will find it (and a number of false positives) with
fcrackzip -b -c a -p aaaaaa ./noradi.zip
which will take between one and thirty minutes on typical machines.
To find out which of these passwords is the right one either try them out or use the –use-unzip option.
Marc
Cracking the noradi.zip password protected sample file on my dual core 1.8 ghz box with 2gb, it took 30 seconds.
linux:~# time fcrackzip -u -b -c a -p aaaaaa noradi.zip
PASSWORD FOUND!!!!: pw == noradi
real 0m29.627s user 0m29.530s sys 0m0.064s
Of course the sample set password for noradi.zip is pretty trivial and with more complex passwords, sometimes cracking the password can take up to 30 minutes or an hour and it all depends on the specific case, but at least now we the free software users have a new tool in the growing arsenal of free software programs 😉
Here are the options passed on to the above fcrackzip command:
-u – Try to decompress with the detected possible archive passwords using unzip (This is necessery to precisely find the archive password, otherwise it will just print out a number of possible matching archive passwords and you have to try each of the passwords one by one. Note that this option depends on a working unzip version installed.)
-c a – include all charsets to be tried with the generated passwords
-b – Select brute force mode – Tries all possible combinations of letters specified
-p aaaaaa – init-password string (Look up for a password between the password length 6 characters long)
FCrackZip is partly written in assembler and thus is generally works fast, to reduce the CPU load fcrackzip will put on the processor its also capable of using external words dictionary file by passing it the option:
-D – The file should be in a format one word per line and be preliminary alphabetically sorted with let's say sort
Also fcrackzip supports parallel file brute force, for example if you have 10 zip files protected with passwords it can paralelly try to brute force the pwds.
As of time of writting frackzip reached version 1.0 and seems to be pretty stable. Happy cracking. Just to make sure fcrackzip's source is not lost somewhere in the line in the long future to come, I've created a fcrackzip download mirror here
I've just installed the phpbb forum on a Debian Linux because we needed a goodquick to install communication media in order to improve our internal communication in a student project in Strategic HR we're developing right now in Arnhem Business School.
Here are the exact steps I followed to have a properly it properly instlled:
1. Install the phpbb3 debian package This was pretty straight forward:
debian:~# apt-get install phpbb3
At this point of installation I've faced a dpkg-reconfigure phpbb deb package configuration issue: I was prompted to pass in the credentials for my MySQL password right after I've selected the MySQL as my preferred database back engine. I've feeded my MySQL root password as well as my preferred forum database name, however the database installation failed because, somehow the configuration procedure tried to connect to my MySQL database with the htcheck user. I guess this has to be a bug in the package itself or something from my previous installation misconfigured the way the debian database backend configuration was operating. My assumption is that my previously installed htcheck package or something beforehand I've done right after the htcheck and htcheck-php packages installation.
after the package configuration failed still the package had a status of properly installed when I reviewed it with dpkg I've thought about trying to manually reconfigure it using the dpkg-reconfigure debian command and I gave it a try like that:
debian:~# dpkg-reconfigure phpbb3
This time along with the other fields I've to fill in the ncurses interface I was prompted for a username before the password prompted appeared. Logically I tried to fill in the root as it's my global privileges MySQL allowed user. However that didn't helped at all and again the configuration tried to send the credentials with user htcheck to my MySQL database server. To deal with the situation I had to approach it in the good old manual way.
2. Manually prepare / create the required phpbb forum database
To completet that connected to the MySQL server with the mysql client and created the proper database like so:
debian:~# mysql -u root -p
mysql>
CREATE database phpbb3forum;
3. Use phpmyadmin or the mysql client command line to create a new user for the phpbb forum
Here since adding up the user using the phpmyadmin was a way easier to do I decided to go that route, anyways using the mysql cli is also an option.
From phpmyadmin It's pretty easy to add a new user and grant privileges to a certain database, to do so navigate to the following database:
Privileges -> -> Add a new user ->
Now type your User name: , Host , Password , Re-type password , also for a Host: you have to choose Local from the drop down menu.
Leave the Database for user field empty as we have already previously created our desired database in step 2 of this article
Now press the "Go" button and the user will get created.
Further after choose the Privileges menu right on the bottom of the page once again, select through the checkbox the username you have just created let's say the previously created user is phpbb3
Go to Action (There is a picture with a man and a pencil on the right side of this button
Scroll down to the page part saying Database-specific privileges and in the field Add privileges on the following database: fill in your previosly created database name in our case it's phpbb3forum
and then press the "Go" button once again. A page will appear where you will have to select the exact privileges you would like to grant on the specific selected database. For some simplicity just check all the checkbox to grant as many privilegs to your database as you could. Then again you will have to press the "Go" button and there you go you should have already configured an username and database ready to go with your new phpbb forum.
4. Create a virtualhost if you would like to have the forum as a subdomain or into a separate domain
If you decide to have the forum on a separate sub-domain or domain as I did you will have to add some kind of Virtualhost into either your Apache configuration /etc/apache2/apache2.conf or into where officially the virutualhosts are laid in Debian Linux in /etc/apache2/sites-available I've personally created a new file like for instance /etc/apache2/sites-available/mysubdomain.mydomain.com
Here is an example content of the new Virtualhost:
# Logfiles ErrorLog /var/log/apache2/yourdomain/error.log CustomLog /var/log/apache2/yourdomain/access.log combined # CustomLog /dev/null combined <Directory /usr/share/phpbb3/www/> Options FollowSymLinks MultiViews -Includes ExecCGI AllowOverride All Order allow,deny allow from all </Directory> </VirtualHost>
In above Virtualhost just change the values for ServerAdmin , ServerName , DocumentRoot , ErrorLog , CustomLog and Directory declaration to adjust it to your situation.
5. Restart the Apache webserver for the new Virtualhost to take affect
debian:~# /etc/init.d/apache2 restart
Now accessing your http://mysubdomain.domain.com should display the installed phpbb3 forum The default username and password for your forum you can use straight are:
username: admin password: admin
So far so good you by now have the PHPBB3 forum properly installed and running, however if you try to Register a new user in the forum you will notice that it's impossible because of a terrible ugly message reading:
Sorry but this board is currently unavailable.
I've spend few minutes online to scrape through the forums before I can understand what I have to stop that annoying message from appearing and allow new users to register in the phpbb forum
That's all now your forum will be ready to go and your users can freely register and if the server where the forum is installed has an already running mail server, they will receive an emails with a registration data concerning their new registrations in your new phpbb forum. Cheers and Enjoy your new shiny phpbb Forum 🙂
Here is how to configure a JBL Go Bluetooth (Wireless) speaker and presumably other Bluetooth external speakers to Debian GNU / Linux Wheezy 7 and Ubuntu 14.04 .1. Install following bunch of deb packages
Here it is notable to mention pavucontrol if you have previously played more extensively on GNU / Linux you should have already used if not it is really cozy volume control tool with a lot of tuning options regarding pulseaudio stream server. Considering that like me you're using a GNOME as a desktop environment you will also need gnome-bluetooth package, e.g.:
debian:~#apt-get install gnome-bluetooth
As Pulseaudio is used as a sound streaming server in GNU / Linux (assuming your Debian version is using it you'll also need to have installed pulseaudio-module-bluetooth)
debian:~#apt-get install pulseaudio-module
For Ubuntu 14.04 GNU / Linux users the list of necessery bluetooth packages is a bit longer, if you're on this OS go and install:
Moreover you will need pulseaudio-module-bluetooth deb package installed in order to be able to select the desired sound output.
Next it is time to restart Bluetooth service
debian:~# service bluetooth restart
[ ok ] Stopping bluetooth: rfcomm /usr/sbin/bluetoothd.
[ ok ] Starting bluetooth: bluetoothd rfcomm.
It is also a good idea to restart pulseaudio snd streaming server in order to load the newly installed pulseaudio bluetooth module settings, to do so issue:
debian:~# killall pulseaudio
And try to establish connection from Gnome-Bluetooth to the JBL Go (press the JBL Go bluetooth button) and search from the Linux bluetooth interface, once founded connect it.
Before JBL Go appears to list listable blootooth devices you will also need to run following command:
This command is to connect bluetooth discovered JBL Go device to the audio sink interface.
It is generally idea to add this line also to /etc/rc.local to make the setting permanently executed on every Linux boot.
Now you can launch pavucontrol and hopefully the JBL GO bluetooth speaker should be visible as an option, check out my below screenshot:
In case you further experience issues connecting the Bluetooth Speaker I would recommend to check out this Debian a2dp page at the end of the page are troubleshooting suggestions.
Troubleshooting
Refused to switch profile to a2dp_sink: Not connected
Bluetooth headset is connected, but ALSA/PulseAudio fails to pick up the connected device or there's no device to pick. This happens because GDM captures A2DP sink on session start, as GDM needs pulseaudio in the gdm session for accessibility. For example, the screen reader requires it. See 805414 for some discussion.
Workaround 1: disable pulseaudio in gdm
In order to prevent GDM from capturing the A2DP sink on session start, edit /var/lib/gdm3/.config/pulse/client.conf (or create it, if it doesn't exist):
autospawn = no
daemon-binary = /bin/true
After that you have to grant access to this file to Debian-gdm user:
In order to auto-connect a2dp for some devices, add this to /etc/pulse/default.pa:
load-module module-switch-on-connect
Logout your Desktop environment and restart gdm3 /etc/init.d/gdm3 restart or Reboot the PC and then it should be fine.
Now the sound device (bluetooth headset) should be accessible through pavucontrol and standard audio device manager.
Workaround 2: disable pulseaudio's bluetooth in gdm
The actual solution package maintainers are looking into next is to simply disable the bluetooth sink in the gdm pulseaudio daemon so that it doesn't take over the device. Add this to /var/lib/gdm3/.config/pulse/default.pa:
#!/usr/bin/pulseaudio -nF
#
# load system wide configuration
.include /etc/pulse/default.pa
### unload driver modules for Bluetooth hardware
.ifexists module-bluetooth-policy.so
unload-module module-bluetooth-policy
.endif
.ifexists module-bluetooth-discover.so
unload-module module-bluetooth-discover
.endif
Though this article explains how to connect a bluetooth speaker connecting Bluetooth Speaker to GNU / Linux is done in analogous way
How to install Toshiba L40 B14 Wireless Adapter ( ID 0bda:8197 Realtek Semiconductor Corp. RTL8187B) on Ubuntu and Debian Linux I've been struggling for more than 10 hours to fix up issues on a Ubuntu Maverick-Meerkaat with a rtl8187B Wireless Adapter
The RTL8187B almost drove me mad. I could see the wlan0 which meant the kernel is detecting the device, I could even bring it up with ifconfig wlan0 up , however when I tried it in gnome's network-manager or wicd the wireless networks were not showing up.
Trying to scan for networks using the commands:
ubuntu:~# iwlist wlan0 scan
was also unsuccesful, trying to bring up and down the wireless wlan0 interface with:
ubuntu:~# iwconfig wlan0 up
or
ubuntu:~# iwconfig wlan0 down
Both returned the error: iwconfig: unknown command "up" and iwconfig: unknown command "down"
Running simply iwconfig was properly returning information about my Wireless Interface wlan0 :
wlan0 IEEE 802.11bg ESSID:off/any
Mode:Managed Access Point: Not-Associated Tx-Power=20 dBm
Retry long limit:7 RTS thr:off Fragment thr:off
Encryption key:off
Power Management:off
The exact information I could get about the wireless device was via the command:
ubuntu:~# lsusb | grep realtek
Bus 001 Device 002: ID 0bda:8197 Realtek Semiconductor Corp. RTL8187B Wireless Adapter
Trying manually to scan for wireless networks from console or gnome-terminal with command returned also the below weird results:
I read a bunch of documentation online concerning the wireless card troubles on Ubuntu, Gentoo, Debian etc.
Just few of all the resources I've read and tried are:
http://rtl-wifi.sourceforge.net/wiki/Main_Page (Returning empty page already a lot resource) http://rtl8187b.sourceforge.net (A fork of rtl-wifi.sourceforge.net which is still available though it was not usable)
Some of the other resources which most of the people recommended as a way to properly install the RTL8187B wireless driver on linux was located on the website:
http://datanorth.net/~cuervo/rtl8187b/ (Trying to access this page returned a 404 error e.g. this page is no-longer usable)
I found even a webpage in Ubuntu Help which claimed to explain how to properly install and configure the RTL8187B wireless driver on which is below:
The questionable file which was claimed to properly be able to make the Realtek Semiconductor Corp. RTL8187B Wireless Adapter to work out was called rl8187b-modified-804.tar.gz. I've made a mirror of rtl8187b-modified-804.tar.gz is here
None of the driver archives rtl8187b-modified-dist.tar.gz and rl8187b-modified-804.tar.gz that was supposed to make the Toshiba L40 realtek wireless to work out, after compiling and installing the drivers from source worked out …
Both archives produced plenty of error messages and it seems on newer kernels like the one on this notebook:
Linux zlatina 2.6.35-28-generic #50-Ubuntu SMP Fri Mar 18 19:00:26 UTC 2011 i686 GNU/Linux, they're no longer usable.
The compile errors I got when I tried compiling the rtl8187b driver provided by the archive rtl8187b-modified-dist were:
root@ubuntu:/home/zlatina/rtl8187b-modified# sh makedrv rm -fr *.mod.c *.mod *.o .*.cmd *.mod.* *.ko *.o *~ make -C /lib/modules/2.6.35-28-generic/build M=/home/zlatina/rtl8187b-modified/ieee80211 CC=gcc modules make[1]: Entering directory `/usr/src/linux-headers-2.6.35-28-generic' scripts/Makefile.build:49: *** CFLAGS was changed in "/home/zlatina/rtl8187b-modified/ieee80211/Makefile". Fix it to use EXTRA_CFLAGS. Stop. make[1]: *** [_module_/home/zlatina/rtl8187b-modified/ieee80211] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-2.6.35-28-generic' make: *** [modules] Error 2 rm -fr *.mod.c *.mod *.o .*.cmd *.ko *~ make -C /lib/modules/2.6.35-28-generic/build M=/home/zlatina/rtl8187b-modified/rtl8187 CC=gcc modules make[1]: Entering directory `/usr/src/linux-headers-2.6.35-28-generic' scripts/Makefile.build:49: *** CFLAGS was changed in "/home/zlatina/rtl8187b-modified/rtl8187/Makefile". Fix it to use EXTRA_CFLAGS. Stop. make[1]: *** [_module_/home/zlatina/rtl8187b-modified/rtl8187] Error 2 make[1]: Leaving directory `/usr/src/linux-headers-2.6.35-28-generic' make: *** [modules] Error 2 root@ubuntu:/home/zlatina/rtl8187b-modified#
I tried a number of fix ups hoping to solve the compile error messages, but my efforts were useless, as it seems many things has changed in newer Ubuntu versions and they could no longer be compiled.
To finally load the Windows XP RTL8187B driver on the Ubuntu I used again ndiswrapper:
root@ubuntu:~# ndiswrapper -m
Further on I used the ndisgtk graphical ndiswrapper interface to once again test if the Windows driver is working on the Ubuntu and it seemed like it is working, however still my wicd was unable to find any wireless network ….
There were many online documentation which claimed that the driver for rtl8187b works out of the box on newer kernel releases (kernel versions > 2.6.24)
Finally I found out there is a driver which is a default one with the Ubuntu e.g. rtl8187.ko , I proceeded and loaded the module:
root@ubuntu:~# modprobe rtl8187
I also decided to check out if the hardware switch button of the Toshiba Satellite L40 notebook is not switched off and guess what ?! The Wireless ON/OFF button was switched OFF!!! OMG …
I switched on the button and wicd immediately started showing up the wireless networks …
To make the rtl8187 module load on Ubuntu boot up, I had to issue the command:
root@ubuntu:~# echo 'rtl8187' >> /etc/modules
Voila after all this struggle the wireless card is working now, it's sad I had to loose about 10 hours of time until I come with the simple solution of using the default provided ubuntu driver rtl8187 , what is strange is how comes that it does not load up automatically.
!!! IMPORTANT UPDATE COMMENT INFO DETECTOR IS NO LONGER SUPPORTED (IS OBSOLETE) AND THE COUNTRY FLAGS AND OPERATING SYSTEM WILL BE NOT SHOWING INSTEAD,
I've come across a nice WordPress plugin that displays country flag, operating system and web browser used in each of posted comments blog comments. Its really nice plugin, since it adds some transperancy and colorfulness to each of blog comments 😉 here is a screenshot of my blog with Comments Info Detector "in action":
Comments Info Detector as of time of writting is at stable ver 1.0.5. The plugin installation and configuration is very easy as with most other WP plugins. To install the plugin;
To enable the plugin Navigate to; Plugins -> Inactive -> Comment Info Detector (Activate)
After having enabled the plugin as a last 3rd step it has to be configured.
3. Configure comment-info-detector wp plugin
By default the plugin is disabled. To change it to enabled (configure it) by navigating to:
Settings -> Comments Info Detector
Next a a page will appear with variout fields and web forms, where stuff can be changed. Here almost all of it should be left as it is the only change should be in the drop down menus near the end of the page:
Display Country Flags Automatically(Change No to Yes) Display Web Browsers and OS Automatically (Change No to Yes
After the two menus are set to "Yes" and pressing on Save Changes the plugin is enabled it will immediately start showing information inside each comment the GeoIP country location flag of the person who commented as well as OS type and Web Browser 🙂
If you happen to have SSH account on a Dedicated server or a VPS that runs on some kind of UNIX like OS such as Linux / *BSD or just a friends Mac OS notebook and you don't want your HTTP traffic to be sniffed (spyed) by your local ISP, e.g. you want to get some kind of simple Web browsing anonimity on the Internet, easier alternative to using SSH Dynamic Tunnel to Proxify encrypted traffic is to just run a Proxy Cache server on remote *nix host to which you have access and configure your browser to use the host as proxy. Besides enhanced traffic privacy other advantage of running a Proxy server is of course the fact that using proxy server cache significantly improve page speed opening times, optimize web-sites content delivery saving you a lot of bandwidth and off-loads remote sites because already cached content at the proxy is served directly from the proxy server instead of each time requested from the server. Those admins who remember "the dawn of mass internet use" should remember that a lot of the small and middle sized internet providers were into the habit to use Squid transparent proxy with huge Disk Cache in order to speed-up their customers internet page opening times and thus offer a superior service and save money for themselves since the ISPs were also paying not for a bandwidth to end providers but for used overall Traffic.
Historically I've personally used TinyProxy a lot to "obfuscate" my traffic (hide my originating host IP) and to save-bandwidth cause in late 1990's I paid my internet bills based on used traffic and starting a TinyProxy saved me network traffic and hence cut my Internet bill but even today it is great to optimize web traffic.
Usually for a single or few persons proxy it is not worthy to use "Full Featured" Complex Caching servers such as Squid Cachebecause installing and configuring one has plenty of dependency packages and a lot of time to spend in configuring (dealing with squid's cryptic squid.conf) it is much better and easier to use a light-weight HTTP/HTTPS proxy service (daemon) such as TinyProxy if using Tor Project (Anonymity Online) Network to protect against network surveillance / traffic analysisPrivoxy.
Below is TinyProxy package description from Debian
apt-cache show tinyproxy|grep -i desc -A 5 Description: A lightweight, non-caching, optionally anonymizing http proxy An anonymizing http proxy which is very light on system resources, ideal for smaller networks and similar situations where other proxies (such as Squid) may be overkill and/or a security risk. Tinyproxy can also be configured to anonymize http requests (allowing for exceptions on a per-header basis).
You might want to change some of below default values:
# User and Group with which TinyProxy will be running User nobody Group nogroup # Default proxy listen port Port 8888 # Specifies IP (interface) to be used for outgoing proxy host connections Bind 192.168.0.1 # To which IP address (interface) tinyproxy will listen for connections. If uncommented it will listen to all available network interfaces Listen 192.168.0.1 # Seconds after which connection will be terminated Timeout 600 # Where proxy queries will be logged (very useful to keep a history for yourself on pages you have visted), sometimes useful if you wipe out browser cache Logfile "/var/log/tinyproxy/tinyproxy.log" # How many clients (connections) can be made to tinyproxy for one client (personal use) 100 is a good value MaxClients 100 # This two values are settings on how many (minumum and maximum) tinyproxy instances will listen for connection from remote hosts – this should be familiar to people who configured Apache webserver, note that tinyproxy uses process fork and not threads MinSpareServers 5 MaxSpareServers 20 # This is how many sites connections the proxy will listen to # if you open 20 sites in 20 tabs which will refresh make here 20 # you might need to raise this if you have 10 clients StartServers 10 # hosts to allow connection to proxy server from (Be careful to allow only access from your network otherwise you might end up being an open proxy), allowing some hax0r to do something terrible proxying through you Allow 127.0.0.1 #The Log Level – "Error" is generally Okay, who may like more information can take Notice – Connect and Info LogLevel Error # pidfile location PidFile "/var/run/tinyproxy/tinyproxy.pid" # The "Via" header is required by the HTTP RFC, but using the real host name # is a security concern. If the following directive is enabled, the string # supplied will be used as the host name in the Via header; otherwise, the # server's host name will be used. # ViaProxyName "tinyproxy
To install TinyProxy on FreeBSD
freebsd# cd /usr/ports/www/tinyproxy freebsd# make install distclean
If you need to change the default TinyProxy port from port 8888 to something else for security reasons edit:
vim /usr/local/etc/tinyproxy.conf
Find port and change is to lets say 7000 or whatever network port is good for you
If you're concerned about transferred data security between your client host (Desktop) machine and remote installed Proxy server that your ISP or some malicious guy could sniff your website login credentials using Man in the Middle attack, its also a good idea to use TinyProxy together with SSH Tunnel, that's pretty easy if you have an SSH client on your machine (if you're on a Mac OS X).
The default behaviour of lynx – console text browser on Linuces, BSD and other free OSes is to always ask, for the accept cookies prompt once an internet web page is opened that requires browser cookies to be enabled.
I should admin, having this "secure by default" (always ask for new cookies) behaviour in lynx was a good practice from a security point of view.
Another reason, why this cookies prompt is enabled by default is back in the days, when lynx was actively developed by programmers the websites with cookies support was not that many and even cookies was mostly required for user/pass authentication (all those who still remember this days the websites that requires authentication was a way less than today) … With this said the current continuing security cautious behaviour in the browser, left from its old days is understandable.
However I personally sometimes, need to use lynx more frequently and this behaviour of always opening a new website in text mode in console to prompts me for a cookie suddenly becomes a big waste of time if you use lynx to browser more than few sites. Hence I decided to change the default way lynx handles cookies and make them enabled by default instead. Actually even in the past, when I was mainly using internet in console on every new server or home Linux install, I was again making the cookies to be permanently accepted. Everyone who used lynx a few times already knows its "annoying" to all time accept cookie prompts … This provoked me to write this short article to explain how enabling of constant cookie accepting in lynx is done
To enable the persistent cookies in lynx, one needs to edit lynx.cfg on different GNU / Linux and BSD* distributions lynx.cfg is located in different directory.
Most of the lynx.cfg usual locations are /etc/lynx/lynx.cfg or /etc/lynx.cfg as of time of writting this post in Debian Squeeze GNU / Linux the lynx.cfg is located in /etc/lynx-cur/lynx.cfg, whether for FreeBSD / NetBSD / OpenBSD users the file is located in /usr/local/etc/lynx.cfg
What I did to allow all cookies is open lynx.cfg in vim edit and change the following lines:
a)
#FORCE_SSL_COOKIES_SECURE:FALSE
with
FORCE_SSL_COOKIES_SECURE:TRUE
b)
#SET_COOKIES:TRUE
uncomment it to:
SET_COOKIES:TRUE
c) next, change
ACCEPT_ALL_COOKIES:FALSE
ACCEPT_ALL_COOKIES:TRUE
Onwards opening any website with lynx auto-accepts the cookies.
For people who care about there security (who still browse in console (surely not many anymore)), permanently allowing the cookies is not a good idea. But for those who are ready to drop off little security for convenience its ok.
Everyone who used Linux is probably familiar with wget or has used this handy download console tools at least thousand of times. Not so many Desktop GNU / Linux users like Ubuntu and Fedora Linux users had tried using wget to do something more than single files download. Actually wget is not so popular as it used to be in earlier linux days. I've noticed the tendency for newer Linux users to prefer using curl (I don't know why).
With all said I'm sure there is plenty of Linux users curious on how a website mirror can be made through wget. This article will briefly suggest few ways to do website mirroring on linux / bsd as wget is both available on those two free operating systems.
1. Most Simple exact mirror copy of website
The most basic use of wget's mirror capabilities is by using wget's -mirror argument:
Creating a mirror like this is not a very good practice, as the links of the mirrored pages will still link to external URLs. In other words link URL will not pointing to your local copy and therefore if you're not connected to the internet and try to browse random links of the webpage you will end up with many links which are not opening because you don't have internet connection.
2. Mirroring with rewritting links to point to localhost and in between download page delay
Making mirror with wget can put an heavy load on the remote server as it fetches the files as quick as the bandwidth allows it. On heavy servers rapid downloads with wget can significantly reduce the download server responce time. Even on a some high-loaded servers it can cause the server to hang completely. Hence mirroring pages with wget without explicity setting delay in between each page download, could be considered by remote server as a kind of DoS – (denial of service) attack. Even some site administrators have already set firewall rules or web server modules configured like Apache mod_security which filter requests to IPs which are doing too frequent HTTP GET /POST requests to the web server. To make wget delay with a 10 seconds download between mirrored pages use:
The -mk stands for -m/-mirror and -k / shortcut argument for –convert-links (make links point locally), –random-wait tells wget to make random waits between o and 10 seconds between each page download request.
Some websites has a robots.txt which restricts content download with clients like wget, curl or even prohibits, crawlers to download their website pages completely.
/robots.txt restrictions are not a problem as wget has an option to disable robots.txt checking when downloading. Getting around the robots.txt restrictions with wget is possible through -e robots=off option. For instance if you want to make a local mirror copy of the whole sub-directory with all links and do it with a delay of 10 seconds between each consequential page request without reading at all the robots.txt allow/forbid rules:
4. Mirror website which is prohibiting Download managers like flashget, getright, go!zilla etc.
Sometimes when try to use wget to make a mirror copy of an entire site domain subdirectory or the root site domain, you get an error similar to:
Sorry, but the download manager you are using to view this site is not supported. We do not support use of such download managers as flashget, go!zilla, or getright
This message is produced by the site dynamic generation language PHP / ASP / JSP etc. used, as the website code is written to check on the browser UserAgent sent. wget's default sent UserAgent to the remote webserver is: Wget/1.11.4
As this is not a common desktop browser useragent many webmasters configure their websites to only accept well known established desktop browser useragents sent by client browsers. Here are few typical user agents which identify a desktop browser:
Mozilla/5.0 (Windows NT 6.1; rv:6.0) Gecko/20110814 Firefox/6.0
Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0
Mozilla/6.0 (Macintosh; I; Intel Mac OS X 11_7_9; de-LI; rv:1.9b4) Gecko/2012010317 Firefox/10.0a4
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.2a1pre) Gecko/20110324 Firefox/4.2a1pre
etc. etc.
If you're trying to mirror a website which has implied some kind of useragent restriction based on some "valid" useragent, wget has the -U option enabling you to fake the useragent.
If you get the Sorry but the download manager you are using to view this site is not supported , fake / change wget's UserAgent with cmd:
For the sake of some wget anonimity – to make wget permanently hide its user agent and pretend like a Mozilla Firefox running on MS Windows XP use .wgetrc like this in home directory.
5. Make a complete mirror of a website under a domain name
To retrieve complete working copy of a site with wget a good way is like so:
Where the arguments meaning is: -r – Retrieve recursively -k – Convert the links in documents to make them suitable for local viewing -p – Download everything (inline images, sounds and referenced stylesheets etc.) -N – Turn on time-stamping -l5 – Specify recursion maximum depth level of 5
6. Make a dynamic pages static site mirror, by converting CGI, ASP, PHP etc. to HTML for offline browsing
It is often websites pages are ending in a .php / .asp / .cgi … extensions. An example of what I mean is for instance the URL http://php.net/manual/en/tutorial.php. You see the url page is tutorial.php once mirrored with wget the local copy will also end up in .php and therefore will not be suitable for local browsing as .php extension is not understood how to interpret by the local browser. Therefore to copy website with a non-html extension and make it offline browsable in HTML there is the –html-extension option e.g.:
A good practice in mirror making is to set a download limit rate. Setting such rate is both good for UP and DOWN side (the local host where downloading and remote server). download-limit is also useful when mirroring websites consisting of many enormous files (documental movies, some music etc.). To set a download limit to add –limit-rate= option. Passing by to wget –limit-rate=200K would limit download speed to 200KB.
Other useful thing to assure wget has made an accurate mirror is wget logging. To use it pass -o ./my_mirror.log to wget.
I've gathered a collection of 15 Audio and Video songs dedicated to the Free Software / Open Source movement . All of the songs are based on the The Free Software Song Anthem written by Richard Mathew Stallman in the year 1991. The motive of the song is a Traditional Bulgarian song called Sadi Moma Bqla Loza – translated to bulgarian to something like Maid is Planting white Vines The original Free Software Song symbolizes all free software and the Free Software Movement and GNU and is in the Bulgarian unique / specific folk rhythm of 7 / 8 beats .
Most of the songs which I post hereby could also be found and downloaded from GNU's official Free Software Song page However some of the songs were only available from Youtube in the non-free format Flash Video (flv) . Hence, since the songs were dedicated to Free Software and apparently were being spread in a non-free format they either was missing any licensing or licensed under GFDL – free music / art GNU like license. To fix up this irragularity and add some freedom in terms of audio format of spreading, I've downloaded them and used ffmpeg2theora to convert the songs to the Free / Open Standard format Ogg Vorbis I'm quite sure that many people, who use Ubuntu or Linux Mint are pretty much unfamiliar with the Free Software Songs existence, also many people most likely have never heard the Free Software Songs or even those who heard it have rarely heard more than 2 or 3 of the song variations. Hereby, I'm sure many people who are lovers of Free Software will highly benefit and get inspired to continue in the Free Software by listening to these post shared little Free Software Song Collection .
The covers of the Original version publicly sang by Richard Stallman are in different musical genres, some of the song performances are in Folklore, played on Piano other covers are performed by musical bands in pop / punk en popular music styles, there are one person performances, cheerful christmas like soundings, 8 bit free software song, Metal free software variations etc. In the collection I've included also few other nice songs which are propaganda on free software, even though not a cover of the Free Software Song , I found them myself worthy to be included in the collection..:
Herein you can download or listen all the Free Software Songs version (Enjoyment is guaranteed! 😉 ):
How to show country flag, web browser type and Operating System in WordPress Comments
Wednesday, February 15th, 2012!!! IMPORTANT UPDATE COMMENT INFO DETECTOR IS NO LONGER SUPPORTED (IS OBSOLETE) AND THE COUNTRY FLAGS AND OPERATING SYSTEM WILL BE NOT SHOWING INSTEAD,
!!!! TO MAKE THE COUNTRY FLAGS AND OS WP FUNCTIONALITY WORK AGAIN YOU WILL NEED TO INSTALL WP-USERAGENT !!!
I've come across a nice WordPress plugin that displays country flag, operating system and web browser used in each of posted comments blog comments.
Its really nice plugin, since it adds some transperancy and colorfulness to each of blog comments 😉
here is a screenshot of my blog with Comments Info Detector "in action":
Comments Info Detector as of time of writting is at stable ver 1.0.5.
The plugin installation and configuration is very easy as with most other WP plugins. To install the plugin;
1. Download and unzip Comments Info Detector
linux:/var/www/blog:# cd wp-content/pluginslinux:/var/www/blog/wp-content/plugins:# wget http://downloads.wordpress.org/plugin/comment-info-detector.zip
...
linux:/var/www/blog/wp-content/plugins:# unzip comment-info-detector.zip
...
Just for the sake of preservation of history, I've made a mirror of comments-info-detector 1.0.5 wp plugin for download here
2. Activate Comment-Info-Detector
To enable the plugin Navigate to;
Plugins -> Inactive -> Comment Info Detector (Activate)After having enabled the plugin as a last 3rd step it has to be configured.
3. Configure comment-info-detector wp plugin
By default the plugin is disabled. To change it to enabled (configure it) by navigating to:
Settings -> Comments Info DetectorNext a a page will appear with variout fields and web forms, where stuff can be changed. Here almost all of it should be left as it is the only change should be in the drop down menus near the end of the page:
Display Country Flags Automatically (Change No to Yes)Display Web Browsers and OS Automatically (Change No to Yes
After the two menus are set to "Yes" and pressing on Save Changes the plugin is enabled it will immediately start showing information inside each comment the GeoIP country location flag of the person who commented as well as OS type and Web Browser 🙂
Tags: action, Auto, blog, Browsers, change, Comment, Comments, configured, country flag, country location, Detector, Display, downloads, Draft, drop, drop down menus, flag web, How to, Inactive, information, installation, Linux, location, mirror, operating system, os type, page, person, plugin, plugin installation, quot, sake, Save Changes, screenshot, show, Stable, time, transperancy, type, unzip, web browser type, web browsers, web forms, wget, Wordpress, Wordpress Comments, writting, www, zip linux
Posted in Web and CMS, Wordpress | 1 Comment »