Posts Tagged ‘variables’

Weblogic – How to change / remove IP/hostname quick and dirty howto

Wednesday, March 11th, 2015

Oracle-Weblogic-Server-logo-how-to-change-ip-hostname-weblogic-quick-and-dirty-howto

This is just quick & dirty doc on how to change/remove IP/host on Oracle WebLogic Application server

– In logs the Error message will be message like:

 

<Oct 21, 2013 1:06:51 AM SGT> <Warning> <Security> <BEA-090504> <Certificate chain received from cluster2.yourdomain.com – 192.168.1.41 failed hostname verification check. Certificate contained cluster1.yourdomain.com but check expected cluster2.yourdomain.com>

 

 

Solution:

On web console – change/remove IP/hostname

 

As root / admin supersuser:

 

– Stop Weblogic Webserver 

As this is RHEL Linux, to stop WLS use standard init script start / stop service command

 

service wls stop

 

– As Application user create directory where new key will be created

 

mkdir /home/uwls11pp/tmp_key
cd /home/uwls11pp/tmp_key

 

– Make backup of current JKS (Keystore File)

 

cp /WLS/app/oracle/wls1036/wlserver_10.3/server/lib/DemoIdentity.jks /WLS/app/oracle/wls1036/wlserver_10.3/server/lib/DemoIdentity.jks_11032015

 

– Execute set env . script

 

/WLS/app/oracle/wls1036/wlserver_10.3/server/bin/setWLSEnv.sh

 

– Copy & paste output from script above and export variables

 

export CLASSPATH;
export PATH;

 

– Check old certificate in keystore

 

/WLS/app/oracle/jdk1.7.0_25/bin/keytool -list -v -keystore /WLS/app/oracle/wls1036/wlserver_10.3/server/lib/DemoIdentity.jks  -storepass DemoIdentityKeyStorePassPhrase

 

– Delete old Weblogic keystore JKS file

 

/WLS/app/oracle/jdk1.7.0_25/bin/keytool -delete -alias demoidentity -keystore /WLS/app/oracle/wls1036/wlserver_10.3/server/lib/DemoIdentity.jks -storepass DemoIdentityKeyStorePassPhrase

 

– Check wether proper Java version is used

 

java -version

 

– Get hostname from hosts file

 

cat /etc/hosts

 

#Replace weblogic1 with your FQDN (Fully Qualified Domain Name) – this step will create new certificate with new hostname

 

java utils.CertGen -cn weblogic1 -keyfilepass DemoIdentityPassPhrase -certfile newcert -keyfile newkey

 

#Import certificate to “official” keystore

 

java utils.ImportPrivateKey -keystore /WLS/app/oracle/wls1036/wlserver_10.3/server/lib/DemoIdentity.jks -storepass DemoIdentityKeyStorePassPhrase -keyfile newkey.pem -keyfilepass DemoIdentityPassPhrase -certfile newcert.pem -alias demoidentity

 

#Recheck once again if correct certificate is in use

 

/WLS/app/oracle/jdk1.7.0_25/bin/keytool -list -v -keystore /WLS/app/oracle/wls1036/wlserver_10.3/server/lib/DemoIdentity.jks  -storepass DemoIdentityKeyStorePassPhrase


– Finally issue as root user restart Weblogic server again

 

 

service wls start

StatusNet – Start your own hosted microblogging twitter like social network on Debian GNU / Linux

Monday, July 14th, 2014

build-your-own-microblogging-service-like-twitter-on-debian-linux-Status.net-logo
I like experimenting with free and open source projects providing social networking capabilities like twitter and facebook. Historically I have run my own social network with Elgg – Open Source Social Network Engine. I had a very positive impression from Elgg as a social engine as, there are plenty of plugins and one can use Elgg to run free alternative to very basic equivalent of facebook, problem with Elgg I had however is if is not all the time monitored it quickly fills up with spam and besides that I found it to be still buggy and not easy to update.
The other social network free software I heard of isBuddyPress which I recently installed with Multisite (MuSite) enabled.

Since BuddyPress is WordPress based and it supports all the nice wordpress plugins, my impression is social networking based on wordpress behaves much more stable and since there is Akismet for WordPress, the amount of spammer registrations is much lower than with Elgg.

Recently I started blogging much more actively and I realized everyday I learn and read too much interesting articles and I don't log them anywhere and thought I need a way besides twitter to keep flashy notes of what I'm doing reading, learning in a short notes. I don't want to use Twitter on purpose, because I don't want to improve twitter's site SEO with adding my own stuff on their website but I want to keep my notes on my own local hosted server.

As I didn't wanted to loose time with Complexity of Elgg anymore and wanted to try to something new and I know the open source microblogging social network (Twitter Equivalent) – identi.ca runs StatusNet – Free and Open Source Social software. StatusNet is well known under the motto of "Decentralized Twitter"

screenshot-status-net-microblogging-twitter-like-free-software-network

I took the time to grab it and install it to my home brew machine pc-freak.net. If you haven't seen StatusNet so far – you can check out demo of my installation here – registration is not freely opened because, i don't want spammers to break in, however if you want to give a try drop me a mail or comment below and I will open access for you ..

There is no native statusnet package for Debian Linux (as I'm running Debian) so to install it, I've grabbed statusnet.

To install StatusNet, everything was pretty straight forward and literally following instructionsf rom INSTALL file, i.e.:

# status.example.com maps to /var/www/status/
cd /var/www/status/
wget http://status.net/statusnet-0.9.6.tar.gz
tar -xzf statusnet-0.9.6.tar.gz --strip-components=1
rm statusnet-0.9.6.tar.gz
cd ..
chgrp www-data status/
chmod g+w status/
cd status/
chmod a+w avatar/ background/ file/

mysqladmin -u "root" -p "sql-root-password" create statusnet
mysql -u root -p
GRANT ALL on statusnet.* TO 'statusnetuser'@'localhost' IDENTIFIED BY 'statusnet-secret-password';

To Change default behaviour of URls to be more SEO friendly and not to show .php in URL (e.g. add so called fancy URLs – described in INSTALL):

cp htaccess.sample .htaccess


Then had to configure a VirtualHost under a subdomain http://statusnet.yourdomain.com/install.php or you can alternatively install and access it in browser via http://your-domain.com/status/install.php

An important note to open here is you have to set the URLs via which status.net will be accessed further before proceeding with the install, if you will be using HTTPS here is time to configure it and test it before proceeding with install …  Just be warned that if you don't set the URLs properly now and try to modify them further you will get a lot of issues hard to solve which will cost you a lot of time and nervee ..

If you want to enable twitter bridging in Statusnet you will need to get Twitter consumer and secret keys, to get that you have to create new application on https://dev.twitter.com/apps afterwards you will be taken to a page containing Consumer Key & Consumer Secret string.
StatusNet installation will auto generate config.php, you can further edit it manually with text editor. Content of my current statusnet config.php is here.

Most important options to change are:

$config['daemon']['user'] = 'www-data';
$config['daemon']['group'] = 'www-data';

www-data is user with which Apache is running by default on Debian Linux.

$config['site']['profile'] = 'private';

By default Status.Net will be set to run as private – e.g. it will be fitted for priv. use – messages posted will not publicly be visible. Here the possible options to choose between are:

$config['site']['profile'] = 'private';
$config['site']['profile'] = 'community';
$config['site']['profile'] = 'singleuser';
$config['site']['profile'] = 'public';

singleuser is pretty self explanatory, setting public option will open registration for any user on the internet – probably your network will quickly be filled with spam – so beware with this option. community will make statusnet publicly visible but, registration will only possible via use creation / invitation to join the network from admin.

vi /var/www/status/config.php
$config['site']['fancy'] = true;

Then to enable twitter to statusnet bridge add to config.php

vi /var/www/status/config.php

addPlugin('TwitterBridge');
$config['twitter']['enabled'] = true;
$config['twitterimport']['enabled'] = true;
$config['avatar']['path'] = '/avatar';
$config['twitter']['consumer_key'] = 'XXXXXXXX';
$config['twitter']['consumer_secret'] = 'XXXXXXXX';
# disable sharing location by default
$config['location']['sharedefault'] = 'false';

Notice, I decided to disable statusnet sharedefault folder, because i don't have a lot of free space to provide to users. If you want to let users be allowed to share files (you the space for that), you might want to set a maximum quote of uploaded files (to prevent your webserver from being DoSed – for example by too many huge uploads), here is some reasonable settings:
 

$config['attachments']['file_quota'] = 7000000;
$config['attachments']['thumb_width'] = 400;
$config['attachments']['thumb_height'] = 300;

 

If you want to get the best out of performance of your new statusnet microblogging service, after each modification of config.php be sure to run:

 

php scripts/checkschema.php

Running checkschema.php is also useful, to check whether adding new plugins to check whether plugin will not throw an error.

Here is some extra useful config.php plugins to enable:
 

addPlugin('Gravatar', array());
#addPlugin('Textile');
addPlugin('InfiniteScroll');
addPlugin('Realtime');
addPlugin('Blog');
addPlugin('SiteNoticeInSidebar');
addPlugin('WikiHashtags');
addPlugin('SubMirror');
addPlugin('LinkPreview');
addPlugin('Blacklist');


If you expect to have quickly growing base of users it is recommended to also check out whether your MySQL is tuned with mysqltuner and optimize it for performance

Another useful think you would like to do is to increased the number of Statusnet avatars in the 'following' – 'followers' – 'groups' sections on my profile page by editing

lib/groupminilist.php

and

lib/profileminilist.php

line 36 in both files.
To get the full list of possible variables that can be set in config.php run in terminal:

 php scripts/setconfig.php -a

If you happen to encounter some oddities and issues with StatusNet after installation, this is most likely to some PHP hardering on compile time or some PHP.ini functions disabled for security or some missing component to install which is described as a prerequirement in statusnet INSTALL file

to debug the issues enable statusnet logging by adding in config.php

$config['site']['logdebug'] = true;
$config['site']['logfile'] = '/var/log/statusnet.log';

By default logs produced will be quite verbose and there will be plenty of information you will probably not need and will lead to a lot of "noise", to get around this, there is the LogFilter Plugin for some reasonable logging use in config.php:

addPlugin('LogFilter', array( 'priority' => array(LOG_ERR => true,
LOG_INFO => true,
LOG_DEBUG => false),
'regex' => array('/About to push/i' => false,
'/twitter/i' => false,
'/Successfully handled item/i' => false)
));

If you want tokeep log of statusnet it is a good idea to rorate logs periodically to keep them from growing too big, to do this create in /etc/logrotate.d new files /etc/logrotate.d/statusnet with following content:

/var/log/statusnet/*.log {
daily
missingok
rotate 7
compress
delaycompress
notifempty
create 770 www-data www-data
sharedscripts
postrotate
/path/to/statusnet/scripts/stopdaemons.sh > /dev/null
/path/to/statusnet/scripts/startdaemons.sh > /dev/null
endscript
}

You will probably want to to add new Links, next to StatusNet main navigation links for logged in users, this is possible by adding new line to

lib/primarynav.php

and

lib/secondarynav.php

You will have to add a PHP context like:

 $this->action->menuItem('http://www.pc-freak.net/blog/',
                              _m('MENU','Pc-Freak.Net Blog'),
                              _('A pC Freak Blog'),
                              false,
                              'nav_pcfreak');

Once you're done with installation, make sure you change permissions or move install.php from /var/www/status, otherwise someone might overwrite your config.php and mess your installation …

chmod 000 /var/www/status/install.php There is plenty of other things to do with StatusNet (Support for communication with Jabber XMPP protocol, notification via SMS etc. There are also some plugins to add more statusnet functionality.


Enjoy micro blogging ! 🙂

Use apt-get with Proxy howto – Set Proxy system-wide in Linux shell and Gnome

Friday, May 16th, 2014

linux-apt-get-configure-proxy-howto-set-proxy-systemwide-in-linux

I juset setup a VMWare Virtual Machine on my HP notebook and installed Debian 7.0 stable Wheezy. Though VMWare identified my Office Internet and configured automatically NAT, I couldn't access the internet from a browser until I remembered all HP traffic is going through a default set browser proxy.
After setting a proxy to Iceweasel, Internet pages started opening normally, however as every kind of traffic was also accessible via HP's proxy, package management with apt-get (apt-get update, apt-get install etc. were failing with errors):


# apt-get update

Ign cdrom://[Debian GNU/Linux 7.2.0 _Wheezy_ – Official i386 CD Binary-1 20131012-12:56] wheezy Release.gpg
Ign cdrom://[Debian GNU/Linux 7.2.0 _Wheezy_ – Official i386 CD Binary-1 20131012-12:56] wheezy Release
Ign cdrom://[Debian GNU/Linux 7.2.0 _Wheezy_ – Official i386 CD Binary-1 20131012-12:56] wheezy/main i386 Packages/DiffIndex
Ign cdrom://[Debian GNU/Linux 7.2.0 _Wheezy_ – Official i386 CD Binary-1 20131012-12:56] wheezy/main Translation-en_US
Err http://ftp.by.debian.org wheezy Release.gpg
  Could not connect to ftp.by.debian.org:80 (86.57.151.3). – connect (111: Connection refused)
Err http://ftp.by.debian.org wheezy-updates Release.gpg
  Unable to connect to ftp.by.debian.org:http:
Err http://security.debian.org wheezy/updates Release.gpg
  Cannot initiate the connection to security.debian.org:80 (2607:ea00:101:3c0b:207:e9ff:fe00:e595). – connect (101: Network is unreachable) [IP: 2607:ea00:101:3c0b:207:e9ff:fe00:e595 80]
Reading package lists…

 

This error is caused because apt-get is trying to directly access above http URLs and because port 80 is filtered out from HP Office, it fails in order to make it working I had to configure apt-get to use Proxy host – here is how:

a) Create /etc/apt/apt.conf.d/02proxy file (if not already existing)
and place inside:
 

Acquire::http::proxy::Proxy "https://web-proxy.cce.hp.com";
Acquire::ftp::proxy::Proxy "ftp://web-proxy.cce.hp.com";


To do it from console / gnome-terminal issue:
echo ''Acquire::http::Proxy "https://web-proxy.cce.hp.com:8088";' >> /etc/apt/apt.conf.d/02proxy
echo ''Acquire::ftp::Proxy "https://web-proxy.cce.hp.com:8088";' >> /etc/apt/apt.conf.d/02proxy

That's all now apt-get will tunnel all traffic via HTTP and FTP proxy host web-proxy.cce.hp.com and apt-get works again.

Talking about Proxyfing Linux's apt-get, its possible to also set proxy shell variables, which are red and understood by many console programs like Console browsers lynx, links, elinks  as well as wget and curl commands, e.g.:

 

export http_proxy=http://192.168.1.5:5187/
export https_proxy=$http_proxy
export ftp_proxy=$http_proxy
export rsync_proxy=$http_proxy
export no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com"

For proxies protected with username and password export variables should look like so: echo -n "username:"
read -e username
echo -n "password:"
read -es password
export http_proxy="http://$username:$password@proxyserver:8080/"
export https_proxy=$http_proxy
export ftp_proxy=$http_proxy
export rsync_proxy=$http_proxy
export no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com"

To make this Linux proxy settings system wide on Debian / Ubuntu there is the /etc/environment file add to it:
 

http_proxy=http://proxy.server.com:8080/
https_proxy=http://proxy.server.com:8080/
ftp_proxy=http://proxy.server.com:8080/
no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com"
HTTP_PROXY=http://proxy.server.com:8080/
HTTPS_PROXY=http://proxy.server.com:8080/
FTP_PROXY=http://proxy.server.com:8080/
NO_PROXY="localhost,127.0.0.1,localaddress,.localdomain.com"


To make proxy global (systemwide) for most (non-Debian specific) Linux distributions shell environments create new file /etc/profile.d/proxy.sh and place something like:

function proxy(){
echo -n "username:"
read -e username
echo -n "password:"
read -es password
export http_proxy="http://$username:$password@proxyserver:8080/"
export https_proxy=$http_proxy
export ftp_proxy=$http_proxy
export rsync_proxy=$http_proxy
export no_proxy="localhost,127.0.0.1,localaddress,.localdomain.com"
echo -e "nProxy environment variable set."
}
function proxyoff(){
unset HTTP_PROXY
unset http_proxy
unset HTTPS_PROXY
unset https_proxy
unset FTP_PROXY
unset ftp_proxy
unset RSYNC_PROXY
unset rsync_proxy
echo -e "nProxy environment variable removed."
}

To set Global Proxy (make Proxy Systemwide) for a user in GNOME Desktop environment launch gnome-control-center

And go to Network -> Network Proxy

/images/gnome-configure-systemwide-proxy-howto-picture1

/images/gnome-configure-systemwide-proxy-howto-picture2

To make proxy settings also system wide for some GUI Gnome GTK3 applications

gsettings set org.gnome.system.proxy mode 'manual'
gsettings set org.gnome.system.proxy.http host 'your-proxy.server.com'
gsettings set org.gnome.system.proxy.http port 8080

find text strings recursively in Linux and UNIX – find grep in sub-directories command examples

Tuesday, May 13th, 2014

unix_Linux_recursive_file_search_string_grep
GNU Grep
is equipped with a special option "-r" to grep recursively. Looking for string in a file in a sub-directories tree with the -r option is a piece of cake. You just do:

grep -r 'string' /directory/

or if you want to search recursively non-case sensitive for text

grep -ri 'string' .
 

Another classic GNU grep use (I use almost daily) is whether you want to match all files containing (case insensitive) string  among all files:

grep -rli 'string' directory-name
 

Now if you want to grep whether a string is contained in a file or group of files in directory recursively on some other UNIX like HP-UX or Sun OS / Solaris where there is no GNU grep installed by default here is how to it:

find /directory -exec grep 'searched string' {} dev/null ;

Note that this approach to look for files containing string on UNIX is very slowThus on not too archaic UNIX systems for some better search performance it is better to use xargs;

find . | xargs grep searched-string


A small note to open here is by using xargs there might be weird results when run on filesystems with filenames starting with "-".

Thus comes the classical (ultimate) way to grep for files containing string with find + grep, e.g.

find / -exec grep grepped-string {} dev/null ;

Another way to search a string recursively in files is by using UNIX OS '*' (star) expression:

grep pattern * */* */*/* 2>/dev/null

Talking about recursive directory text search in UNIX, should mention  another good GNU GREP alternative ACK – check it on betterthangrep.com 🙂 . Ack is perfect for programmers who have to dig through large directory trees of code for certain variables, functions, objects etc.

 

MySQL SSL Configure Howto – How to Make MySQL communication secured

Wednesday, January 15th, 2014

mysql-over-ssl-how-to-configure-logo how to configure ssl on mysql server

Recently I've been asked How to make communication to MySQL database encrypted. The question was raised by a fellow developer who works on developing a Desktop standalone application in Delphi Programming Language with DevArt an (SQL Connection Component capable to connect Delphi applications to multiple databases like MySQL, Oracle, PostgreSQL, Interbase, Firebird etc.

Communicating in Secured form to MySQL database is not common task to do, as MySQL usually communicates to applications hosted on same server or applications to communicate to MySQL are in secured DMZ or administrated via phpMyAdmin web interface.

MySQL supports encrypted connections to itself using Secure Socket Layer (SSL) encryption. Setting up MySQL db to be communicated encrypted is a must for standalone Desktop applications which has to extract / insert data via remote SQL.
Configuring SQL to support communicated queries encrpytion is supported by default and easily configured on most standard Linux version distributions (Debian, RHEL, Fedora) with no need to recompile it.
1. Generate SSL Certificates

$ mkdir /etc/mysql-ssl && cd mysql-ssl

# Create CA certificate
$ openssl genrsa 2048 > ca-key.pem
$ openssl req -new -x509 -nodes -days 3600 \
         -key ca-key.pem -out ca-cert.pem

Create server certificate, remove passphrase, and sign it
server-cert.pem is public key, server-key.pem is private key
$ openssl req -newkey rsa:2048 -days 3600 \
         -nodes -keyout server-key.pem -out server-req.pem

$ openssl rsa -in server-key.pem -out server-key.pem
$ openssl x509 -req -in server-req.pem -days 3600 \
         -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 -out server-cert.pem

Create client certificate, remove passphrase, and sign it
client-cert.pem is public key and client-key.pem is private key
$ openssl req -newkey rsa:2048 -days 3600 \
         -nodes -keyout client-key.pem -out client-req.pem

$ openssl rsa -in client-key.pem -out client-key.pem
$ openssl x509 -req -in client-req.pem -days 3600 \
         -CA ca-cert.pem -CAkey ca-key.pem -set_serial 01 -out client-cert.pem

After generating the certificates, verify them:

$ openssl verify -CAfile ca-cert.pem server-cert.pem client-cert.pem
 

2. Add SSL support variables to my.cnf

Once SSL key pair files are generated in order to active SSL encryption support in MySQL server, add to (/etc/my.cnf,  /etc/mysql/my.cnf, /usr/local/etc/my.cnf … ) or wherever config is depending on distro

# SSL
ssl-ca=/etc/mysql-ssl/ca-cert.pem
ssl-cert=/etc/mysql-ssl/server-cert.pem
ssl-key=/etc/mysql-ssl/server-key.pem

3. Restart MySQL server

/etc/init.d/mysqld restart
...

4. Create SQL user to require SSL login

Create new user with access to database;

GRANT ALL ON Sql_User_DB.* TO Sql_User@localhost;
FLUSH PRIVILEGES;

To create administrator privileges user:

GRANT ALL PRIVILEGES ON *.* TO ‘ssluser’@'%’ IDENTIFIED BY ‘pass’ REQUIRE SSL;
FLUSH PRIVILEGES;

5. Test SSL Connection with MySQL CLI client or with few lines of PHP

To use mysql cli for testing whether SSL connection works:

$ mysql -u ssluser -p'pass' –ssl-ca /etc/mysql-ssl/client-cert.pem –ssl-cert /etc/mysql-ssl/client-key.pem

Once connected to MySQL to verify SSL connection works fine:

mysql> SHOW STATUS LIKE 'Ssl_Cipher';
 +---------------+--------------------+
| Variable_name | Value              |
 +---------------+--------------------+
| Ssl_cipher    | DHE-RSA-AES256-SHA |
+---------------+--------------------+

If you get this output this means MySQL SSL Connection is working as should.

Alternative way is to use test-mysqli-ssl.php script to test availability to mysql over SSL.

$conn=mysqli_init();
mysqli_ssl_set($conn, '/etc/mysql-ssl/client-key.pem', '/etc/mysql-ssl/client-cert.pem', NULL, NULL, NULL);
if (!mysqli_real_connect($conn, '127.0.0.1', 'ssluser', 'pass')) { die(); }
$res = mysqli_query($conn, 'SHOW STATUS like "Ssl_cipher"');
print_r(mysqli_fetch_row($res));
mysqli_close($conn);

Note: Change username password according to your user / pass before using the script

That's all now you have mysql communicating queries data over SSL

 

How to configure NTP server (ntpd) to synchronize server clock over the Internet on FreeBSD

Friday, February 10th, 2012

 

FreeBSD ntpd logo / How to configure ntpd to synchronize with internet time servers on FreeBSD

On FreeBSD ntpd , ntpdc , ntpdate , ntpq doesn't need to be installed via a specific package like on GNU/Linux as they're part of the FreeBSD world (binary standardly shipped with FreeBSD basis system).

The FreeBSD handbook has a chapter explaining thoroughfully on ntp on FreeBSD ,however for the lazy ones here is a short few steps tutorial on how to install and configure ntpd on bsd :

1. Copy sample ntp.conf file to /etc/

freebsd# cp -rpf /usr/src/etc/ntp.conf /etc/ntp/

No need for any modifications if you don't want to apply some specific restrictions on whom can access the ntpd server. If you update regularly the FreeBSD system with freebsd-update or directly by rebuilding the FreeBSD kernel / world adding restrictions might be not necessery..

If you check /usr/src/etc/ntp.conf you will notice freebsd project people are running their own ntp servers , by default ntpd will use this servers to fetch timing information. The exact server hosts which as of time of writting are used can be seen in ntp.conf and are:

server 0.freebsd.pool.ntp.org iburst maxpoll 9
server 1.freebsd.pool.ntp.org iburst maxpoll 9
server 2.freebsd.pool.ntp.org iburst maxpoll 9

2. Add ntpd daemon to load on system boot via /etc/rc.conf

By default ntpd is disabled on FreeBSD, you can see if it is disabled or enabled by invoking:

freebsd# /etc/rc.d/ntpd rcvar
# ntpd
ntpd_enable=NO

To Enable ntpd to get loaded each time it boots , following 3 lines has to be added in /etc/rc.conf .

ntpdate_enable="YES"
ntpdate_flags="europe.pool.ntp.org"
ntpd_enable="YES"

Quick way to add them is to use echo :

echo 'ntpdate_enable="YES" >> /etc/rc.conf
echo 'ntpdate_flags="europe.pool.ntp.org" >> /etc/rc.conf
echo 'ntpd_enable="YES" >> /etc/rc.conf

Now as the 3 rc.conf vars are set to "YES", the ntpd can be started. Without having this variables in /etc/rc.conf , "/etc/rc.d/ntpd start" will refuse to start ntpd.

3. Start the ntpd service

freebsd# /etc/rc.d/ntpd start
...

One interesting note to make is ntpd can also operate without specifying any config file (/etc/ntp.conf), the only requirement for the server to start is to have a properly set ntpdate server, like lets say (ntpdate_flags="europe.pool.ntp.org")

4. Permit only certain host or localhost to "talk" to the ntpd server

If you want to imply some ntp server restrictions, the configuration directives are same like on Linux:

To allow only a a host inside a local network with IP 192.168.0.2 as well as localhost, to be able to fetch time information via ntpd server put inside /etc/ntp.conf:

restrict 127.0.0.1
restrict 192.168.0.1 mask 255.255.255.0 nomodify notrap

If you want to prohibit ntpd to serve as a Network Time Server, to any other host except localhost, add in /etc/ntp.conf :

restrict default ignore

Allowing and denying certain hosts can be also done on pf (packet filter) or ipfw firewall level, and in my view is easier (and less confusing), than adding restrictions through ntp.conf. Besides that using directly the server firewall to apply restrictions is more secure. If for instance a remote exploit vulnerability is discovered affecting your ntpd server. this will not affect you externally as access to the UDP port 123 will be disabled on a firewall level.
Something good to mention is NTP servers communicate between each other using the UDP source/destination (port 123). Hence if the NTPD server has to be publicly accessible and there is a firewall already implemented, access to source/dest port 123 should be included in the configured firewall …

5. Check if the ntp server is running properly / ntp server query operations

[root@pcfreak /home/hipo]# ps axuww|grep -i ntp
root 15647 0.0 0.2 4672 1848 ?? Ss 2:49PM 0:00.04 /usr/sbin/ntpd -c /etc/ntp.conf -p /var/run/ntpd.pid -f /var/db/ntpd.drift

To query the now running ntpd server as well as set various configuration options "on the fly" (e.g. without need for ntp.conf edits and init script restart), a tool called ntpdc exists. ntpdc tool could be used to connect to localhost running ntpd as well as to connect and manage remotely a ntpd server.
The most basic use of ntpdc is to check (server peers).:
freebsd# ntpdc localhost
ntpdc> peers
remote local st poll reach delay offset disp
===================================================

kgb.comnet.bg 83.228.93.76 2 64 377 0.00282 -0.050575 0.06059
*billing.easy-la 83.228.93.76 2 64 377 0.01068 -0.057400 0.06770
=ns2.novatelbg.n 83.228.93.76 2 64 377 0.01001 -0.055290 0.06058

ntpdc has also a non-interactive interface, handy if there is a need for requests to a ntpd to be scripted. To check ntpd server peers non-interactively:

freebsd# ntpdc -p localhost
===================================================
kgb.comnet.bg 83.228.93.76 2 64 377 0.00284 -0.043157 0.06184
=billing.easy-la 83.228.93.76 2 64 377 0.01059 -0.042648 0.05811
*ns2.novatelbg.n 83.228.93.76 2 64 377 0.00996 -0.041097 0.06094

ntpdc has plenty of other ntpd query options, e.g. :

ntpdc> help
ntpdc commands:
addpeer controlkey fudge keytype quit timeout
addrefclock ctlstats help listpeers readkeys timerstats
addserver debug host loopinfo requestkey traps
addtrap delay hostnames memstats reset trustedkey
authinfo delrestrict ifreload monlist reslist unconfig
broadcast disable ifstats passwd restrict unrestrict
clkbug dmpeers iostats peers showpeer untrustedkey
clockstat enable kerninfo preset sysinfo version
clrtrap exit keyid pstats sysstats

ntpdc is an advanced query tool for ntpd , servers. Another tool exists called ntpq which syntax is almost identical to ntpdc . The main difference between the two is ntpq is a monitoring tool mostly used just for monitoring purposes, where ntpdc can also change plenty of things in the server configuration.

For people who want to learn more on ntpd the man page is a great reading , containing chapters describing thoroughfully exactly how NTPD time servers operate, etc.

How to debug mod_rewrite .htaccess problems with RewriteLog / Solve mod_rewrite broken redirects

Friday, September 30th, 2011

Its common thing that CMS systems and many developers custom .htaccess cause issues where websites depending on mod_rewrite fails to work properly. Most common issues are broken redirects or mod_rewrite rules, which behave differently among the different mod_rewrite versions which comes with different versions of Apache.

Everytime there are such problems its necessery that mod_rewrite’s RewriteLog functionality is used.
Even though the RewriteLog mod_rewrite config variable is well described on httpd.apache.org , I decided to drop a little post here as I’m pretty sure many novice admins might not know about RewriteLog config var and might benefit of this small article.
Enabling mod_rewrite requests logging of requests to the webserver and process via mod_rewrite rules is being done either via the specific website .htaccess (located in the site’s root directory) or via httpd.conf, apache2.conf etc. depending on the Linux / BSD linux distribution Apache config file naming is used.

To enable RewriteLog near the end of the Apache configuration file its necessery to place the variables in apache conf:

1. Edit RewriteLog and place following variables:

RewriteLogLevel 9
RewriteLog /var/log/rewrite.log

RewriteLogLevel does define the level of logging that should get logged in /var/log/rewrite.log
The higher the RewriteLogLevel number defined the more debugging related to mod_rewrite requests processing gets logged.
RewriteLogLevel 9 is actually the highest loglevel that can be. Setting the RewriteLogLevel to 0 will instruct mod_rewrite to stop logging. In many cases a RewriteLogLevel of 3 is also enough to debug most of the redirect issues, however I prefer to see more, so almost always I use RewriteLogLevel of 9.

2. Create /var/log/rewrite.log and set writtable permissions

a. Create /var/log/rewrite.log

freebsd# touch /var/log/rewrite.log

b. Set writtable permissons

Either chown the file to the user with which the Apache server is running, or chmod it to permissions of 777.

On FreeBSD, chown permissions to allow webserver to write in file, should be:

freebsd# chown www:www /var/log/rewrite.log

On Debian and alike distros:

debian:~# chown www-data:www-data /var/log/rewrite.log

On CentOS, Fedora etc.:

[root@centos ~]# chown httpd:httpd /var/log/rewrite.log

On any other distribution, you don’t want to bother to check the uid:gid, the permissions can be set with chmod 777, e.g.:

linux# chmod 777 /var/log/rewrite.log

Next after RewriteLog is in conf to make configs active the usual webserver restart is required.

To restart Apache On FreeBSD:

freebsd# /usr/local/etc/rc.d/apache2 restart
...

To restart Apache on Debian and derivatives:

debian:~# /etc/init.d/apache2 restart
...

On Fedora and derivive distros:

[root@fedora ~]# /etc/init.d/httpd restart
...

Its common error to forget to set proper permissions to /var/log/rewrite.log this has puzzled me many times, when enabling RewriteLog’s logging.

Another important note is when debugging for mod_rewrite is enabled, one forgets to disable logging and after a while if the /var/log partition is placed on a small partition or is on an old server with less space often the RewriteLog fills in the disk quickly and might create website downtimes. Hence always make sure RewriteLog is disabled after work rewrite debugging is no longer needed.

The way I use to disable it is by commenting it in conf like so:

#RewriteLogLevel 9
#RewriteLog /var/log/rewrite.log

Finally to check, what the mod_rewrite processor is doing on the fly its handy to use the well known tail -f

linux# tail -f /var/log/rewrite.log

A bunch of time in watching the requests, should be enough to point to the exact problem causing broken redirects or general website malfunction.
Cheers 😉

Possible way to increase Linux TCP/IP port thoroughput via sysctl kernel variable

Thursday, August 25th, 2011

Sysctl is a great way to optimize Linux. sysctl has a dozens of values which could drastically improve server networking and overall performance.

One of the many heplful variables to optimize the way the Linuz kernel works on busy servers is net.ipv4.ip_local_port_range .

The default sysctl setting for net.ipv4.ip_local_port_range on Debian, Ubuntu Fedora, RHEL, CentOS is:

net.ipv4.ip_local_port_range = 32768 65536

This means that the kernel and the corresponding server running services instructing the Linuz kernel open new port sockets can only open local ports in the range of 32768 – 65536 .
On a regular Desktop GNU/Linux machine or a not high iron server this settins is perfectly fine, however on a high scale servers the local port range in the interval of 32768-65536 might be insufficient at times, especially if there are programs which require binding of many local ports.

Therefore on a high load servers, generally it’s a good to raise the port range to be assigned by kernel to 8912 – 65536 , to do so the setting has to be changed like shown below:

linux:~# sysctl -w net.ipv4.ip_local_port_range = 8192 65536
...

If changing this setting on the server doesn’t show any negative impact on performance in few hours time or a day or even better decreases the server average load, it’s a good idea that it be added to sysctl.conf to load up the setting on next kernel boot.

linux:~# echo 'net.ipv4.ip_local_port_range' >> /etc/sysctl.conf

Enjoy 😉

How to harden Linux Security and imprpove network efficiency on Kernel sysctl Level to Stop SYN flood

Friday, July 8th, 2011

Power up Linux and protect against DDoS with sysctl var optimization

Some long time ago I’ve written an article Optimizing Linux tcp/ip networking

In the article I’ve examined a number of Linux kernel sysctl variables, which significantly improve the way TCP/IP networking is handled by a non router Linux based servers.

As the time progresses I’ve been continuing to read materials on blogs and internet sites on various tips and anti Denial of Service rules which one could apply on newly installed hosting (Apache/MySql/Qmail/Proxy) server to improve webserver responce times and tighten the overall security level.

In my quest for sysctl 😉 I found a few more handy sysctl variables apart from the old ones I incorporate on every Linux server I adminstrate.
The sysctl variables improves the overall network handling efficiency and protects about common SYN/ACK Denial of service attacks.

Here are the extra sysctl variables I started incorporating just recently:

############ IPv4 Sysctl Settings ################
#Enable ExecShield protection (randomize virtual assigned space to protect against many exploits)
kernel.randomize_va_space = 1
#Increase the number of PIDs processes could assign this is very needed especially on more powerful servers
kernel.pid_max = 65536
# Prevent against the common 'syn flood attack'
net.ipv4.tcp_syncookies = 1
# Controls the use of TCP syncookies two is generally a better idea, though you might experiment
#net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_synack_retries = 2
##################################################
#
############## IPv6 Sysctl Settings ################
# Number of Router Solicitations to send until assuming no routers are present.
net.ipv6.conf.default.router_solicitations = 0
# Accept Router Preference in RA? Again not necessery if the server is not a router
net.ipv6.conf.default.accept_ra_rtr_pref = 0
# Learn Prefix Information in Router Advertisement (Unnecessery) for non-routers
net.ipv6.conf.default.accept_ra_pinfo = 0
# disable accept of hop limit settings from other routers (could be used for DoS)
net.ipv6.conf.default.accept_ra_defrtr = 0
# disable ipv6 global unicasts server assignments
net.ipv6.conf.default.autoconf = 0
# neighbor solicitations to send out per address (better if disabled)
net.ipv6.conf.default.dad_transmits = 0
# disable assigning more than 1 address per network interface
net.ipv6.conf.default.max_addresses = 1
#####################################################

 

To use this settings paste the above sysctl variables in /etc/sysctl.conf and ask sysctl command to read and apply the newly added conf settings:

server:~# sysctl -p
...

Hopefully you should not get errors while applying the sysctl settings, if you get some errors, it’s possible some of the variable is differently named (depending on the Linux kernel version) or the Linux distribution on which sysctl’s are implemented.

For some convenience I’ve created unified sysctl variables /etc/sysct.conf containing the newly variables I started implementing to servers with the ones I already exlpained in my previous post Optimizing Linux TCP/IP Networking

Here is the optimized / hardened sysctl.conf file for download

I use this exact sysctl.conf these days on both Linux hosting / VPS / Mail servers etc. as well as on my personal notebook 😉

Here is also the the complete content of above’s sysctl.conf file, just in case if somebody wants to directly copy/paste it in his /etc/sysctl.conf

# Sysctl kernel variables to improve network performance and protect against common Denial of Service attacks
# It's possible that not all of the variables are working on all Linux distributions, test to make sure
# Some of the variables might need a slight modification to match server hardware, however in most cases it should be fine
# variables list compiled by hip0
### http://www.pc-freak.net
#### date 08.07.2011
############ IPv4 Sysctl Kernel Settings ################
net.ipv4.ip_forward = 0
# ( Turn off IP Forwarding )
net.ipv4.conf.default.rp_filter = 1
# ( Control Source route verification )
net.ipv4.conf.default.accept_redirects = 0
# ( Disable ICMP redirects )
net.ipv4.conf.all.accept_redirects = 0
# ( same as above )
net.ipv4.conf.default.accept_source_route = 0
# ( Disable IP source routing )
net.ipv4.conf.all.accept_source_route = 0
# ( - || - )net.ipv4.tcp_fin_timeout = 40
# ( Decrease FIN timeout ) - Useful on busy/high load server
net.ipv4.tcp_keepalive_time = 4000
# ( keepalive tcp timeout )
net.core.rmem_default = 786426
# Receive memory stack size ( a good idea to increase it if your server receives big files )
##net.ipv4.tcp_rmem = "4096 87380 4194304"
net.core.wmem_default = 8388608
#( Reserved Memory per connection )
net.core.wmem_max = 8388608
net.core.optmem_max = 40960
# ( maximum amount of option memory buffers )
# tcp reordering, increase max buckets, increase the amount of backlost
net.ipv4.tcp_max_tw_buckets = 360000
net.ipv4.tcp_reordering = 5
##net.core.hot_list_length = 256
net.core.netdev_max_backlog = 1024
#Enable ExecShield protection (randomize virtual assigned space to protect against many exploits)
kernel.randomize_va_space = 1
#Increase the number of PIDs processes could assign this is very needed especially on more powerful servers
kernel.pid_max = 65536
# Prevent against the common 'syn flood attack'net.ipv4.tcp_syncookies = 1
# Controls the use of TCP syncookies two is generally a better idea, though you might experiment
#net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_synack_retries = 2
###################################################
############## IPv6 Sysctl Settings ################
# Number of Router Solicitations to send until assuming no routers are present.
net.ipv6.conf.default.router_solicitations = 0
# Accept Router Preference in RA? Again not necessery if the server is not a router
net.ipv6.conf.default.accept_ra_rtr_pref = 0
# Learn Prefix Information in Router Advertisement (Unnecessery) for non-routersnet.
ipv6.conf.default.accept_ra_pinfo = 0
# disable accept of hop limit settings from other routers (could be used for DoS)
net.ipv6.conf.default.accept_ra_defrtr = 0
# disable ipv6 global unicasts server assignmentsnet.
ipv6.conf.default.autoconf = 0
# neighbor solicitations to send out per address (better if disabled)
net.ipv6.conf.default.dad_transmits = 0
# disable assigning more than 1 address per network interfacenet.
ipv6.conf.default.max_addresses = 1
#####################################################
# Reboot if kernel panic
kernel.panic = 20

These sysctl settings will tweaken the Linux kernel default network settings performance and you will notice the improvements in website responsiveness immediately in some cases implementing this kernel level goodies will make the server perform better and the system load might decrease even 😉

This optimizations on a kernel level are not only handy for servers, their implementation on Linux Desktop should also have a positive influence on the way the network behaves and could improve significantly the responce times of opening pages in Firefox/Opera/Epiphany Torrent downloads etc.

Hope this kernel tweakenings are helpful to someone.
Cheers 😉

How to configure pine (alpine) console client to work with vpopmail pop3 and imap protocol

Monday, June 13th, 2011

I needed to check my mail via ssh connection, as my installed squirrelmail is curently broken and I’m away from my own personal computer.

I did some online research on how this can be achieved and thanksfully I finallyfound a way to check my pop3 and imap mailbox with a console client called alpine , better known in unix community under the name pine .

I installed pine on my Debian with apt:


debian:~# apt-get install alpine

Here is my pine configuration file .pinerc used to fetch my mail with pine:

a .pinerc conf file to check my pop3 mail

To use that file I placed it in my home directory ~/ , e.g.:


debian:~# wget http://www.pc-freak.net/files/.pinerc
...

To attune the pop3 server configuration in the sample .pinerc above one needs to change the value of:


inbox-path=

For example to configure pine to fetch mail from the pop3 server mail.pc-freak.net and store it locally in my home directory within a file called INBOX
I have configured the inbox-path .pinerc variable to look like so:


inbox-path={mail.pc-freak.net/pop3/user=hipo@pc-freak.net}INBOX

In above configuration’s inbox-path variable configuration the /pop3/ specifies I want to fetch my mail via the pop3 protocol , if one wants to use imap this has to be substituted with /imap/

The value user=hipo@pc-freak.net specifies my vpopmail created user which in my case is obviously hipo@pc-freak.net

The other variables which are good to be changed in .pinerc config are:


personal-name=

This variable has to be set to the name of the Email Sender which will be set, if pine is used to send email.

I also changed the user-domain variable as it’s used to set the domain name from which the pine client will send the emails from:

As my domain is pc-freak.net I’ve set the domain name variable to be:


user-domain=pc-freak.net

Now after launching pine it prompted me for my email password, putting in the pass did fetch all my new unread mails via pop3 protocol.

The only annoying thing was that each time I quit pine and start it up again, I’m now asked to enter the email password.

This behaviour is really shitty, but thanksfully one can easily workaround that by letting pine be constantly running detached in gni screen session.