Posts Tagged ‘tomcat’

Fix “FAIL – Application at context path /application-name could not be started” deployment error in Tomcat manager

Thursday, October 1st, 2015


While deploying an environment called "Interim" which is pretty much like a testing Java application deployed from a Java EAR (Enterprise Archive) file from within a Tomcat Manager GUI web interface after stopping the application and trying to start it, the developers come across the error:


FAIL – Application at context path /application-name could not be started

The error puzzled me for a while until I checked the catalina.out I've seen a number of thrown Java Eceptions errors like:

Okt 01, 2015 10:48:46 AM org.springframework.web.context.ContextLoader initWebApplicationContext

Schwerwiegend: Context initialization failed

org.springframework.beans.factory.BeanCreationException: Error creating bean with name '' defined in ServletContex

t resource [/WEB-INF/pp-server-beans.xml]: Cannot create inner bean ‘(inner bean)’ of type [org.jvnet.jax_ws_commons.spring.SpringService] while setting bean property

'service'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name '(inner bean)#33': FactoryBean threw exception on

 object creation; nested exception is java.lang.OutOfMemoryError: PermGen space

I've googled a bit about the error:

"FAIL – Application at context path /application-name could not be started"

and come across this Stackoverflow thread and followed suggested solution to fix web.xml tag closing error but it seems there was no such error in my case, I then also tried solution suggested by this thread (e.g. adding in file:

org.apache.catalina.core.ContainerBase.[Catalina].level = INFO
org.apache.catalina.core.ContainerBase.[Catalina].handlers = java.util.logging.ConsoleHandler

unfortunately this helped neither to solve the error when it is tried to be started from tomcat manager.

After asking for help a colleague Kostadin, he pointed me to take a closer look in the error (which is a clear indication) that the reserved space is not enough (see below err):

java.lang.OutOfMemoryError: PermGen space

And he pointed me then to Solution (which is to modify the present tomcat settings which looked like this:

# Heap size settings

export JAVA_OPTS="-Xms2048M -Xmx2048M"


# SSCO test page parameter


# Default garbage collector settings

export JAVA_OPTS="$JAVA_OPTS -XX:MaxPermSize=128M"


# Aggressive garbage collector settings.

# Do not use! For testing purposes only!

#export JAVA_OPTS="$JAVA_OPTS -Xss128k -XX:ParallelGCThreads=20 -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:SurvivorRatio=8 -XX:TargetSurvivorRatio=90 -XX:MaxTenuringThreshold=31 -XX:+AggressiveOpts -XX:MaxPermSize=128M"


####### DO NOT CHANGE BELOW HERE #######

# Disable X11 usage


export JAVA_OPTS="$JAVA_OPTS -Djava.awt.headless=true"


# Garbage collection log settings

export JAVA_OPTS="$JAVA_OPTS -verbose:gc -XX:+PrintGCTimeStamps -XX:+PrintGCDetails -Xloggc:/web/tomcat/current/logs/gc.log -XX:-TraceClassUnloading"


# Enable JMX console








The solution is to to add following two options to export JAVA_OPTS (above options):

-XX:PermSize=512m -XX:MaxPermSize=512m

After modifications, my new line within looked like so:


JAVA_OPTS="-Xms2048M -Xmx2048M -XX:PermSize=512m -XX:MaxPermSize=512m"

Finally to make new JAVA_OPTS settings, I've restarted Tomcat with:


cd /web/tomcat/bin/
sleep 5; ./

And hooray it works fine thanks God! 🙂

Share this on

Adding Listing and Deleting SSL Certificates in keystore Tomcat Application server / What is keystore

Thursday, December 5th, 2013

Apache Tomcat keystore delete import list logo

 I work on ongoing project where Tomtat Application servers configured to run Clustered located behind Apache with mod_proxy configured to use ReverseProxy are used. One of customers which required a java application deployment experienced issues with application's capability to connect to SAP database.

After some investigation I figured out, the application is unable to connect to the SAP db server becuse remote host webserver running some SAP related stuff was not connecting due to expired certificate in Tomcat Keystore known also as JKS / Java Keystore– (.keystore) – which is a file containing multiple remote hosts imported certificates.

The best and shortest definition of keystore is:

Keystore entry = private + public key pair = identified by an alias

The keystore protects each private key with its individual password, and also protects the integrity of the entire keystore with a (possibly different) password.

Managing Java imported certificates later used by Tomcat is done with a command line tool part of JDK (Java Development Kit) called keystore. Keystore is usually located under /opt/java/jdk/bin/keytool. My Java VM is installed in /opt/ anyways usual location of keytool is $JAVA_HOME/bin/

Keytool has capabilities to create / modify / delete or import new SSL certificates and then Java applications can access remote applications which requires Secure Socket Layer handshake . Each certificate kept in .keystore file (usually located somewhere under Tomcat web app server directory tree), lets say – /opt/tomcat/current/conf/.keystore

1. List current existing imported SSL certificates into Java's Virtual Machine

tomcat-server:~# /opt/java/jdk/bin/keytool -list -keystore /opt/tomcat/current/conf/.keystore
Command returns output similar to;

Entry type: trustedCertEntry

Issuer: CN=E.ON Internal Devices Sub CA V2, OU=CA, O=EGP, C=DE
Serial number: 67460001001c6aa51fd25c0e8320
Valid from: Mon Dec 27 07:05:33 GMT 2010 until: Fri Dec 27 07:05:22 GMT 2013
Certificate fingerprints:
         MD5:  D1:AA:D5:A9:A3:D2:95:28:F1:79:57:25:D3:6A:16:5E
         SHA1: 73:CE:ED:EC:CA:18:E4:E4:2E:AA:25:58:E0:2B:E4:D4:E7:6E:AD:BF
         Signature algorithm name: SHA1withRSA
         Version: 3


#1: ObjectId: Criticality=true
KeyUsage [

#2: ObjectId: Criticality=true
  PathLen: undefined

#3: ObjectId: Criticality=false
AuthorityInfoAccess [
   accessLocation: URIName:,
   accessLocation: URIName:]

#4: ObjectId: Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: D3 52 C7 63 0F 98 BF 6E   FE 00 56 5C DF 35 62 22  .R.c…n..V\.5b"
0010: F2 B9 5B 8F                                        ..[.

Note that password that will be promtped has is by default changeit (in case if you don't have explicitly changed it from Tomcat's default config server.xml).

2. Delete Old expired SSL host Certificate from Java Keystore
It is good practice to always make backup of old .keystore before modifying, so I ran:

tomcat-server:~# cp -rpf /opt/tomcat/current/conf/.keystore /opt/tomcat/current/conf/.keystore-05-12-2013

In my case first I had to delete old expired SSL certificate with:

tomcat-server:~# /opt/java/jdk/bin/keytool -delete -alias "your-hostname" -v -keystore /opt/tomcat/current/conf/.keystore

Then to check certificate is no longer existent in keystore chain;
tomcat-server:~# /opt/java/jdk/bin/keytool -list -keystore /opt/tomcat/current/conf/.keystore

-keystore – option is obligitory it does specify where keystore file is located
-list – does list the certificate
-v – stands for verbose


3. Finally to import new SSL from already expored via a browser url in keystore

tomcat-server:~# /opt/java/jdk/bin/keytool -importcert -file /tmp/your-hostname.cer -alias -keystore /opt/tomcat/current/conf/.keystore

More complete information on how to deal with keystore is available from Apache Tomcat's SSL Howto – a must read documentation for anyone managing Tomcat.

Share this on

Maximal protection against SSH attacks. If your server has to stay with open SSH (Secure Shell) port open to the world

Thursday, April 7th, 2011

Brute Force Attack SSH screen, Script kiddie attacking
If you’re a a remote Linux many other Unix based OSes, you have defitenily faced the security threat of many failed ssh logins or as it’s better known a brute force attack

During such attacks your /var/log/messages or /var/log/auth gets filled in with various failed password logs like for example:

Feb 3 20:25:50 linux sshd[32098]: Failed password for invalid user oracle from port 51490 ssh2
Feb 3 20:28:30 linux sshd[32135]: Failed password for invalid user oracle1 from port 42778 ssh2
Feb 3 20:28:55 linux sshd[32141]: Failed password for invalid user test1 from port 51072 ssh2
Feb 3 20:30:15 linux sshd[32163]: Failed password for invalid user test from port 47481 ssh2
Feb 3 20:33:20 linux sshd[32211]: Failed password for invalid user testuser from port 51731 ssh2
Feb 3 20:35:32 linux sshd[32249]: Failed password for invalid user user from port 38966 ssh2
Feb 3 20:35:59 linux sshd[32256]: Failed password for invalid user user1 from port 55850 ssh2
Feb 3 20:36:25 linux sshd[32268]: Failed password for invalid user user3 from port 36610 ssh2
Feb 3 20:36:52 linux sshd[32274]: Failed password for invalid user user4 from port 45514 ssh2
Feb 3 20:37:19 linux sshd[32279]: Failed password for invalid user user5 from port 54262 ssh2
Feb 3 20:37:45 linux sshd[32285]: Failed password for invalid user user2 from port 34755 ssh2
Feb 3 20:38:11 linux sshd[32292]: Failed password for invalid user info from port 43146 ssh2
Feb 3 20:40:50 linux sshd[32340]: Failed password for invalid user peter from port 46411 ssh2
Feb 3 20:43:02 linux sshd[32372]: Failed password for invalid user amanda from port 59414 ssh2
Feb 3 20:43:28 linux sshd[32378]: Failed password for invalid user postgres from port 39228 ssh2
Feb 3 20:43:55 linux sshd[32384]: Failed password for invalid user ftpuser from port 47118 ssh2
Feb 3 20:44:22 linux sshd[32391]: Failed password for invalid user fax from port 54939 ssh2
Feb 3 20:44:48 linux sshd[32397]: Failed password for invalid user cyrus from port 34567 ssh2
Feb 3 20:45:14 linux sshd[32405]: Failed password for invalid user toto from port 42350 ssh2
Feb 3 20:45:42 linux sshd[32410]: Failed password for invalid user sophie from port 50063 ssh2
Feb 3 20:46:08 linux sshd[32415]: Failed password for invalid user yves from port 59818 ssh2
Feb 3 20:46:34 linux sshd[32424]: Failed password for invalid user trac from port 39509 ssh2
Feb 3 20:47:00 linux sshd[32432]: Failed password for invalid user webmaster from port 47424 ssh2
Feb 3 20:47:27 linux sshd[32437]: Failed password for invalid user postfix from port 55615 ssh2
Feb 3 20:47:54 linux sshd[32442]: Failed password for www-data from port 35554 ssh2
Feb 3 20:48:19 linux sshd[32448]: Failed password for invalid user temp from port 43896 ssh2
Feb 3 20:48:46 linux sshd[32453]: Failed password for invalid user service from port 52092 ssh2
Feb 3 20:49:13 linux sshd[32458]: Failed password for invalid user tomcat from port 60261 ssh2
Feb 3 20:49:40 linux sshd[32464]: Failed password for invalid user upload from port 40236 ssh2
Feb 3 20:50:06 linux sshd[32469]: Failed password for invalid user debian from port 48295 ssh2
Feb 3 20:50:32 linux sshd[32479]: Failed password for invalid user apache from port 56437 ssh2
Feb 3 20:51:00 linux sshd[32492]: Failed password for invalid user rds from port 45540 ssh2
Feb 3 20:51:26 linux sshd[32501]: Failed password for invalid user exploit from port 53751 ssh2
Feb 3 20:51:51 linux sshd[32506]: Failed password for invalid user exploit from port 33543 ssh2
Feb 3 20:52:18 linux sshd[32512]: Failed password for invalid user postgres from port 41350 ssh2
Feb 3 21:02:04 linux sshd[32652]: Failed password for invalid user shell from port 54454 ssh2
Feb 3 21:02:30 linux sshd[32657]: Failed password for invalid user radio from port 35462 ssh2
Feb 3 21:02:57 linux sshd[32663]: Failed password for invalid user anonymous from port 44290 ssh2
Feb 3 21:03:23 linux sshd[32668]: Failed password for invalid user mark from port 53285 ssh2
Feb 3 21:03:50 linux sshd[32673]: Failed password for invalid user majordomo from port 34082 ssh2
Feb 3 21:04:43 linux sshd[32684]: Failed password for irc from port 50918 ssh2
Feb 3 21:05:36 linux sshd[32695]: Failed password for root from port 38577 ssh2
Feb 3 21:06:30 linux sshd[32705]: Failed password for bin from port 53564 ssh2
Feb 3 21:06:56 linux sshd[32714]: Failed password for invalid user dev from port 34568 ssh2
Feb 3 21:07:23 linux sshd[32720]: Failed password for root from port 43799 ssh2
Feb 3 21:09:10 linux sshd[32755]: Failed password for invalid user bob from port 50026 ssh2
Feb 3 21:09:36 linux sshd[32761]: Failed password for invalid user r00t from port 58129 ssh2
Feb 3 21:11:50 linux sshd[537]: Failed password for root from port 58358 ssh2

This brute force dictionary attacks often succeed where there is a user with a weak a password, or some old forgotten test user account.
Just recently on one of the servers I administrate I have catched a malicious attacker originating from Romania, who was able to break with my system test account with the weak password tset .

Thanksfully the script kiddie was unable to get root access to my system, so what he did is he just started another ssh brute force scanner to crawl the net and look for some other vulnerable hosts.

As you read in my recent example being immune against SSH brute force attacks is a very essential security step, the administrator needs to take on a newly installed server.

The easiest way to get read of the brute force attacks without using some external brute force filtering software like fail2ban can be done by:

1. By using an iptables filtering rule to filter every IP which has failed in logging in more than 5 times

To use this brute force prevention method you need to use the following iptables rules:
linux-host:~# /sbin/iptables -I INPUT -p tcp --dport 22 -i eth0 -m state -state NEW -m recent -set
linux-host:~# /sbin/iptables -I INPUT -p tcp --dport 22 -i eth0 -m state -state NEW
-m recent -update -seconds 60 -hitcount 5 -j DROP

This iptables rules will filter out the SSH port to an every IP address with more than 5 invalid attempts to login to port 22

2. Getting rid of brute force attacks through use of hosts.deny blacklists

sshbl – The SSH blacklist, updated every few minutes, contains IP addresses of hosts which tried to bruteforce into any of currently 19 hosts (all running OpenBSD, FreeBSD or some Linux) using the SSH protocol. The hosts are located in Germany, the United States, United Kingdom, France, England, Ukraine, China, Australia, Czech Republic and setup to report and log those attempts to a central database. Very similar to all the spam blacklists out there.

To use sshbl you will have to set up in your root crontab the following line:

*/60 * * * * /usr/bin/wget -qO /etc/hosts.deny

To set it up from console issue:

linux-host:~# echo '*/60 * * * * /usr/bin/wget -qO /etc/hosts.deny' | crontab -u root -

These crontab will download and substitute your system default hosts with the one regularly updated on , thus next time a brute force attacker which has been a reported attacker will be filtered out as your Linux or Unix system finds out the IP matches an ip in /etc/hosts.deny

The /etc/hosts.deny filtering rules are written in a way that only publicly known brute forcer IPs will only be filtered for the SSH service, therefore other system services like Apache or a radio, tv streaming server will be still accessible for the brute forcer IP.

It’s a good practice actually to use both of the methods 😉
Thanks to Static (Multics) a close friend of mine for inspiring this article.

Share this on