OBIEE11g: Managed server services Stuck: java.lang.OutOfMemoryError: GC overhead limit exceeded error

One fine day, if you suddenly face an issue of OBIEE being not working, managed server services in ‘Hang’ or ‘STUCK’ state, try to restart the services but no luck etc. Reason behind all this scenarios could be the following:

java.lang.OutOfMemoryError: GC overhead limit exceeded.

you will find above error in bi_server.log (bi_server is the name of your managed server) at below path:

/$OBIEE_HOME$/user_projects/domains/bifoundation_domain/servers/bi_server1/logs/bi_server1.log.

Error: looks like this:

####<Jun 12, 2017 8:22:37 PM IST> <Error> <HTTP> <etdcrpt02> <bi_server1> 
<[STUCK] ExecuteThread: '11' for queue: 
'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> 
<005KKckpyjA4qmmpsoL6iX0001CH00001^> <1497279157635> <BEA-101017> 
<[ServletContext@1533767336[app:bisecurity module:bisecurity path:
/bisecurity spec-version:2.5 version:11.1.1]] 
Root cause of ServletException.
java.lang.OutOfMemoryError: GC overhead limit exceeded

Cause of error:

 The detail message "GC overhead limit exceeded" indicates that the 
garbage collector is running all the time and Java program is making very 
slow progress. After a garbage collection, if the Java process is spending 
more than approximately 98% of its time doing garbage collection and if it 
is recovering less than 2% of the heap and has been doing so far the last 
5 (compile time constant) consecutive garbage collections, 
then a java.lang.OutOfMemoryError is thrown. This exception is typically 
thrown because the amount of live data barely fits into the Java heap 
having little free space for new allocations.

Solution:

Instant: restart all services (admin server, node manager, managed server and OPMN).

while restarting managed server, first you need to stop/force stop (to force stop use kill -9 PID ) managed server.

Permanent fix: 

Increase the heap size. The java.lang.OutOfMemoryError exception for 
GC Overhead limit exceeded can be turned off with the command line 
flag -XX:-UseGCOverheadLimit.

 

 

 

Thanks till next time

Akash (@akashdgarg)

 

 

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s