/ java.lang.OutOfMemoryError – Weblogic Session size too big ~ Java EE Support Patterns

11.18.2011

java.lang.OutOfMemoryError – Weblogic Session size too big

A major problem was brought to our attention recently following a migration of a Java EE Portal application from Weblogic 8.1 to Weblogic 11g.

This case study will demonstrate how you can analyze an IBM JRE Heap Dump (.phd format) in order to determine the memory footprint of your application HttpSession objects.

Environment specifications (case study)

-         Java EE server: Oracle Weblogic Server 11g
-         Middleware OS: AIX 5.3
-         Java VM: IBM JRE 1.6.0
-         Platform type: Portal application

Monitoring and troubleshooting tools

-         Memory Analyzer 1.1 via IBM support assistant (IBM  JRE Heap Dump analysis)

Step #1 – Heap Dump generation

A Heap Dump file was generated following an OutOfMemoryError. The IBM JRE Heap Dump format is as per below (phd extension stands for Portal Heap Dump).

* Please note that you can also manually generate a IBM JRE Heap Dump by either using dumpHeap JMX operation via JConsole or by adding IBM_HEAPDUMP=true in your Java environment variables along with kill –QUIT command *

// Portal Heap Dump generated on 2011-09-22
heapdump.20110922.003510.1028306.0007.phd

Step #2 – Load the Heap Dump file in MAT

We used the Memory Analyzer (MAT) tool from IBM Support Assistant in order to load our generated Heap Dump. The Heap Dump parsing process can take quite some time. Once the processing is completed, you will be able to see a Java Heap overview along with a Leak Suspect report.




Step #3 – Locate the Weblogic Session Data objects

In order to understand the runtime footprint of your application HttpSession (user session) objects, you first need to locate the Weblogic HttpSession internal data structure. Such internal data structured with using memory Session persistence is identified as:

weblogic.servlet.internal.session.MemorySessionData

Unless these objects are showing up from the Leak Suspects report, the best way to locate them is to simply load the Histogram as per below and sort the Class Names. You should be able to easily locate the MemorySessionData class name and determine how many Objects instances. 1 instance of MemorySessionData corresponds to one user session of one of your Java EE Web Application.










Step #4 – HttpSession memory footprint analysis

It is now time to analyze the memory footprint of each of your HttpSession data. Simply right click on your mouse over weblogic.servlet.internal.session.MemorySessionData
 and select: List objects > with incoming references


For our case study, large HttpSession (MemorySessionData) objects were found using up to 52 MB for a single session objects which did explain why our Java Heap was depleted under heavy load.

At this point, you can explore the Session data object dig within a single instance of MemorySessionData. This will allow you to look at all your application Session data attributes and determine the source(s) of memory allocation. Simply select right click on one MemorySessionData and select >> List objects > with outgoing references.


Using this approach, our development team was able to identify the source of high memory allocation within our application HttpSession data and fix the problem.

Conclusion

I hope this tutorial along with our case study has helped you understand how useful and powerful a Heap Dump analysis can be in order to understand and identify your application HttpSession memory footprint.

Please don’t hesitate to post any comment or question.

4 comments:

Hi P_H,
Do you have any information regarding this?
java.lang.OutOfMemoryError: getNewTla

And what is your recomendation on what type of JVM should be used for Weblogic 10.3 on Linux 64bit machine?

Thanks,
~Sujoy

Hi Sujoy,

java.lang.OutOfMemoryError: getNewTla

This is a common problem when using the Oracle Jrockit VM. Quite often we believe that this is a TLA tuning problem but in fact is often symptom of Java Heap depletion. Do you have history of your JRockit Java Heap e.g. verbose:gc, monitoring tool etc. so we can confirm? Also, if you are using Jrockit R28+, JVM Heap Dump can be generated automatically from OOM events.

For WL103 and Linux 64-bit, my recommended JVM is Sun HotSpot 64-bit version. Again it depends of the size of your Java EE environment that you are planning to deploy and the hardware specifications. Can you please elaborate on the hardware specs & CPU core availablity?

Thanks.
P-H

Hi P-H,
we are seeing getnewTla problem. Here are our hardware specs:
-XXtlaSize:min=8k,preferred=512k,wasteLimit=8k -Xms3584m -Xmx3584m -Xverify:none -da . When we got this error we increased the tlasize from 4k to 8k . Still we are not able to see any improvement. version of WebLogic Server is 10.3.3.0 and linux 64 bit . jrockit with java version as
1.6.0_29. Can you help in this regard.

Hi Hemanth,

Can you please post the exact error that you are getting from the JRockit JVM?

Thanks.

Post a Comment