Saturday, October 29, 2011

Experimenting with ADF BC Application Module Pool Tuning

This week we were performing advanced ADF BC Application  Module pool tuning. There are few lessons learned, I would like to share with you. Tests were performed using latest JDeveloper/ADF release 11g R2 (11.1.2.1.0) (Build 6081).  Test case is based on sample application from Optimizing Oracle ADF Application Pool post. This sample contains two Application Modules, one is with default AM tuning settings, second AM is tuned:
  • 2 minutes Time to Live
  • 1 minute Idle Instance Timeout
  • 0.5 minute Pool Pooling Interval
Experiment:
  • Deploy sample application with AM tuning into ADF Library
  • Import ADF Library into Main Application
  • Main Application contains its own local AM
  • Verify if tuning for AM from ADF Library was applied successfully
Lessons learned:
  • There is no guarantee that AM tuning options will be applied always
  • Applied AM tuning options might depend on first AM initialized
  • AM Reserved mode might be more stable, comparing to AM Managed mode
AM passivation/activation relationship with database transaction:
  • AM passivation happens - database transaction is lost for current user (database lock is lost)
  • Make sure no AM passivation happens during transaction period
By default, AM is running in Managed mode, this means AM is running with AM pool enabled. If we need to support critical transactional operations, we need to ensure AM Passivation will not happen during predefined period of time. However, it looks like there is no guarantee, even after applying proper tuning parameters - AM running in Managed mode, can be still passivated (even there is enough space in AM pool). This is a main reason, why you may prefer to run AM in Reserved mode, to ensure long running transactions. Read more about Managed and Reserved modes - 43.2.2.3 About State Management Release Levels. Please test all tuning options carefully, before applying.

Download test cases for today experiment - DatabaseConnectionLeak_v2.zip and MainApp.zip.

Experiment: ADF BC Application Module Pool Tuning for ADF Application with Multiples AM's

Test case application implements two AM's and two ADF TF's for each AM:


This application is deployed into ADF library and imported into Main application. As described above, one AM is tuned to be destroyed in 2 minutes:


Main application implements two physical JSPX pages. First page brings data from local AM, second page integrates ADF regions from imported library:


As you can see from DataBindings file for Main application, there is local Data Control usage declared:


If we check the same for application packaged into ADF library, this application contains two Data Control usages - both pointing to different AM's (one AM is tuned):


Test was performed with Shared Data Control and with Isolated Data Control - there was no difference in results. Default - Shared Data Control for ADF task flow:


How to reproduce performed test:

1. Run Main application and access first page - local Data Control is loaded:


2. Press Edit button - navigate to second page. Data Control from imported library will be loaded:


3. Imported library contains tuning for AM to be destroyed after 2 minutes of inactivity. This tuning option is ignored, AM instance still keeps database connection active even after 2 minutes of inactivity:


4. From Main application, expand Data Controls section. Expand HrModuleDataControl imported from ADF library, drag and drop EmployeesView1 into first page of Main application:



5. This time, AM from imported library gets initialized much earlier, when first page from Main application is loaded. We have different behavior now - AM tuning options are applied and AM instance is destroyed, database connection is released after 2 minutes of inactivity:


Main goal of this post was to describe inconsistent behavior of AM pool tuning. If you will experience similar issues with AM tuning, please report through Oracle Metalink - this will help to improve AM pooling.

Update - based on Chris Muir request, showing number of AM pools. There are 3 AM's and 1 active user - it creates 3 AM pools (11g R1):


One of 3 AM's is configured with 2 minutes time to live, but as we can see - AM instance remains active in the pool, even after 2 minutes time:


6 comments:

Chris Muir said...

Hi Andrejus

Regards the database lock being lost, do you think this is why Oracle recommends we use optimistic locking? Obviously with optimistic locking the lock is delayed till commit time & processed within 1 request, such that passivation isn't an issue.

CM.

Chris Muir said...

Hi Andrejus

Sorry for the double post, pecking on my iPad.

However in considering the post, can you elaborate the number of AM pools via DMS Spy or FMW Control please? Why I ask is I think you're seeing a 11.1.1.X.0 behavior for AMs not a 11.1.2.X.0 behavior which is curious. But the number of AM pools will indicate what's going on. You'll need to check the number of AM pools in both scenarios.

Regards,

CM.

Andrej Baranovskij said...

Hi Chris,

- Optimistic lock is not suitable in such case, when we need to run long running transactions (lets say keep lock for specific user for 0.5 hour). Often even pessimistic locking is not enough, we need to set lock programmatically when user is accessing record (for example banking application, when records should be locked and no change allowed until transaction not completed)

- Generally, yes - correct, its why Oracle recommends Optimistic. We did several tests, with Pessimistic locking, in stress test environment - lock sometimes can be lost during a bit longer wait time.

- I have tested with both 11g R1 and 11g R2, same results. Regarding number of AM pools I will double check and will post my findings here.

Andrejus

Chris Muir said...

Hi Andrejus

Certainly I understand the technical need for "better-than" optimistic locking, was just wondering if what you posted provides the key why Oracle recommend us to use optimistic locking.

As for AM pools, how many are there? From my understanding 11.1.2.X should create 3, but I wonder if you're seeing less?

Regards,

CM.

Andrej Baranovskij said...

Hi Chris,

Is hard to say, if this explains why Oracle recommends to use optimistic locking. I guess - no. Most probably, this is just some issue related to AM pools.

I have checked how many AM pools are created in 11g R1. There are 3 AM's in the system, so it created 3 AM pools (I uploaded screenshots as well, end of post). Each AM pool contains active AM instances. All TF's are accessed directly, there is no nested TF call for sample app.

I can't test right now for 11g R2, how many AM pools are created - I will need to install FMW control on standalone WLS server for 11g R2. But I will keep you posted.

Andrejus

M.Ahmed said...

Hellow Andrejus,
I want to know the best practice for Application module
Is it better to use one application module for my application or multiple application modules and if I will use more than one AM
what is the better to use nested app module or separated app module.
and what is the optimal number of views that I put in the app module

as I have a large application that serves about 350 user and this application consist of one application module and the passivation problem happens a lot.

I know this is a lot of question but I really need help in this.