URGENT Stale Connection Issue!!

[memory721]

 

My company has been struggling with JS since August.  When I restart the server, I can run reports with no problems.  However, after a period of latency (approximately 1.5 hours) when I try to log in and run reports, I run into problems like the system just hangs and I can’t go any further.  Working with the DBAs at our company we are seeing that when I’m logged in, several other connections are just popping up and then there are connections that just stay open for days on end. 

Have you heard of this situation before?  Do you have any suggestions for us because we are running out of ideas here.

[shmee]

I have not heard or seen that before but it might help us track the issue down if you posted any relevant logs, and your system configuration (eg. server version, OS it is running on, amount of RAM etc.)

 

You also may want to check the power saving config on your NIC on the server, to make sure it is not going to sleep or something lame like that.

[tkavanagh]

Have you looked at <js-install>/scipts/extra-configs folder?

There are a couple of samples that show how to add connectionPooling properties that can help make sure that DB connections do not "go stale" or "die". Not sure whether this is the issue you are getting but perhaps this can help.

There is definitely a well-known connection pooling issue where connections "go stale" after 12 hours or so. So, the additional properties are used to get rid of the old ones and create fresh ones.

(And in JS 4.0, these extra-configs are found in:

  <js-install>/buildomatic/samples/extra/extra-configs)

[memory721]

Thank you so much for this suggestion.  We are going to be implementing these changes today and testing to see if this takes care of our problems.  Thank you so very much!

[memory721]

We tried the fixes you suggested and unfortunately we did not see any improvement.  The interesting thing is that we scheduled a report to run every 15 minutes and that seemed to keep the system from hanging.  As soon as we deleted the scheduled reports, our problems came back.  It seems as though we need something to keep the connections warm otherwise Jasperserver seems to fall asleep and we get the system to hang.  Have you seen this before?

[tkavanagh]

Oh, another thing to check is something with Tomcat that has always been very confusing.

Tomcat takes the:

    <tomcat>/webapps/{web-app-folder}/META-INF/context.xml

and copies it  to:

    <tomcat>/conf/Catalina/localhost/{web-app-name}.xml

So, in the case of jasperserver, you might see a file here:

    <tomcat>/conf/Catalina/localhost/jasperserver.xml

*Then*, tomcat uses the datasource connection file found in conf and *not* the one that you update inside the jasperserver war file.

So, maybe the connection pooling changes you are making are not actually getting used...? I have found that setting the connection pooling properties fixed these kinds of timeout issues.

You can manually delete the xml file found in the conf folder. Then you new changes *must* take effect.

This issue is also discussed in the Troubleshooting Appendix of the Install Guide. See item A.2.3 "Special Configuration Case under Tomcat"

Hopefully, this will help...

[memory721]

This is really interesting.  We are going to start testing this solution.  thank you for the information.  I will definitely update you with our findings.

 

Thank you so much!

 

 

[memory721]

After speaking with the JS admin, I have found that he did not use the WAR file distribution for the app.  We are using a packaged Ant build script for their source code project.

Any other ideas you have will be greatly appreciated!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

 

Thank you.