JasperServer stop working after some days

[fgalarraga]

Hi,
I'm using Jasper 6.3 with Tomcat 8 and Java 1.8 running over Windows Server 2012R2.

The VM is dedicated to Jasper and has 16 GB of RAM with 4 vCPU of 2 Ghz.

Jasper has a low workload but there is a dashboard running 24x7.

The product works with a good performance for 10 or 15 days and then it works too slow until stop to work.Seeing the VM resources I found the Tomcat process is using near 5 GB of RAM and 25 % of CPU

Tomcat shows the log:

13-Oct-2019 04:48:02.369 SEVERE [http-apr-8080-exec-1] org.apache.catalina.core.ApplicationDispatcher.invoke El Servlet.service() para servlet jsp lanzó una excepción java.lang.IllegalStateException: No puedo crear una sesión después de llevar a cabo la respueta at org.apache.catalina.connector.Request.doGetSession(Request.java:2784) at org.apache.catalina.connector.Request.getSession(Request.java:2161) at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:895) at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:229) at org.apache.catalina.core.ApplicationHttpRequest.getSession(ApplicationHttpRequest.java:585) at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:229) at org.apache.catalina.core.ApplicationHttpRequest.getSession(ApplicationHttpRequest.java:585) at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:229) at org.apache.catalina.core.ApplicationHttpRequest.getSession(ApplicationHttpRequest.java:585) at org.apache.catalina.core.ApplicationHttpRequest.getSession(ApplicationHttpRequest.java:530) at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238) at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238) at javax.servlet.http.HttpServletRequestWrapper.getSession(HttpServletRequestWrapper.java:238) at org.apache.jasper.runtime.PageContextImpl.initialize(PageContextImpl.java:134) at org.apache.jasper.runtime.JspFactoryImpl.internalGetPageContext(JspFactoryImpl.java:109) at org.apache.jasper.runtime.JspFactoryImpl.getPageContext(JspFactoryImpl.java:60) at org.apache.jsp.WEB_002dINF.jsp.templates.reportGeneratorProperties_jsp._jspService(reportGeneratorProperties_jsp.java:88) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:396) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:340) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:721) at org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:584) at org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:523) at org.apache.jasper.runtime.JspRuntimeLibrary.include(JspRuntimeLibrary.java:934) at org.apache.jasper.runtime.PageContextImpl.doInclude(PageContextImpl.java:675) at org.apache.jasper.runtime.PageContextImpl.include(PageContextImpl.java:669) at org.apache.tiles.jsp.context.JspTilesRequestContext.include(JspTilesRequestContext.java:103) at org.apache.tiles.jsp.context.JspTilesRequestContext.dispatch(JspTilesRequestContext.java:96) at org.apache.tiles.renderer.impl.TemplateAttributeRenderer.write(TemplateAttributeRenderer.java:44) at org.apache.tiles.renderer.impl.AbstractBaseAttributeRenderer.render(AbstractBaseAttributeRenderer.java:103) at org.apache.tiles.impl.BasicTilesContainer.render(BasicTilesContainer.java:669) at org.apache.tiles.impl.BasicTilesContainer.render(BasicTilesContainer.java:689) at org.apache.tiles.impl.BasicTilesContainer.renderContext(BasicTilesContainer.java:176) at org.apache.tiles.jsp.taglib.InsertTemplateTag.renderContext(InsertTemplateTag.java:91) at org.apache.tiles.jsp.taglib.InsertTemplateTag.render(InsertTemplateTag.java:81) at org.apache.tiles.jsp.taglib.RenderTag.doEndTag(RenderTag.java:220) at org.apache.jsp.WEB_002dINF.decorators.main_jsp._jspx_meth_t_005finsertTemplate_005f15(main_jsp.java:14120) at org.apache.jsp.WEB_002dINF.decorators.main_jsp._jspx_meth_c_005fif_005f11(main_jsp.java:13888) at org.apache.jsp.WEB_002dINF.decorators.main_jsp._jspService(main_jsp.java:2838) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:432) at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:396) at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:340) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:721) at org.apache.catalina.core.ApplicationDispatcher.doInclude(ApplicationDispatcher.java:584) at org.apache.catalina.core.ApplicationDispatcher.include(ApplicationDispatcher.java:523) at com.opensymphony.module.sitemesh.filter.PageFilter.applyDecorator(PageFilter.java:165) at com.opensymphony.module.sitemesh.filter.PageFilter.doFilter(PageFilter.java:63) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.ApplicationDispatcher.invoke(ApplicationDispatcher.java:721) at org.apache.catalina.core.ApplicationDispatcher.processRequest(ApplicationDispatcher.java:466) at org.apache.catalina.core.ApplicationDispatcher.doForward(ApplicationDispatcher.java:391) at org.apache.catalina.core.ApplicationDispatcher.forward(ApplicationDispatcher.java:318) at org.springframework.web.servlet.view.InternalResourceView.renderMergedOutputModel(InternalResourceView.java:238) at org.springframework.web.servlet.view.AbstractView.render(AbstractView.java:264) at org.springframework.web.servlet.DispatcherServlet.render(DispatcherServlet.java:1201) at org.springframework.web.servlet.DispatcherServlet.processDispatchResult(DispatcherServlet.java:986) at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:933) at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:851) at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:953) at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:855) at javax.servlet.http.HttpServlet.service(HttpServlet.java:644) at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:829) at javax.servlet.http.HttpServlet.service(HttpServlet.java:725) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:291) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.jaspersoft.jasperserver.api.security.csrf.JSCsrfGuardFilter.doFilter(JSCsrfGuardFilter.java:74) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.jaspersoft.jasperserver.api.security.csrf.CrossDomainCommunicationFilter.doFilter(CrossDomainCommunicationFilter.java:117) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330) at com.jaspersoft.jasperserver.api.security.IPadSupportFilter.doFilter(IPadSupportFilter.java:67) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.war.common.JSSwitchUserProcessingFilter.doFilter(JSSwitchUserProcessingFilter.java:157) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.invoke(FilterSecurityInterceptor.java:118) at org.springframework.security.web.access.intercept.FilterSecurityInterceptor.doFilter(FilterSecurityInterceptor.java:84) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.access.ExceptionTranslationFilter.doFilter(ExceptionTranslationFilter.java:113) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:113) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.ji.jaxrs.diagnostic.DiagnosticLoggingFilterPRO.doFilter(DiagnosticLoggingFilterPRO.java:43) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.api.metadata.user.service.impl.MetadataAuthenticationProcessingFilter.doFilter(MetadataAuthenticationProcessingFilter.java:98) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.war.util.RequestParameterAuthenticationFilter.doFilter(RequestParameterAuthenticationFilter.java:99) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.multipleTenancy.MTBasicProcessingFilter.doFilter(MTBasicProcessingFilter.java:194) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.war.UserPreferencesFilter.doFilter(UserPreferencesFilter.java:235) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter.doFilter(AbstractAuthenticationProcessingFilter.java:199) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.ji.license.LicenseCheckFilter.doFilter(LicenseCheckFilter.java:103) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.ji.license.JILicenseFilter.doFilter(JILicenseFilter.java:86) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.war.NullFilter.doFilter(NullFilter.java:43) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.war.NullFilter.doFilter(NullFilter.java:43) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.war.UserPreferencesFilter.doFilter(UserPreferencesFilter.java:235) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.api.logging.filter.BasicLoggingFilter.doFilter(BasicLoggingFilter.java:57) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.api.security.WebAppSecurityFilter.doFilter(WebAppSecurityFilter.java:80) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.war.MultipartRequestWrapperFilter.doFilter(MultipartRequestWrapperFilter.java:95) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.war.NullFilter.doFilter(NullFilter.java:43) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.jaspersoft.jasperserver.api.security.encryption.EncryptionFilter.doFilter(EncryptionFilter.java:150) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.context.SecurityContextPersistenceFilter.doFilter(SecurityContextPersistenceFilter.java:87) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at com.ca.bicoe.cajasperserver.preauth.sso.filters.BasicRefererBasedCsrfFilter.doFilter(BasicRefererBasedCsrfFilter.java:110) at org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:342) at org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:192) at org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:160) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.jaspersoft.jasperserver.war.util.SessionDecoratorFilter.doFilter(SessionDecoratorFilter.java:63) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.jaspersoft.jasperserver.war.util.CharacterEncodingFilter.doFilter(CharacterEncodingFilter.java:67) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.jaspersoft.jasperserver.war.httpheaders.ResourceHTTPHeadersFilter.doFilter(ResourceHTTPHeadersFilter.java:77) at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:343) at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:260) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at com.jaspersoft.jasperserver.war.P3PFilter.doFilter(P3PFilter.java:43) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:239) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:618) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:142) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79) at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:610) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:537) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1081) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:658) at org.apache.coyote.http11.Http11AprProtocol$Http11ConnectionHandler.process(Http11AprProtocol.java:277) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2403) at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2392) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Unknown Source)[/code]

The workaround I've been using is recycle the Tomcat Windows service. When service starts, Japser works for 10 or 15 days with low resources usage until the problem appears again.

What could be the problem root cause? Any idea?

Regards,

Fabio.

[danoldenkampgmail.com]

I believe 6.3 may be your version of Jaspersoft studio.  If not you should update.

What version of Jaspersoft Server?  Current is 7.2.

Look in jasperserver.properties

jasperreportsinstallpath(often on c:)apache-tomcatwebappsjasperserverWEB-INFclasses

https://community.jaspersoft.com/wiki/where-find-jasperreports-server-release-version-information

 

Looks like an exception caused the service to fail.

apache.catalina.core.ApplicationDispatcher.invoke The Servlet.service () for servlet jsp threw an exception java.lang.IllegalStateException : I cannot create a session after performing the response at org.apache.catalina.connector.Request.doGetSession (Request.java:2784) at org.apache.catalina.connector.Request.getSession 

 

Does the service stop?  You can try setting the Jasperreports Tomcat service to recover.

Have you look at the server application event log for Tomcat errors?

You can set Windows up so certain events trigger a script to recycle the service if the above doesn't work.

https://blogs.technet.microsoft.com/wincat/2011/08/25/trigger-a-powershell-script-from-a-windows-event/

Execute jasperserver batch file to stop and start.

servicerun.bat

servicerun.bat START

[fgalarraga]

Thank you.

JasperServer is 6.3 version. I use it as a component in a CA Technologies suite (Service Management and Infrastructure Management).

Right now, I'm analyzing the solution compatibility matrix to upgrade Jasper.

Jasper Windows service does not stop. The problem looks like a memory leak of version I use.

I'm thinking in a periodical service recycle (perhaps once a week) as a workaround until I can upgrade the version.

Regards,

Fabio.