AWS Jaspersoft Performance

[mjlib.cam]

Hi,

 

We have the solution Amazon web service (AWS) with a m2.4xlarge instance, which has 68.4 RAM and 8 cores.

 

We also have the following settings for java, which comes by default in the AMI

 

("m2.4xlarge") echo "-Xms2048m-Xmx62000m-XX: MaxPermSize = 2048m-Xss2m";

 

As database we use 'Redshift', which is very fast. And we limit data downloaded to 1000000 rows

 

But ... the performance of the tool is not as good as expected.

 

Many reports we build, after much waiting, simply stay blank or not loaded.

 

We review the log of the tool but does not show any errors.

[abroitman]

Hi,

I'm sorry to hear you're not getting the performance you expected from the Jaspersoft server. We have many users and use cases of successful large deployments with high performance so I'm confident we can, with some more details, identify the root cause for the issues you outline.

RedShift is indeed a very fast BigData DWH, however, there are a few places performance issues can initiate, from the queries optimization, cache settings and Jasperserver configuration. Can you answer the following questions:

1. Are you experiencing the issues in reports or when using Ad-Hoc data exploration
   1. Are you trying to list 1,000,000 rows or are you aggregating them?
   2. Is the issue happen when rendering a large table of data, chars or both?
2. Can you share the configuration of your AWS Ad-Hoc (Go to Manage -> Server Setting -> Ad-Hoc settings)
   1. Ad Hoc Dataset Row Limit
   2. Ad Hoc Query Timeout (seconds)
   3. Optimize Queries for Domain-based Reports
   4. Optimize Queries for JDBC-based Reports

With this information we can begin estimating where the issue may be located and focus our next steps.

Thanks,

Assaf

 

[mjlib.cam]

Hi, thanks for the answer

1. Are you experiencing the issues in reports or when using Ad-Hoc data exploration: In both.
   1. Are you trying to list 1,000,000 rows or are you aggregating them? The data source is much larger, has 70 million of data
      but aggregated data reach the million.
      
      
      
   2. Is the issue happen when rendering a large table of data, chars or both? Large tables
      
   
   
   
2. Can you share the configuration of your AWS Ad-Hoc (Go to Manage -> Server Setting -> Ad-Hoc settings)
   1. Ad Hoc Dataset Row Limit:  1000000
      
   2. Ad Hoc Query Timeout (seconds): 3600
      
   3. Optimize Queries for Domain-based Reports: checked
      
   4. Optimize Queries for JDBC-based Reports: checked
      
   
   
   

[abroitman]

Thank you for the extra details. I have a few follow up questions based on the above.

1. When using ad-hoc, are you using Topics or Domains? 
2. Can you please attach a screenshot of your ad-hoc cache after trying to run one of the reports which fail. (Go to manage -> server settings -> Ad-Hoc Cache -> Find your query). If you cannot add a screenshot, please share the results for the following:
   1. Query/Fetch
   2. Memory Used
   3. Rows (Click on the query to get this data point)
3. Please share the server logs for your query execution, to configure the server to track your execution do the following:
   1. As superuser, go to Manage/Server settings/Log settings, and set the following to DEBUG:
      • Ad Hoc cache activity (CachedData): Logs the lifetime of cached data sets
      • Ad Hoc AJAX requests (adhocAjaxRequests): Logs details about server requests for the Ad Hoc UI, including timings
   2. Run your ad-hoc report
   3. Search Jaspersoft server log for the events above (use name in parenthesis as the search term, i.e CachedData)
   4. Post the results

Thanks,

Assaf

[mjlib.cam]

Hi, 

1. When using ad-hoc, are you using Topics or Domains? Domains

-----------------------------------------------Open Ad-hoc View Log----------------------------------------------------------------

2013-11-28 21:35:55,633  WARN CommonDomainDataStrategy,http-bio-8443-exec-305:880 - error while trying to retrieve query native string

2013-11-28 21:36:13,073  INFO adhocAjaxRequests,http-bio-8443-exec-297:214 - start request 0: action = loadState clientKey = 3963260150995

2013-11-28 21:36:13,088  INFO adhocAjaxRequests,http-bio-8443-exec-298:214 - start request 1: action = generateFilterPanel clientKey = 3963260150995

2013-11-28 21:36:13,089 DEBUG adhocAjaxRequests,http-bio-8443-exec-298:220 - decorate = no

2013-11-28 21:36:13,098  WARN CommonDomainDataStrategy,http-bio-8443-exec-297:880 - error while trying to retrieve query native string

2013-11-28 21:36:13,103  INFO adhocAjaxRequests,http-bio-8443-exec-298:230 -   end request 1: time = 15 ms

2013-11-28 21:36:13,114  INFO adhocAjaxRequests,http-bio-8443-exec-297:230 -   end request 0: time = 41 ms

2013-11-28 21:37:50,966  INFO adhocAjaxRequests,http-bio-8443-exec-307:214 - start request 2: action = addAdhocFilter clientKey = 3963260150995

2013-11-28 21:37:50,967 DEBUG adhocAjaxRequests,http-bio-8443-exec-307:220 - addAdhocFilterFields = juro_latam_emitido_ago_leg.file_date

2013-11-28 21:37:51,042 DEBUG JRJdbcQueryExecuter,http-bio-8443-exec-307:254 - SQL query string: select "file_date"

from "juro_latam"."emitido_ago_leg"

group by "file_date"

order by "file_date"

 limit 1000001

 

2013-11-28 21:38:09,383  INFO adhocAjaxRequests,http-bio-8443-exec-307:230 -   end request 2: time = 18417 ms

2013-11-28 21:38:09,393 DEBUG adhocAjaxRequests,http-bio-8443-exec-307:23 - modFlags = [addAdhocFilter]

2013-11-28 21:38:46,832  INFO adhocAjaxRequests,http-bio-8443-exec-304:214 - start request 3: action = addAdhocFilter clientKey = 3963260150995

2013-11-28 21:38:46,833 DEBUG adhocAjaxRequests,http-bio-8443-exec-304:220 - addAdhocFilterFields = juro_latam_emitido_ago_leg.msttkt

2013-11-28 21:38:46,930 DEBUG JRJdbcQueryExecuter,http-bio-8443-exec-304:254 - SQL query string: select "msttkt"

from "juro_latam"."emitido_ago_leg"

group by "msttkt"

order by "msttkt"

 limit 1000001

 

thanks