Hi,
We have the solution Amazon web service (AWS) with a m2.4xlarge instance, which has 68.4 RAM and 8 cores.
We also have the following settings for java, which comes by default in the AMI
("m2.4xlarge") echo "-Xms2048m-Xmx62000m-XX: MaxPermSize = 2048m-Xss2m";
As database we use 'Redshift', which is very fast. And we limit data downloaded to 1000000 rows
But ... the performance of the tool is not as good as expected.
Many reports we build, after much waiting, simply stay blank or not loaded.
We review the log of the tool but does not show any errors.
4 Answers:
Posted on November 26, 2013 at 10:41am
Hi,
I'm sorry to hear you're not getting the performance you expected from the Jaspersoft server. We have many users and use cases of successful large deployments with high performance so I'm confident we can, with some more details, identify the root cause for the issues you outline.
RedShift is indeed a very fast BigData DWH, however, there are a few places performance issues can initiate, from the queries optimization, cache settings and Jasperserver configuration. Can you answer the following questions:
- Are you experiencing the issues in reports or when using Ad-Hoc data exploration
- Are you trying to list 1,000,000 rows or are you aggregating them?
- Is the issue happen when rendering a large table of data, chars or both?
- Can you share the configuration of your AWS Ad-Hoc (Go to Manage -> Server Setting -> Ad-Hoc settings)
- Ad Hoc Dataset Row Limit
- Ad Hoc Query Timeout (seconds)
- Optimize Queries for Domain-based Reports
- Optimize Queries for JDBC-based Reports
With this information we can begin estimating where the issue may be located and focus our next steps.
Thanks,
Assaf
Posted on November 26, 2013 at 11:04am
Hi, thanks for the answer
- Are you experiencing the issues in reports or when using Ad-Hoc data exploration: In both.
- Are you trying to list 1,000,000 rows or are you aggregating them? The data source is much larger, has 70 million of databut aggregated data reach the million.
- Is the issue happen when rendering a large table of data, chars or both? Large tables
- Are you trying to list 1,000,000 rows or are you aggregating them? The data source is much larger, has 70 million of data
- Can you share the configuration of your AWS Ad-Hoc (Go to Manage -> Server Setting -> Ad-Hoc settings)
- Ad Hoc Dataset Row Limit: 1000000
- Ad Hoc Query Timeout (seconds): 3600
- Optimize Queries for Domain-based Reports: checked
- Optimize Queries for JDBC-based Reports: checked
Posted on November 26, 2013 at 3:30pm
Thank you for the extra details. I have a few follow up questions based on the above.
- When using ad-hoc, are you using Topics or Domains?
- Can you please attach a screenshot of your ad-hoc cache after trying to run one of the reports which fail. (Go to manage -> server settings -> Ad-Hoc Cache -> Find your query). If you cannot add a screenshot, please share the results for the following:
- Query/Fetch
- Memory Used
- Rows (Click on the query to get this data point)
- Please share the server logs for your query execution, to configure the server to track your execution do the following:
- As superuser, go to Manage/Server settings/Log settings, and set the following to DEBUG:
- Ad Hoc cache activity (CachedData): Logs the lifetime of cached data sets
- Ad Hoc AJAX requests (adhocAjaxRequests): Logs details about server requests for the Ad Hoc UI, including timings
- Run your ad-hoc report
- Search Jaspersoft server log for the events above (use name in parenthesis as the search term, i.e CachedData)
- Post the results
- As superuser, go to Manage/Server settings/Log settings, and set the following to DEBUG:
Thanks,
Assaf
Posted on November 28, 2013 at 1:40pm
Hi,
- When using ad-hoc, are you using Topics or Domains? Domains
-----------------------------------------------Open Ad-hoc View Log----------------------------------------------------------------
2013-11-28 21:35:55,633 WARN CommonDomainDataStrategy,http-bio-8443-exec-305:880 - error while trying to retrieve query native string
2013-11-28 21:36:13,073 INFO adhocAjaxRequests,http-bio-8443-exec-297:214 - start request 0: action = loadState clientKey = 3963260150995
2013-11-28 21:36:13,088 INFO adhocAjaxRequests,http-bio-8443-exec-298:214 - start request 1: action = generateFilterPanel clientKey = 3963260150995
2013-11-28 21:36:13,089 DEBUG adhocAjaxRequests,http-bio-8443-exec-298:220 - decorate = no
2013-11-28 21:36:13,098 WARN CommonDomainDataStrategy,http-bio-8443-exec-297:880 - error while trying to retrieve query native string
2013-11-28 21:36:13,103 INFO adhocAjaxRequests,http-bio-8443-exec-298:230 - end request 1: time = 15 ms
2013-11-28 21:36:13,114 INFO adhocAjaxRequests,http-bio-8443-exec-297:230 - end request 0: time = 41 ms
2013-11-28 21:37:50,966 INFO adhocAjaxRequests,http-bio-8443-exec-307:214 - start request 2: action = addAdhocFilter clientKey = 3963260150995
2013-11-28 21:37:50,967 DEBUG adhocAjaxRequests,http-bio-8443-exec-307:220 - addAdhocFilterFields = juro_latam_emitido_ago_leg.file_date
2013-11-28 21:37:51,042 DEBUG JRJdbcQueryExecuter,http-bio-8443-exec-307:254 - SQL query string: select "file_date"
from "juro_latam"."emitido_ago_leg"
group by "file_date"
order by "file_date"
limit 1000001
2013-11-28 21:38:09,383 INFO adhocAjaxRequests,http-bio-8443-exec-307:230 - end request 2: time = 18417 ms
2013-11-28 21:38:09,393 DEBUG adhocAjaxRequests,http-bio-8443-exec-307:23 - modFlags = [addAdhocFilter]
2013-11-28 21:38:46,832 INFO adhocAjaxRequests,http-bio-8443-exec-304:214 - start request 3: action = addAdhocFilter clientKey = 3963260150995
2013-11-28 21:38:46,833 DEBUG adhocAjaxRequests,http-bio-8443-exec-304:220 - addAdhocFilterFields = juro_latam_emitido_ago_leg.msttkt
2013-11-28 21:38:46,930 DEBUG JRJdbcQueryExecuter,http-bio-8443-exec-304:254 - SQL query string: select "msttkt"
from "juro_latam"."emitido_ago_leg"
group by "msttkt"
order by "msttkt"
limit 1000001
thanks
