[#13301] - Regression and performance issues while filling large reports

Category:
Bug report
Priority:
Urgent
Status:
New
Project: Severity:
Critical
Resolution:
Open
Component: Reproducibility:
Always
Assigned to:
0

When we have exceptionally large reports with lots of elements which can be stretched and moved down by other elements above them, then we are seeing a number of issues.

Here is what appears to be going on:
* All of the elements in the fill element container are getting prepared and moved on every single page
* If there are 200 pages then this means that each element is getting prepared 200 times over, and moving each of its dependent elements 200 times over
* It seems that each element has a huge number of dependent elements each of which are getting moved down all the time

When we generate a 200 page report with 10,000 elements it is taking 1 minute to fill it.
When we generate a 240 page report it is taking 2 minutes to fill it.
When we generate a 280 page report it is taking 4 minutes to fill it.

We would expect that 400 pages would take twice as long as 200 pages, instead the time taken is doubling about every 40 pages.

Doing profiling we can see that most of the time is spent in JRFillElement.moveDependantElements() and JRFillElementContainer.prepareElements(...) which calls that.

There appears to be a regression in 6.16.0 that wasn't there in earlier versions such as 5.1.0 (or likewise when using legacy stretching).
In 6.16.0 it now causes stack overflow errors, as moveDependantElements() calls moveDependantElements() recursively (this is not an issue for small reports with only a few dependent elements).

Ideally each element would only get processed once, and when element A moves dependent element B which will in turn move dependent element C then A should not need to move C as well, as the y position of B has already been changed by A, so when B moves C, it will already include that change in y position.

Without fully understanding what the code is doing here are some suggestions which would significantly improve performance (however they need to be checked to make sure they don't break any edge cases):

Consider changing JRFillElement.moveDependantElements():
* Move the calculations which don't change during the loop out of the loop:
int offset = getRelativeY() + getStretchHeight() - getY() - getHeight() + getCollapsedHeightAbove(); for example
then in the loop: int newRelativeY = offset + element.getY() - element.getCollapsedHeightAbove();
* Remove the call to element.moveDependantElements(); as that is causing stack overflow errors (it might need to be replaced with something else)

In JRFillElementContainer.prepareElements(...)
* Directly below JRFillElement element = ySortedElements[i]; in the loop add something like:
if (currentOverflow && isOverflowAllowed && element.getY() > firstY + availableHeight) {
break;
}
this would allow the loop to be quickly exited in case it is already overflowing and so the remaining elements would be processed again on the following pages anyway.
* Move the element.moveDependantElements(); line inside of the if (element.isToPrint()) block.
This makes a huge difference to performance as all of these elements which are not going to be printed again anyway, don't need to move their dependent elements yet again. I can see that the developer is concerned about hidden elements pushing other elements, so there might be more to it than this, however currently Jasper Reports takes hours to generate some reports which obviously is not a workable solution.

v6.16
JasperFillManager
yoseph's picture
3
Joined: Jul 8 2013 - 12:51am
Last seen: 4 days 11 hours ago

2 Comments:

#1

Do you have a self contained runnable report to reproduce the behaviour that you are describing? That would make the investigation much easier.

Regards,
Lucian

#2

Sorry, but we cannot currently provided a self contained runnable report for this.
The reports we are using for this contain confidential data.
We can tell you that all of the elements are getting stretched.
If we were to find time to strip out the confidential data, then what format what you like the self contained runnable report in?
It would probably be faster just to create a report with 10,000 elements, all placed vertically, and filling all of them with data which causes them all to stretch.

Feedback