[#13301] - Regression and performance issues while filling large reports

Category:
Bug report
Priority:
Urgent
Status:
New
Project: Severity:
Critical
Resolution:
Open
Component: Reproducibility:
Always
Assigned to:
0

When we have exceptionally large reports with lots of elements which can be stretched and moved down by other elements above them, then we are seeing a number of issues.

Here is what appears to be going on:
* All of the elements in the fill element container are getting prepared and moved on every single page
* If there are 200 pages then this means that each element is getting prepared 200 times over, and moving each of its dependent elements 200 times over
* It seems that each element has a huge number of dependent elements each of which are getting moved down all the time

When we generate a 200 page report with 10,000 elements it is taking 1 minute to fill it.
When we generate a 240 page report it is taking 2 minutes to fill it.
When we generate a 280 page report it is taking 4 minutes to fill it.

We would expect that 400 pages would take twice as long as 200 pages, instead the time taken is doubling about every 40 pages.

Doing profiling we can see that most of the time is spent in JRFillElement.moveDependantElements() and JRFillElementContainer.prepareElements(...) which calls that.

There appears to be a regression in 6.16.0 that wasn't there in earlier versions such as 5.1.0 (or likewise when using legacy stretching).
In 6.16.0 it now causes stack overflow errors, as moveDependantElements() calls moveDependantElements() recursively (this is not an issue for small reports with only a few dependent elements).

Ideally each element would only get processed once, and when element A moves dependent element B which will in turn move dependent element C then A should not need to move C as well, as the y position of B has already been changed by A, so when B moves C, it will already include that change in y position.

Without fully understanding what the code is doing here are some suggestions which would significantly improve performance (however they need to be checked to make sure they don't break any edge cases):

Consider changing JRFillElement.moveDependantElements():
* Move the calculations which don't change during the loop out of the loop:
int offset = getRelativeY() + getStretchHeight() - getY() - getHeight() + getCollapsedHeightAbove(); for example
then in the loop: int newRelativeY = offset + element.getY() - element.getCollapsedHeightAbove();
* Remove the call to element.moveDependantElements(); as that is causing stack overflow errors (it might need to be replaced with something else)

In JRFillElementContainer.prepareElements(...)
* Directly below JRFillElement element = ySortedElements[i]; in the loop add something like:
if (currentOverflow && isOverflowAllowed && element.getY() > firstY + availableHeight) {
break;
}
this would allow the loop to be quickly exited in case it is already overflowing and so the remaining elements would be processed again on the following pages anyway.
* Move the element.moveDependantElements(); line inside of the if (element.isToPrint()) block.
This makes a huge difference to performance as all of these elements which are not going to be printed again anyway, don't need to move their dependent elements yet again. I can see that the developer is concerned about hidden elements pushing other elements, so there might be more to it than this, however currently Jasper Reports takes hours to generate some reports which obviously is not a workable solution.

v6.16
JasperFillManager
yoseph's picture
3
Joined: Jul 8 2013 - 12:51am
Last seen: 2 months 1 week ago

6 Comments:

#1

Do you have a self contained runnable report to reproduce the behaviour that you are describing? That would make the investigation much easier.

Regards,
Lucian

#2

Sorry, but we cannot currently provided a self contained runnable report for this.
The reports we are using for this contain confidential data.
We can tell you that all of the elements are getting stretched.
If we were to find time to strip out the confidential data, then what format what you like the self contained runnable report in?
It would probably be faster just to create a report with 10,000 elements, all placed vertically, and filling all of them with data which causes them all to stretch.

#3

A JRXML that can be filled with an empty data source - as in new JREmptyDataSource(1000) - you be ideal.

I'm not sure I understand the report that you suggest creating. Do you mean a report with 10,000 design elements, or elements that repeat 10,000 times when the report get filled?

#4

I don't believe an empty data source would be sufficient to reproduce the issue.
The text needs to be long enough to cause each element to stretch beyond its original size.

We are talking about a report with 10,000 design elements.

#5

10,000 design elements sounds like an atypical use case to me. Could you describe the use case? Maybe there's another way to achieve the same result without so many design elements.

#6

We dynamically create reports.
Sometimes users require reports which include many years of data and hence require a huge number of design elements.
Most of the time the reports the users generate are small (hence no issue with those ones).
In our industry there is no way of avoiding these huge reports.
Some users really do need 800 page reports on occasion.

To avoid the regression we are using an older version of Jasper Reports before the regression was introduced.
We need these things fixed in Jasper Reports, not to change the use cases.

Feedback
randomness