Memory leak printing large report with subreport?

[christiaan_se]

Hi,

I have a simple large report (200.000 record) where each record has a simple sub report of 500 records. Printing it results in a java.lang.OutOfMemoryError: Java heap space after printing 15.000 records. (I also noticed that after printing a couple of thousands or records things really start to slow down, but this is probably caused by this) I use a swapfilevirtualizer and have set Xmx256M When printing just the main report (so without subreport) this doesn't result in OutOfMemory.

 

I've attached a simple testcase to reproduce this.

 

It occurs at:

net.sf.jasperreports.engine.fill.JRVerticalFiller(net.sf.jasperreports.engine.fill.JRBaseFiller).fill(java.util.Map) line: 783

net.sf.jasperreports.engine.fill.JRVerticalFiller(net.sf.jasperreports.engine.fill.JRBaseFiller).fill(java.util.Map, net.sf.jasperreports.engine.JRDataSource) line: 666

net.sf.jasperreports.engine.fill.JRFiller.fillReport(net.sf.jasperreports.engine.JasperReport, java.util.Map, net.sf.jasperreports.engine.JRDataSource) line: 89

net.sf.jasperreports.engine.JasperFillManager.fillReport(net.sf.jasperreports.engine.JasperReport, java.util.Map, net.sf.jasperreports.engine.JRDataSource) line: 601

test.Test.main(java.lang.String[]) line: 37

[file name=testcaseoutofmem.zip size=3860]http://www.jasperforge.org/components/com_joomlaboard/uploaded/files/testcaseoutofmem.zip[/file]

[lucianc]

Hi

 

I don't think the subreport is causing a memory leak. The JVM runs out of memory simply due to the size of the report.

 

To be more specific: your sample produces a 6,900 pages report when the subreport is removed. When the subreport is present and each subreport instance renders 500 rows, empirical tests show that the resulting report would be around 460 times larger, making it around 3 milion pages long.

 

Report virtualization does not guarantee filling of arbitrarily large reports using a constant amount of memory. The required memory is still (directly) proportinal to the report size, because some amount of memory is still consumed by each page of the resulting report (but at a much slower rate than when virtualization is not used).

 

You really need 3 milion pages report, you'll definitely need more than 256M of heap space and you'll also need a huge amount of free space on your disk (and a file system that supports really large files, if you're planning to use the swap file virtualizer).

 

Also, you might revise the parameters for the virtualizer. For example, you set the swap file block size to 100 bytes, which is too small consiudereing that your report pages usually produce 6-7 kbytes of virtual data.

 

HTH,

Lucian

[christiaan_se]

Hi Lucian,

thanks for the reply. I was actually under the wrong impression that the report generation process is streambased, so thats why I thought it was a memory leak. Unfortunately our users do need these kind of reports (not printed, but in pdf format), so I hope I can solve it with the heap space increase.

 

kind regards,

Christiaan

[rahulbhatt04]

Hi

 

I am facing Same error I am using Swap Virtualizer with following param

block Size=1000

Increment = 500

Max Number of Pages in cache=4

I am trying to generate a report with 900000 records with 2 gigs memory on Linux system

I get HTML perfect in case of Excel it generates Swap file correctly but while Expoting I get following error

 

Caused by: java.lang.OutOfMemoryError: Java heap space

        at java.lang.String.<init>(String.java:208)

        at java.lang.StringBuffer.toString(StringBuffer.java:586)

        at java.io.ObjectInputStream$BlockDataInputStream.readUTFBody(ObjectInputStream.java:2968)

        at java.io.ObjectInputStream$BlockDataInputStream.readUTF(ObjectInputStream.java:2764)

        at java.io.ObjectInputStream.readString(ObjectInputStream.java:1567)

        at java.io.ObjectInputStream.readTypeString(ObjectInputStream.java:1375)

        at java.io.ObjectStreamClass.readNonProxy(ObjectStreamClass.java:634)

        at java.io.ObjectInputStream.readClassDescriptor(ObjectInputStream.java:789)

        at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1534)

        at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1466)

        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1699)

        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1305)

        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348)

        at java.util.ArrayList.readObject(ArrayList.java:591)

        at sun.reflect.GeneratedMethodAccessor51.invoke(Unknown Source)

        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)

        at java.lang.reflect.Method.invoke(Method.java:585)

        at java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:946)

        at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1809)

        at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1719)

        at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1305)

        at java.io.ObjectInputStream.readObject(ObjectInputStream.java:348)

        at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.readData(JRAbstractLRUVirtualizer.java:604)

        at net.sf.jasperreports.engine.fill.JRSwapFileVirtualizer.pageIn(JRSwapFileVirtualizer.java:112)

        at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.requestData(JRAbstractLRUVirtualizer.java:493)

        at net.sf.jasperreports.engine.base.JRVirtualPrintPage.ensureVirtualData(JRVirtualPrintPage.java:312)

        at net.sf.jasperreports.engine.base.JRVirtualPrintPage.getElements(JRVirtualPrintPage.java:304)

        at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportPage(JRXlsAbstractExporter.java:443)

        at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportReportToStream(JRXlsAbstractExporter.java:383)

        at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportReport(JRXlsAbstractExporter.java:186)

 

[ggiannini]

Hi,

 

I' have the same memory problem, the swap file virtualizer all works, the trouble come in exporting via JExcelApiExporter!

I developed a Servlet application to generate report from a database using
JasperReport to create the template report print (i used the File Virtualizer to
resolve the initial out of memory issues). But i'm in trouble with the export of
the *.jrprint object to Excel using JExcelApi implemented in the JasperReport
Exporter.

With small report all works, but when generating a large report (about 200000
rows of 15 coloums splitted in two or more sheets of 65536 rows, using the
MAX_ROW_PER_SHEET property in JasperReport) the Out of Memory bring up!

I'm trying to modifying the source code of JExcelApi setting the
useTemporaryFile to True and recompile it: with the small reports the temporary
file is generated and all works, but with large report file, the temporary file
is generated with 0 bytes and then, after a few minutes of memory ram growing to
the max (the Xmx setting is 1024 in JVM), the Out of memory message bring up!

I suppose that the useTemporaryFile works only at one sheet a time, but when the
sheet is large, this setting dont works.

[mcibra]

 Hi, I have the same problem. It is no problem to fill report, but when I am exporting report, is problem with heap space. 

 

Swap file is not solution, because it can help only with filling report, not exporting.

Any suggestions, please?

 

Thanks

 

Michal

 

Code:

JExcelApiExporter exporter= new JExcelApiExporter();		exporter.setParameter(JRXlsExporterParameter.IS_REMOVE_EMPTY_SPACE_BETWEEN_ROWS, Boolean.TRUE);		exporter.setParameter(JRXlsExporterParameter.IS_ONE_PAGE_PER_SHEET,		Boolean.TRUE);		exporter.setParameter(				JRXlsExporterParameter.IS_DETECT_CELL_TYPE,				Boolean.TRUE); 				exporter.setParameter(				JRXlsExporterParameter.IS_WHITE_PAGE_BACKGROUND,				Boolean.FALSE); 				exporter.setParameter(				JRXlsExporterParameter.IS_REMOVE_EMPTY_SPACE_BETWEEN_ROWS,				Boolean.TRUE);						exporter.setParameter(JRExporterParameter.JASPER_PRINT, jpv);		exporter.setParameter(JRExporterParameter.OUTPUT_FILE_NAME,				Enviroment.DIRECTORY_PRINT_TEMP + File.separator + nazovVystupu + "_" + TIME + ".xls");		//Pass the sheet names		exporter.setParameter(JRXlsExporterParameter.SHEET_NAMES, nazvyZositov);		exporter.exportReport();

[lucianc]

mcibra
Wrote:

Hi, I have the same problem. It is no problem to fill report, but when I am exporting report, is problem with heap space.

Then it's not the same problem.

And there's only one solution to your problem, namely increasing the heap size.  Excel exporters use libraries that keep the entire exported workbook in memory, so the export is inherently unscalable.

Regards,

Lucian

[jasperdev11]

When u are running reports, you can easily differentiate between code part, jasper filling part, jasper export part

Jasper filling part will take long time no doubt, you need to set java -Xms512m -Xmx1536m .... your heap size and then set ur swapfilevirtualizer parameters effectively by running some tests.

Ran 6-10 M records 355 MB size file with this memory