Jump to content

Out of memory exception


felipinVzla

Recommended Posts

Hi, i'm using the jasperReports 1.2.5 and when fill a report of 40000 records results in a out of memory exception. The report use a DataSource not a DataBase connection. This problem is happening running the fill in my machine. Also this implementation take the rows in segments of 5000, using the methods of JRAbstractBeanDataSource class.

 

I'm look the sample in demo/samples/virtualizer and the problem still.

 

The jdk is 1.5.06 and the tomcat 5.5.17.

 

And i want to know if the exportReport() method of JRAbstractExporter kepp the report in memory to do the export. Because exporting a excel a report of 65000 records don't work and repeat the out of memory exception. To the excel export i'm using JExcelApiExporter class. This problem is happening doing the report request to a server.

 

Thanks...

Link to comment
Share on other sites

  • Replies 7
  • Created
  • Last Reply

Top Posters In This Topic

Hi,

 

Do you mean you have an enhanced custom data source able to fetch records in multiple steps (data pagination) in order to minimize memory consumption?

If you do, that's good. Maybe you could lower the number of records read in a fetch operation to see if this helps. 5000 records at a time might be too much, depending on the size of the records.

 

If you have used a virtualizer, then the problem might not be related to JasperReports itself, but rather to the way the XLS exporting libraries work. They seem to store the generated XLS document in-memory, even we then save it into a file or send it to an output stream.

So we optimized memory consumption in JasperReports by using a report virtualizer, but we cannot do the same for the exporters, because we rely on third-party libraries such as POI or JExcelAPI which do not seem to have similar mechanisms.

 

And then a third issue. It depends on how you deliver the XLS content to the browser, assuming you are in a Web application. Some browsers need to know the length of the binary content they receive beforehand and this is why our XLS servlets output first to an in-memory byte array output stream, just to be able to measure the resulting document and only after that they output to the response output stream. Fortunately, there is a parameter available to turn this kind of buffering off and save up memory.

 

I hope this helps.

Teodor

Link to comment
Share on other sites

  • 4 months later...

I have the same problem.

 

I'm using POI to export some data from an DB.

 

The problem is that our users (stupid users ) want to extract ~ 400.000 rows of data. I have managed to write in xls file only 10.000 rows and in different workbooks. However, at 200.000 rows Tomcat is throwing an "Out of memory " exception . Tomcat instance can have up to 256 M RAM.

 

I don't want to raise Tomcat virtual memory.

Link to comment
Share on other sites

teodord wrote:

So we optimized memory consumption in JasperReports by using a report virtualizer, but we cannot do the same for the exporters, because we rely on third-party libraries such as POI or JExcelAPI which do not seem to have similar mechanisms.

I was running into the same issue, so I took a look at JRXlsAbstractExporter. I believe I have come up with an approach that optimizes memory consumption even when IS_ONE_PAGE_PER_SHEET is false.

 

The approach basically consists of making two passes over the pages in exportReportToStream. The first pass simply calculates all the xCut values. On the second pass, exportPage is called once per page rather than once at the end. The pre-calculated xCut values are passed to exportPage, and then on to the JRGridLayout constructor. JRGridLayout need only calculate the yCut values for that page. It can use the pre-calculated xCut values. This allows each page to be exported as it is hit rather than all elements on all pages being collected and exported at one time.

 

This seems to work well for me, however, there may be side-effects that I haven't considered. I can provide the modified source code if this is something that interests you.

Link to comment
Share on other sites

greghilton wrote:

teodord wrote:
So we optimized memory consumption in JasperReports by using a report virtualizer, but we cannot do the same for the exporters, because we rely on third-party libraries such as POI or JExcelAPI which do not seem to have similar mechanisms.

I was running into the same issue, so I took a look at JRXlsAbstractExporter. I believe I have come up with an approach that optimizes memory consumption even when IS_ONE_PAGE_PER_SHEET is false.

 

The approach basically consists of making two passes over the pages in exportReportToStream. The first pass simply calculates all the xCut values. On the second pass, exportPage is called once per page rather than once at the end. The pre-calculated xCut values are passed to exportPage, and then on to the JRGridLayout constructor. JRGridLayout need only calculate the yCut values for that page. It can use the pre-calculated xCut values. This allows each page to be exported as it is hit rather than all elements on all pages being collected and exported at one time.

 

This seems to work well for me, however, there may be side-effects that I haven't considered. I can provide the modified source code if this is something that interests you.

 

Ok, good, please i like you provide me the modified source code.

 

Many regards... :)

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

×
×
  • Create New...