DaveMcClellan

I've worked around the problem with the large (40,000+ rows) CSV export. Looking at the code for the JRCsvExporter, I can see it loads a full page at a time to export, rather than one row. iReport creates templates with the column names in the "columnheader" band and fields in the "detail" band. So originally I had resized all other bands to 0 after removing their elements, removed the margins and set "Ingore Pagination" in the report properties so that the "columnheader" would be output once - resulting in one very large page. The CSV exporter code doesn't appear to have changed significantly between 1.3.3 and 3.0.0, so the 1.42 JVM and 1.3.3 JR must have used less heap than the 1.5 JVM and 3.0.0 JR. I've changed the template by re-enabling pagination, moving the CSV "column headers" out of the "columnheader" band and into the "title" band, and resizing the "columnheader" band to 0. This results in no visible page breaks, a single copy of the column headers in the first row of output, and all of the detail row rows being exported as they were before. Are there plans to update JRCsvExporter to be less memory intensive for large-page reports (e.g., load the JRPrint a row at a time rather than a page?)

After a week and a half of running under the new environment (on Linux Java 5 / Tomcat 5.5), it appears that some report templates cause the same problem I reported earlier with the Windows JVM - they do not release the JRSwapFileVirtualizer. The old server had been running for the better part of a year with a limit of 10 contexts. The same set of reports running on the new server, with the maxSize moved up to 25, got the following error today, after running for only about 10 days: [2008-07-21 12:30:00,188] ERROR - JasperReportRunnerBS.fillAndDeliverRptTemplate(1045) | Exception running report QNTXtrSiteFax: The virtualizer is used by more contexts than its in-memory cache size 25 Once the server gets this exception, no report afterwards will run. My guess is that some reports with some type of element (subreport, grouping, etc.) or some type of export (PDF, CSV, Text, TIFF via JRGraphics2DExporter) result in a context not being released - even after garbage collection. This was not a problem in JR 1.3.3 / Java 1.4.2. It appears I'm going to have to use a JRFileVirtualizer for all of my virtualization. Note: I have to run the app I mentioned in my last post, with the large CSV export, in a special Java 1.4.2 / JasperReports 1.3.3 container. It will not run in JR 2.05 or 3.0.0, and JR 1.3.3 cannot be run under Java 5 without getting the Perm Gen memory overflow.

Lucian - This week we upgraded from JR 1.33 to JR 3.0 (as part of a larger upgrade from Java 1.42 / Tomcat 5.0 to Java 1.5 / Tomcat 5.5). Most of our reports are running ok under the new setup, but the large (several thousand report) CSV-export report is failing with an OutOfMemory error at export time. I can run this same report to completion on the old server, using a 256Mb heap, but get an OutOfMemory error in the JRCsvExporter / JRGridLayout class in a 512Mb heap on the new server. I'm still using the JRSwapFileVirtualizer, so the report fills ok - it just doesn't export. Traceback is: Code: [2008-07-12 09:18:45,874] ERROR - JasperReportRunnerBS.runReport(705) | runReport(req,sched) : Trapped Exception class java.lang.OutOfMemoryError : Java heap space java.lang.OutOfMemoryError: Java heap space at java.lang.String.<init>(String.java:208) at java.lang.StringBuffer.toString(StringBuffer.java:586) at net.sf.jasperreports.engine.export.JRGridLayout.createWrappers(JRGridLayout.java:924) at net.sf.jasperreports.engine.export.JRGridLayout.<init>(JRGridLayout.java:144) at net.sf.jasperreports.engine.export.JRCsvExporter.exportPage(JRCsvExporter.java:274) at net.sf.jasperreports.engine.export.JRCsvExporter.exportReportToWriter(JRCsvExporter.java:260) at net.sf.jasperreports.engine.export.JRCsvExporter.exportReport(JRCsvExporter.java:207) at com.ateb.obc.service.impl.JasperReportRunnerBS.fillAndDeliverRptTemplate(JasperReportRunnerBS.java:1489) Did memory management change between 1.33, 2.05 and 3.0? You mentioned that some of the exporters try to load a whole page at a time to export, which would be bad for this line-oriented data set if the CSV exporter does so. Is there anything I can do other than going back to 1.33? Thanks - Dave McClellan

I use the JRConcurrentSwapFile + JRSwapFileVirtualizer on Linux-hosted copies of the webapp, but on Win 2003 hosts I have to use a per-report JRFileVirtualizer. I never got an answer on why the swap virtualizer is broken on Windows. The JRFileVirtualizer works ok since the application runs one report at a time (sometimes only seconds apart); I don't know that it would work as well in your batch scenario.

The maxSize change you gave me's been working perfectly in our webapp running under Tomcat 5.5 on Linux. I set the max size and throttle the total simultaneous reports and we've run hundreds through a size-10 cache. We're now seeing something odd with the cache when the web-app's running on a Windows 2003 server. Java 1.5 (012), same Tomcat 5.5.23, JasperReports 1.3.3, same code of ours I run on Linux. When running under 2003, the contexts don't appear to be releasing the virtualizer / cache after we run the report. I do this sequence: Code: fill template Params Map, including a put(JRParameter.REPORT_VIRTUALIZER, virtualizer) JasperPrint prtRpt = JasperFillManager.fillReport( reportTemplate, templateParams, connection ); for( de : delivery-list ) { create exporter of appropriate type (CSV, HTML, PDF, TXT , whatever) exporter.setParameter( JRExporterParameter.JASPER_PRINT, prtRpt ); ... set exporter params exporter.exportReport(); } return status; And that works fine on Linux; all the reports share the same cache. On Windows, the first maxSize reports run normally. The maxSize+1 report and all succeeding ones return this error (maxSize == 10 in my app): The virtualizer is used by more contexts than its in-memory cache size 10 Apparently the context isn't being released when my local variable prtRpt goes out of scope, garbage collected, finalize(), or whatever signals the cache that the context isn't using it. On Windows only. Has anyone else heard of this issue? And is there a step I can add to explicitly release the cache from the context when I finish all my exports? Right now I'm working around it by created a JRFileVirtualizer() per report instead of a shared JRSwapFileVirtualizer(). But I'd like to understand why the context isn't being released (or releasing the cache), and I prefer using the shared swap-cache as it's less expensive long term and creates less temp files to clean up. Thanks! - Dave

Our application runs multiple Jasper Reports using a Quartz scheduler (under Spring) to start each report - so that we can schedule them. Some of these reports are complex and use one or more sub-reports. I recently had to add a JRSwapFileVirtualizer as one of the reports was running over a dataset that grew large in a hurry and we ran out of heap. I'm using a shared one, so I only have to create it once on app startup. The javadocs for the constructor say: maxSize - the maximum size (in JRVirtualizable objects) of the paged in cache Which I thought would be the number of cached pages not flushed to swap. I set it to 4, twice what the examples showed. It apparently also relates to number of simultaneous contexts which can use the SwapFile, as occasionally a report would fail on fill with the error: The virtualizer is used by more contexts than its in-memory cache size 4 My question is - what is considered a context? A single call to JasperFillManager.fillReport(), or each report and sub-report it calls? That will help me set the maxSize, and control the number of simultaneous reports which Quartz will run.