JasperReport GC overhead with huge report

0
I am using jasper report to generate a huge Excel report that might extend to 100k+ lines. 
 
I first get my report lines then feed the jasperReport object with them. This is done without a problem. 
 
The fillReport method takes some time but seems to be doing okay as well. 
 
But when exporting my report I get huge memory usage and eventually my GC gets overloaded and my app crashes. For the record, this is the exception I'm getting:
 
java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.Arrays.copyOf(Unknown Source)      at java.io.ByteArrayOutputStream.grow(Unknown Source)     at java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source)     at java.io.ByteArrayOutputStream.write(Unknown Source)     at java.io.ObjectOutputStream$BlockDataOutputStream.write(Unknown Source)     at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)     at java.io.ObjectOutputStream.defaultWriteObject(Unknown Source)     at net.sf.jasperreports.engine.fill.JRTemplatePrintElement.writeObject(JRTemplatePrintElement.java:363)     at sun.reflect.GeneratedMethodAccessor359.invoke(Unknown Source)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)     at java.lang.reflect.Method.invoke(Unknown Source)     at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)     at java.io.ObjectOutputStream.writeSerialData(Unknown Source)     at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)     at java.io.ObjectOutputStream.writeObject0(Unknown Source)     at java.io.ObjectOutputStream.writeObject(Unknown Source)     at java.util.ArrayList.writeObject(Unknown Source)     at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)     at java.lang.reflect.Method.invoke(Unknown Source)     at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)     at java.io.ObjectOutputStream.writeSerialData(Unknown Source)     at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)     at java.io.ObjectOutputStream.writeObject0(Unknown Source)     at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)     at java.io.ObjectOutputStream.writeSerialData(Unknown Source)     at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)     at java.io.ObjectOutputStream.writeObject0(Unknown Source)     at java.io.ObjectOutputStream.writeObject(Unknown Source)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.writeData(JRAbstractLRUVirtualizer.java:704)     at net.sf.jasperreports.engine.fill.JRSwapFileVirtualizer.pageOut(JRSwapFileVirtualizer.java:87)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.virtualizeData(JRAbstractLRUVirtualizer.java:664)
I would like to know whether there's a way to optimize this use of memory and prevent this crash, specially when I need to be able to let my users generate these reports simultaneously. I have tried using swap files and virtualizers but this seems to be only optimizing the fillreport process.
This is the bit of code where I call my report generation function:
 List<LineaInformeFacturacionExcelCVO> listaLineasInformeFacturacionExcelCVO = new ArrayList<LineaInformeFacturacionExcelCVO>();            List<LineaInformeFacturacionExcel> lineasInformeFacturacionExcel = lineaInformeFacturacionExcelDao.getLineasInformeFacturacionExcel(filtros);          getBytesInformeExcel(reportRealPath, parameters, listaLineasInformeFacturacionExcelCVO, true, jrExporterParameters);
This is the function itself:
public byte[] getBytesInformeExcel(String reportRealPath, Map<String, Object> parameters, List<?> list, boolean virtualize, Map<JRExporterParameter, Object> jrExporterParameters) throws EmptyReportException, JRException {
 
    if (list != null && list.isEmpty()) {                throw new EmptyReportException();            }
              // Acordarse de pasarlo entre subrepors            if (parameters == null)                parameters = new HashMap<String, Object>();
            Locale spainLocale = new Locale("es","ES");                 TimeZone europeMadridTimezone = TimeZone.getTimeZone("Europe/Madrid");                              parameters.put(JRParameter.REPORT_LOCALE, spainLocale);            parameters.put(JRParameter.REPORT_TIME_ZONE, europeMadridTimezone);    
            JRSwapFileVirtualizer virtualizer = null;            if (virtualize) {                           virtualizer = new JRSwapFileVirtualizer(250, new JRSwapFile(System.getProperty("java.io.tmpdir"), 250, 250));                parameters.put(JRParameter.REPORT_VIRTUALIZER, virtualizer);                                }
            parameters.put(JRParameter.IS_IGNORE_PAGINATION, false);
            JasperReport jasperReport;
            byte[] reportBytes = null;            JasperPrint jasperPrint = null;
            try {
                jasperReport = (JasperReport) JRLoader.loadObjectFromFile(reportRealPath);         
                Crono crono = new Crono();                crono.startCrono();                LOG.info("Iniciando JasperFillManager.fillReport() ");
                if (list == null) {                    Connection connection = lineaInformeFacturacionDao.obtenerConnection();                              jasperPrint = JasperFillManager.fillReport(jasperReport, parameters, connection);                }                else {                    jasperPrint = JasperFillManager.fillReport(jasperReport, parameters, new JRBeanCollectionDataSource(list));                }
                LOG.info("Fin JasperFillManager.fillReport() :" + crono.stopCrono() + " ms.");
                // Comprueba si el report está vacío.                if (jasperPrint.getPages().isEmpty() || jasperPrint.getPages().get(0).getElements().isEmpty()) {                       throw new EmptyReportException();                }      
                ByteArrayOutputStream xlsReport = new ByteArrayOutputStream();
                JRExporter exporter = new JRXlsExporter();;                String formato = parametrosConfiguracion.getPropertie("informes.excel.formato");                if(StringUtils.isNotBlank(formato)) {                    if(formato.equals(parametrosConfiguracion.getPropertie("excel.formato.xls"))) {                        exporter = new JRXlsExporter();                    } else if(formato.equals(parametrosConfiguracion.getPropertie("excel.formato.xlsx"))) {
                        exporter = new JRXlsxExporter();                    }                }                       
                exporter.setParameter(JRXlsExporterParameter.JASPER_PRINT, jasperPrint);                exporter.setParameter(JRXlsExporterParameter.OUTPUT_STREAM, xlsReport);
                // Carga los parámetros por defecto                exporter.setParameter(JRXlsExporterParameter.IS_ONE_PAGE_PER_SHEET, Boolean.FALSE);                exporter.setParameter(JRXlsExporterParameter.IS_DETECT_CELL_TYPE, Boolean.TRUE);                exporter.setParameter(JRXlsExporterParameter.IS_WHITE_PAGE_BACKGROUND, Boolean.FALSE);                exporter.setParameter(JRXlsExporterParameter.IS_REMOVE_EMPTY_SPACE_BETWEEN_ROWS, Boolean.TRUE);
                // Carga estos parámetros pasados como parámetro sobreescribiendo los por defecto                 if (jrExporterParameters != null) {                    exporter.setParameters(jrExporterParameters);                    exporter.setParameter(JRXlsExporterParameter.JASPER_PRINT, jasperPrint);                    exporter.setParameter(JRXlsExporterParameter.OUTPUT_STREAM, xlsReport);                }           
                crono = new Crono();                crono.startCrono();                LOG.info("Iniciando exporter.exportReport() ");                  exporter.exportReport();                LOG.info("Fin exporter.exportReport() :" + crono.stopCrono() + " ms.");
                reportBytes = xlsReport.toByteArray();    
 
            }finally{                if (virtualize) {                    virtualizer.cleanup();                }            }
            return reportBytes;}
And finally, if this is of any help, here's a screenshot of a memory profiling program at the moment of generating this report:
Crash happens at the second GC use peak.
Exporting directly to File with
exporter.setParameter(JRXlsExporterParameter.OUTPUT_FILE, new File("c://archivo.xlsx"));
instead of a ByteStream produces the following exception:
 java.lang.OutOfMemoryError: Java heap space     at java.util.Arrays.copyOf(Unknown Source)     at java.io.ByteArrayOutputStream.grow(Unknown Source)     at java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source)     at java.io.ByteArrayOutputStream.write(Unknown Source)     at java.io.ObjectOutputStream$BlockDataOutputStream.drain(Unknown Source)     at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(Unknown Source)     at java.io.ObjectOutputStream.writeObject0(Unknown Source)     at java.io.ObjectOutputStream.writeObject(Unknown Source)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.writeData(JRAbstractLRUVirtualizer.java:704)     at net.sf.jasperreports.engine.fill.JRSwapFileVirtualizer.pageOut(JRSwapFileVirtualizer.java:87)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.virtualizeData(JRAbstractLRUVirtualizer.java:664)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.evict(JRAbstractLRUVirtualizer.java:485)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.requestData(JRAbstractLRUVirtualizer.java:630)     at net.sf.jasperreports.engine.base.ElementsBlock.ensureData(VirtualizableElementList.java:463)     at net.sf.jasperreports.engine.base.ElementsBlock.ensureDataAndTouch(VirtualizableElementList.java:432)     at net.sf.jasperreports.engine.base.ElementsBlock.get(VirtualizableElementList.java:283)     at net.sf.jasperreports.engine.base.ElementsBlockList.get(VirtualizableElementList.java:717)     at net.sf.jasperreports.engine.base.VirtualizableElementList.get(VirtualizableElementList.java:96)     at net.sf.jasperreports.engine.base.VirtualizableElementList.get(VirtualizableElementList.java:54)     at net.sf.jasperreports.engine.export.JRGridLayout.createWrappers(JRGridLayout.java:922)     at net.sf.jasperreports.engine.export.JRGridLayout.<init>(JRGridLayout.java:140)     at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportPage(JRXlsAbstractExporter.java:1000)     at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportReportToStream(JRXlsAbstractExporter.java:983)     at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportReport(JRXlsAbstractExporter.java:650)     at com.services.informes.InformesService.getBytesInformeExcel(InformesService.java:920)     at com.services.informes.InformesService.getBytesInformeFacturacionExcel(InformesService.java:277)     at com.controllers.InformesController.exportarInformeFacturacionExcel(InformesController.java:391)     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)     at java.lang.reflect.Method.invoke(Unknown Source)     at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:212)
This is the memory profiling in this case:
gabriel.sanmartindiaz's picture
Joined: Jun 12 2013 - 4:07am
Last seen: 5 years 4 months ago

1 Answer:

1

What JasperReports version are you using?  5.1.2 has some improvements related to memory usage.

Regards,

Lucian

lucianc's picture
7382
Joined: Jul 17 2006 - 1:10am
Last seen: 3 weeks 3 days ago

It does, I was using 5.1.0. Thank god you released this 4 days ago! :)

gabriel.sanmartindiaz - 7 years 10 months ago
Feedback
randomness