JasperReport GC overhead with huge report

[gabriel.sanmartindiaz]

I am using jasper report to generate a huge Excel report that might extend to 100k+ lines. 

 

I first get my report lines then feed the jasperReport object with them. This is done without a problem. 

 

The fillReport method takes some time but seems to be doing okay as well. 

 

But when exporting my report I get huge memory usage and eventually my GC gets overloaded and my app crashes. For the record, this is the exception I'm getting:

 

java.lang.OutOfMemoryError: GC overhead limit exceeded[/code]

at java.util.Arrays.copyOf(Unknown Source)    [/code]

 at java.io.ByteArrayOutputStream.grow(Unknown Source)     [/code]

at java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source)     at java.io.ByteArrayOutputStream.write(Unknown Source)     at java.io.ObjectOutputStream$BlockDataOutputStream.write(Unknown Source)     at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)     at java.io.ObjectOutputStream.defaultWriteObject(Unknown Source)     at net.sf.jasperreports.engine.fill.JRTemplatePrintElement.writeObject(JRTemplatePrintElement.java:363)     at sun.reflect.GeneratedMethodAccessor359.invoke(Unknown Source)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)     at java.lang.reflect.Method.invoke(Unknown Source)     at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)     at java.io.ObjectOutputStream.writeSerialData(Unknown Source)     at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)     at java.io.ObjectOutputStream.writeObject0(Unknown Source)     at java.io.ObjectOutputStream.writeObject(Unknown Source)     at java.util.ArrayList.writeObject(Unknown Source)     at sun.reflect.GeneratedMethodAccessor154.invoke(Unknown Source)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)     at java.lang.reflect.Method.invoke(Unknown Source)     at java.io.ObjectStreamClass.invokeWriteObject(Unknown Source)     at java.io.ObjectOutputStream.writeSerialData(Unknown Source)     at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)     at java.io.ObjectOutputStream.writeObject0(Unknown Source)     at java.io.ObjectOutputStream.defaultWriteFields(Unknown Source)     at java.io.ObjectOutputStream.writeSerialData(Unknown Source)     at java.io.ObjectOutputStream.writeOrdinaryObject(Unknown Source)     at java.io.ObjectOutputStream.writeObject0(Unknown Source)     at java.io.ObjectOutputStream.writeObject(Unknown Source)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.writeData(JRAbstractLRUVirtualizer.java:704)     at net.sf.jasperreports.engine.fill.JRSwapFileVirtualizer.pageOut(JRSwapFileVirtualizer.java:87)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.virtualizeData(JRAbstractLRUVirtualizer.java:664)[/code]

    

 

 

 

 

I would like to know whether there's a way to optimize this use of memory and prevent this crash, specially when I need to be able to let my users generate these reports simultaneously. I have tried using swap files and virtualizers but this seems to be only optimizing the fillreport process.

 

This is the bit of code where I call my report generation function:

 

 List<LineaInformeFacturacionExcelCVO> listaLineasInformeFacturacionExcelCVO = new ArrayList<LineaInformeFacturacionExcelCVO>();            List<LineaInformeFacturacionExcel> lineasInformeFacturacionExcel = lineaInformeFacturacionExcelDao.getLineasInformeFacturacionExcel(filtros);          getBytesInformeExcel(reportRealPath, parameters, listaLineasInformeFacturacionExcelCVO, true, jrExporterParameters);[/code]

    

 

This is the function itself:

 

public byte[] getBytesInformeExcel(String reportRealPath, Map<String, Object> parameters, List<?> list, boolean virtualize, Map<JRExporterParameter, Object> jrExporterParameters) throws EmptyReportException, JRException {



    if (list != null && list.isEmpty()) {                throw new EmptyReportException();            }

              // Acordarse de pasarlo entre subrepors            if (parameters == null)                parameters = new HashMap<String, Object>();

            Locale spainLocale = new Locale("es","ES");                 TimeZone europeMadridTimezone = TimeZone.getTimeZone("Europe/Madrid");                              parameters.put(JRParameter.REPORT_LOCALE, spainLocale);            parameters.put(JRParameter.REPORT_TIME_ZONE, europeMadridTimezone);    

            JRSwapFileVirtualizer virtualizer = null;            if (virtualize) {                           virtualizer = new JRSwapFileVirtualizer(250, new JRSwapFile(System.getProperty("java.io.tmpdir"), 250, 250));                parameters.put(JRParameter.REPORT_VIRTUALIZER, virtualizer);                                }

            parameters.put(JRParameter.IS_IGNORE_PAGINATION, false);

            JasperReport jasperReport;

            byte[] reportBytes = null;            JasperPrint jasperPrint = null;

            try {

                jasperReport = (JasperReport) JRLoader.loadObjectFromFile(reportRealPath);         

                Crono crono = new Crono();                crono.startCrono();                LOG.info("Iniciando JasperFillManager.fillReport() ");

                if (list == null) {                    Connection connection = lineaInformeFacturacionDao.obtenerConnection();                              jasperPrint = JasperFillManager.fillReport(jasperReport, parameters, connection);                }                else {                    jasperPrint = JasperFillManager.fillReport(jasperReport, parameters, new JRBeanCollectionDataSource(list));                }

                LOG.info("Fin JasperFillManager.fillReport() :" + crono.stopCrono() + " ms.");

                // Comprueba si el report está vacío.                if (jasperPrint.getPages().isEmpty() || jasperPrint.getPages().get(0).getElements().isEmpty()) {                       throw new EmptyReportException();                }      

                ByteArrayOutputStream xlsReport = new ByteArrayOutputStream();

                JRExporter exporter = new JRXlsExporter();;                String formato = parametrosConfiguracion.getPropertie("informes.excel.formato");                if(StringUtils.isNotBlank(formato)) {                    if(formato.equals(parametrosConfiguracion.getPropertie("excel.formato.xls"))) {                        exporter = new JRXlsExporter();                    } else if(formato.equals(parametrosConfiguracion.getPropertie("excel.formato.xlsx"))) {

                        exporter = new JRXlsxExporter();                    }                }                       

                exporter.setParameter(JRXlsExporterParameter.JASPER_PRINT, jasperPrint);                exporter.setParameter(JRXlsExporterParameter.OUTPUT_STREAM, xlsReport);

                // Carga los parámetros por defecto                exporter.setParameter(JRXlsExporterParameter.IS_ONE_PAGE_PER_SHEET, Boolean.FALSE);                exporter.setParameter(JRXlsExporterParameter.IS_DETECT_CELL_TYPE, Boolean.TRUE);                exporter.setParameter(JRXlsExporterParameter.IS_WHITE_PAGE_BACKGROUND, Boolean.FALSE);                exporter.setParameter(JRXlsExporterParameter.IS_REMOVE_EMPTY_SPACE_BETWEEN_ROWS, Boolean.TRUE);

                // Carga estos parámetros pasados como parámetro sobreescribiendo los por defecto                 if (jrExporterParameters != null) {                    exporter.setParameters(jrExporterParameters);                    exporter.setParameter(JRXlsExporterParameter.JASPER_PRINT, jasperPrint);                    exporter.setParameter(JRXlsExporterParameter.OUTPUT_STREAM, xlsReport);                }           

                crono = new Crono();                crono.startCrono();                LOG.info("Iniciando exporter.exportReport() ");                  exporter.exportReport();                LOG.info("Fin exporter.exportReport() :" + crono.stopCrono() + " ms.");

                reportBytes = xlsReport.toByteArray();    



            }finally{                if (virtualize) {                    virtualizer.cleanup();                }            }

            return reportBytes;}[/code]

 

 

And finally, if this is of any help, here's a screenshot of a memory profiling program at the moment of generating this report:

 

http://i.stack.imgur.com/YfqNI.png

 

Crash happens at the second GC use peak.

 

Exporting directly to File with 

 

    exporter.setParameter(JRXlsExporterParameter.OUTPUT_FILE, new File("c://archivo.xlsx"));

 

instead of a ByteStream produces the following exception:

 

 java.lang.OutOfMemoryError: Java heap space     at java.util.Arrays.copyOf(Unknown Source)     at java.io.ByteArrayOutputStream.grow(Unknown Source)     at java.io.ByteArrayOutputStream.ensureCapacity(Unknown Source)     at java.io.ByteArrayOutputStream.write(Unknown Source)     at java.io.ObjectOutputStream$BlockDataOutputStream.drain(Unknown Source)     at java.io.ObjectOutputStream$BlockDataOutputStream.setBlockDataMode(Unknown Source)     at java.io.ObjectOutputStream.writeObject0(Unknown Source)     at java.io.ObjectOutputStream.writeObject(Unknown Source)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.writeData(JRAbstractLRUVirtualizer.java:704)     at net.sf.jasperreports.engine.fill.JRSwapFileVirtualizer.pageOut(JRSwapFileVirtualizer.java:87)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.virtualizeData(JRAbstractLRUVirtualizer.java:664)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.evict(JRAbstractLRUVirtualizer.java:485)     at net.sf.jasperreports.engine.fill.JRAbstractLRUVirtualizer.requestData(JRAbstractLRUVirtualizer.java:630)     at net.sf.jasperreports.engine.base.ElementsBlock.ensureData(VirtualizableElementList.java:463)     at net.sf.jasperreports.engine.base.ElementsBlock.ensureDataAndTouch(VirtualizableElementList.java:432)     at net.sf.jasperreports.engine.base.ElementsBlock.get(VirtualizableElementList.java:283)     at net.sf.jasperreports.engine.base.ElementsBlockList.get(VirtualizableElementList.java:717)     at net.sf.jasperreports.engine.base.VirtualizableElementList.get(VirtualizableElementList.java:96)     at net.sf.jasperreports.engine.base.VirtualizableElementList.get(VirtualizableElementList.java:54)     at net.sf.jasperreports.engine.export.JRGridLayout.createWrappers(JRGridLayout.java:922)     at net.sf.jasperreports.engine.export.JRGridLayout.<init>(JRGridLayout.java:140)     at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportPage(JRXlsAbstractExporter.java:1000)     at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportReportToStream(JRXlsAbstractExporter.java:983)     at net.sf.jasperreports.engine.export.JRXlsAbstractExporter.exportReport(JRXlsAbstractExporter.java:650)     at com.services.informes.InformesService.getBytesInformeExcel(InformesService.java:920)     at com.services.informes.InformesService.getBytesInformeFacturacionExcel(InformesService.java:277)     at com.controllers.InformesController.exportarInformeFacturacionExcel(InformesController.java:391)     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)     at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)     at java.lang.reflect.Method.invoke(Unknown Source)     at org.springframework.web.method.support.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:212)[/code]

   

 

This is the memory profiling in this case:

http://i.stack.imgur.com/UGCs3.png

 

 

 

[lucianc]

What JasperReports version are you using?  5.1.2 has some improvements related to memory usage.

Regards,

Lucian

[gabriel.sanmartindiaz]

It does, I was using 5.1.0. Thank god you released this 4 days ago! :)