Facing the same problem. Looking into using in-memory database and RAM drives (not SSD) if the data fits in memory.
There's also the problem with cache not being updated when underlying data changes but this is another issue. :-(
EDIT: Tried using columnar database without too much difference in performance. Performance problem is with Table - Group and with Crosstab - Columns. Filtering is also slow. The problem seems to be with the JasperReports Server software architecture because I can process over 2,400,000 statement details to generate over 120,000 pdf files with just souped up JasperReports library in under 10 minutes.
Database is returning result set in under 10 seconds but the JasperReports Server is just too slow processing the result and displaying.