Retriving large recoredset (about 1 million )

[2005 IR Help]

By: Javed Momin - javedmomin

Retriving large recoredset (about 1 million )

2004-06-27 23:15

Hi

I am trying to fetch 1 million record from database with 5 fields on client machine (i.e windows) it takes 1 hr 20 min. I used heap memory space (i.e -Xmx1024m)

If I am running the same report on sun solaris machine it takes 7 hr . I used heap memory space (i.e -Xmx1024m)

 

so why this report is so slow on sun solaris machine ? can some one give me help on this topic ?

 

Javed

 

 

 

 

By: karthika - karthika

RE: Retriving large recoredset (about 1 million )

2004-06-28 01:59

Check ur report alighnment

 

 

 

 

By: Javed Momin - javedmomin

RE: Retriving large recoredset (about 1 million )

2004-06-28 06:26

Then how its working with windows (i.e 1 hr 20 min) but on sun solaris it take 6 times slow.

On solaris i have to chage aligment is it ?

 

 

 

 

By: Chuck Deal - cdeal

RE: Retriving large recoredset (about 1 million )

2004-06-28 06:48

I would be leaning more towards a JVM problem.

 

Have you tried JUST returning the resultset, outside of Jasper, in each environment? What I would be looking for is time differences, but more importantly, is the time difference RATIO the same? (is the Solaris box still six times slower)

 

This might help narrow down the problem. If the ratio is maintained, then the problem would seem to be environmental. Otherwise, maybe the Jasper engine is doing something differently on the Solaris box. Are the JVMs the same version, same vendor?

 

 

 

 

By: Javed Momin - javedmomin

RE: Retriving large recoredset (about 1 million )

2004-06-28 07:10

Yes I am using on both side (i.e sun solaris and windows ) jdk1.4.1. I dont know sun solaris very much.

 

I tried various resultset on windows as well as sun solaris .

 

eg. for 40K - Windows - 20 fields report - 8 min 46 sec.

for 40K - sun solaris - 20 fields report - 67 min 21 sec.

 

for 111K - Windows - 20 fields report - 25 min 45 sec.

 

for 111K - sun solaris - 20 fields report - 101 min 54 sec.

 

for 988K - windows - 5 fields report - 80 min 35 sec.

for 988K - sun solaris - 5 fields report - 426 min 41 sec.

 

So how i can increase the performance on sun solaris machine ?

 

 

 

 

 

 

 

 

By: Chuck Deal - cdeal

RE: Retriving large recoredset (about 1 million )

2004-06-28 09:25

I am not very good with Solaris either.

 

I want to keep drilling down until we find the source of your problem. That may help you come up with a solution.

 

Now, in your last post, you reported statistics for different size datasets. You mentioned "20 fields report". Did you get those statistics by running Jasper? Or did you write a stand-alone driver program to do that? I would prefere a stand-alone program so that we can eliminate Jasper as a cause. Next, what does the time that you reported represent? Is that the time the driver spent running? Is that the time it took to make the JDBC call?

 

We want to split out the times a bit if we can. Maybe the problem lies in the JDBC driver? Are you connecting to the same database, just using two different clients?

 

 

 

 

 

By: Javed Momin - javedmomin

RE: Retriving large recoredset (about 1 million )

2004-06-28 22:38

I given the time for pdf generated. I am using the same datasource for both client as well as server.

 

I am using store procedure and getting resultset in getObject() method and sending this resultset directly to jasperRunManager

 

JasperRunManager.runReportToPdfFile(fileName, parameters, new JRResultSetDataSource(myResultSet));

 

So i am getting resultset within 3 min using store procedure on both windows as well as sun solaris for 1 million records but for filling this report it takes so much time on sun solaris ?

 

 

 

 

 

 

By: Chuck Deal - cdeal

RE: Retriving large recoredset (about 1 million )

2004-06-29 07:51

At least the problem has been narrowed down to Jasper Rpeorts and the filling process. However, I do not have much more info to contribute. I would check to back sure that the two JVMs are the same version and REVISION (ie. j2sdk1.4.2_03).

 

What version of JasperReports? I know a while back, someone found a problem with the way Jasper was constructing one of the objects used in rendering. It had something to do with rendering text as glyphs instead of text (or something like that).

 

 

 

 

By: Sean Barenz - forceinfinity

RE: Retriving large recoredset (about 1 million )

2004-06-30 08:35

Actually I have a similar kind of problem with Jasper, only its the Linux machine that's taking alot longer the the Windows machine (which is a laptop). In all the tests I've ran, about the only thing I've not eliminated as the source of the problem is the JVM performance on the Linux machine and how Jasper meshes with it. However it seems to me that it's the AWT component that might be slowing it down.

 

 

 

 

By: Javed Momin - javedmomin

RE: Retriving large recoredset (about 1 million )

2004-06-30 23:07

I am still not found solution for on sun solaris machine.

I tried report containing 20 fields and fetching 1 million recordset. I am not able to get any output (pdf not generated). Its machine hangs up or JasperReport does not support for huge recordset on sun solaris.

 

 

 

 

 

By: Javed Momin - javedmomin

RE: Retriving large recoredset (about 1 million )

2004-06-30 02:19

On Sun solaris I am using jdk141_05 and on windows j2sdk1.4.2_03. So it might be a problem ?

I am using jasperreports-0.5.2 version. on both sun as well as windows. If i will upgrade new release version purformance will chage ?

How should I can improve performance ?

 

 

 

 

By: Chuck Deal - cdeal

RE: Retriving large recoredset (about 1 million )

2004-07-01 07:08

I am going to say that the JVM difference could be a big problem.

 

As for the Jasper version, the fix I was referring to (I think) was included in at least 0.5.2. So, you should be ok there.

 

See if you can use a newer JDK on the Sun box.

 

 

 

 

By: Javed Momin - javedmomin

RE: Retriving large recoredset (about 1 million )

2004-07-01 07:24

Yes Now i loaded j2sdk1.4.2.05 version on both client as well as server. I am getting result also very closer. Now i am running 1 million recordset hope i should get result within 3 1/2 hrs. If it gets then it might be solve the problem.

 

thanks for your help.

 

 

 

 

By: Javed Momin - javedmomin

RE: Retriving large recoredset (about 1 million )

2004-07-01 22:27

I have no result for 1 million recordset on sun solaris its machine hang up again?

 

 

 

 

 

By: Javed Momin - javedmomin

RE: Retriving large recoredset (about 1 million )

2004-07-06 23:06

Can i used text file where all 1 million record is store and after that i can fetch that record from jasper engine ? or can you suggest any other way ?