Round off error?

[2001 JI Open Discussion]

By: Kevin Gorski - gorskika

Round off error?

2002-08-16 09:50

Hi again, I've made a report that uses a simple variable with the "Sum" compute type for a column of Floats. The data that is passed in for this field is always rounded to 2 places (using BigDecimal in another part of my program). When my CustomDataSource class has it's getFieldValue method called I print to the console what the value for that field is, so I can confirm that all of the values are correct in rounding. My problem is that sometimes the sum of the column comes out "7.29000003" when it should be "7.29". Is this a bug or is there something I should know?

 

 

By: Teodor Danciu - teodord

RE: Round off error?

2002-08-19 05:29

 

Hi,

 

This is how "float" operations work.

You might find some answers here:

 

http://java.sun.com/products/jdk/1.2/docs/api/java/math/BigDecimal.html#BigDecimal(double)

 

If formatting the result using the "pattern" attribute

of the text field does not suit your needs, then you

should consider declaring your sum variable as

a java.math.BigDecimal in order to perform precise

calculations.

 

Thank you,

Teodor

 

 

 

By: John Pollard - johnnypoll

RE: Round off error?

2002-08-19 05:46

Teodor, I think you slightly misunderstood Kevin's question. Say for example you have the following values to sum up:

 

1.004

1.004

1.004

 

If these are rounded to 2 dp then the they would appear as:

 

1.00

1.00

1.00

 

Then you might hope to see a sum at the bottom of 3.00, but actually you would see

 

3.01 (1.004 + 1.004 + 1.004 = 3.012)

 

So the problem really is that the summing is too accurate rather than not accurate enough.

 

The solution might be to do the rounding of the data supplied to each row before Jasper gets hold of it, so in the above Jasper is passed 1.00, 1.00, 1.00 in which case the sum will be as expected.

 

 

By: Teodor Danciu - teodord

RE: Round off error?

2002-08-19 06:08

 

Hi,

 

But Kevin said that he already rounds the field

values in his custom data source...

So he does not have those 1.004 values...

He even said that he displays those values

for debug purposes and everything is ok.

Am I wrong?

 

I suspect that passing from BigDecimal to float

introduces unwanted decimals and I think I saw

this explained somewhere...

 

Thank you,

Teodor

 

 

 

By: Bernd Proissl - berndproissl

RE: Round off error?

2002-08-24 12:29

> I suspect that passing from BigDecimal to float

> introduces unwanted decimals and I think I saw

> this explained somewhere...

 

maybe http://developer.java.sun.com/developer/bugParade/bugs/4018937.html

 

 

 

 

By: Kevin Gorski - gorskika

RE: Round off error?

2002-08-19 07:52

Ok, here's a more specific example. My reports look something like this:

info1 info2 info3 time ...

- - - 1.00

- - - 0.01

- - - 0.01

- - - 2.01

total: 3.03

 

And total is a calculation="Sum" of the time column. What I'm saying is that with some values (which are still in X.XX format) I get some adding error ( meaning that total would come out to 3.03000004 instead of 3.03). Perhaps if I use the pattern attribute Teodor mentioned on the total it will stop showing this at seemingly random intervals. Thanks for your help... if this still occurs I'll post a test case.

 

 

By: Kevin Gorski - gorskika

Solution?

2002-08-20 08:11

It seems that adding a pattern of "#.##" to each of my sums at least hides any lingering oddities. I suspect that Teodor is right about the BigDecimal to Float conversion... there doesn't seem to be any other source for error. Thanks again for your help.