When comparing floating point numbers, make sure you round to an acceptable level of rounding for the type of application you are using.
Languages Focus
A comparison operator compares two values either literals as in "Hello" and 3 or variables as in X and Counter. Most languages use the same operators for comparing both numbers and strings. Perl, for example, uses separate sets of comparison operators for numbers and strings.
There are several techniques for handling computer rounding errors. For ASP Classic, consider using the Round function. For example:
If Round(.1+.1+.1, 4) = .3 Then Response.Write "yes" & "<br>" 'Yes is displayed! Else Response.Write "no" & "<br>" End If
Reserve Floating Point Values
Because computers have trouble representing floating point values, you may want to reserve the use of floating point literals for imprecise measurements such length, height, weight, etc. The very nature of measurements is imprecise.
Because computers have trouble with certain floating point values (such as .1), you need to decide what level of accuracy is "good enough" for your application. Perhaps establish a standard for each application. For example, you could establish 6 decimal points as a standard and require all developers to use ObjectPAL's round() method when comparing floating point number. For example, you could rewrite the above routine as follows and get the result you expect.
If round(.1 + .1 + .1, 6) = round(.3, 6) Then msgInfo("", "correct") Else msgInfo("", "not correct") EndIf
The following are practice certification questions with answers highlighted. These questions were prepared by Mike Prestwood and are intended to stress an important aspect of this KB post. All our practice questions are intended to prepare you generally for passing any certification test as well as prepare you for professional work.