PxPlus User Forum

Main Board => Discussions => Programming => Topic started by: steezware on September 30, 2024, 11:46:26 AM

Title: Precision Anamoly when Dividing by 1
Post by: steezware on September 30, 2024, 11:46:26 AM
0100 BEGIN
0110 PRECISION 18
0120 FOR X=3.084820 TO 3.084829 STEP .000001
0130 PRINT X/1
0140 NEXT

What's up with certain divide-by-one values showing extra digits?

And this does not happen if  110 PRECISION 14
  or if I use PRC with 14    130 PRINT PRC(X/1, 14)

Seems like some places in the Pvx doc it mentions internal 18-digit precision and other places 14-digit - what's up with that?
Title: Re: Precision Anamoly when Dividing by 1
Post by: Mike King on September 30, 2024, 12:07:44 PM
Regarding precision, internally numbers are saved as 64 Bit (4 x 16 bit) integers with a precision providing 18 digits.  Older versions only had 48 bits (3 x 16) which only supported 14 digits. The current documentation should refer to 18 digits.

When Pxplus/Providex was originally developed hardware didn't directly support 64 bit math operations thus as hardware matured the language was able to support higher precisions.