Precision Anamoly when Dividing by 1

Started by steezware, September 30, 2024, 11:46:26 AM

Previous topic - Next topic

steezware

0100 BEGIN
0110 PRECISION 18
0120 FOR X=3.084820 TO 3.084829 STEP .000001
0130 PRINT X/1
0140 NEXT

What's up with certain divide-by-one values showing extra digits?

And this does not happen if  110 PRECISION 14
  or if I use PRC with 14    130 PRINT PRC(X/1, 14)

Seems like some places in the Pvx doc it mentions internal 18-digit precision and other places 14-digit - what's up with that?

Mike King

Regarding precision, internally numbers are saved as 64 Bit (4 x 16 bit) integers with a precision providing 18 digits.  Older versions only had 48 bits (3 x 16) which only supported 14 digits. The current documentation should refer to 18 digits.

When Pxplus/Providex was originally developed hardware didn't directly support 64 bit math operations thus as hardware matured the language was able to support higher precisions.
Mike King
President - BBSysco Consulting
eMail: mike.king@bbsysco.com