PxPlus User Forum

Main Board => Discussions => Language => Topic started by: Marty on September 23, 2018, 01:43:19 PM

Title: Rounding
Post by: Marty on September 23, 2018, 01:43:19 PM

Why are these answers different - Pvx+ V9.00

->floatingpoint
->round off
->set_param 'rn'=0
->set_param 'nr'
->set_param 'rs'=0
->? (2/6374)-(1/6557.5)
-.152497140678612276E-03
->a=(2/6374)-(1/6557.5)
->? a
 .1612775690797811975E-03
->

Title: Re: Rounding
Post by: chrisk on September 23, 2018, 09:36:39 PM

Print followed by parenthesis is interpreted as you passing a channel number. That means the first part of your formula (2/6374) is evaluated to zero. So the remainder of the print ‘-(1/6557.5)’ is output to screen.

You can fix this by putting a 0+ in front of the formula:

->? 0+(2/6374)-(1/6557.5)   
 .1612775690797811975E-03   

Title: Re: Rounding
Post by: Marty on September 23, 2018, 11:16:04 PM

Thanks Chris.