PxPlus User Forum

Twitter Twitter Twitter

Author Topic: Rounding  (Read 1719 times)

Marty

  • Member
  • **
  • Posts: 5
    • View Profile
Rounding
« on: September 23, 2018, 01:43:19 PM »
Why are these answers different - Pvx+ V9.00

->floatingpoint
->round off
->set_param 'rn'=0
->set_param 'nr'
->set_param 'rs'=0
->? (2/6374)-(1/6557.5)
-.152497140678612276E-03
->a=(2/6374)-(1/6557.5)
->? a
 .1612775690797811975E-03
->

chrisk

  • Member
  • **
  • Posts: 14
    • View Profile
Re: Rounding
« Reply #1 on: September 23, 2018, 09:36:39 PM »

Print followed by parenthesis is interpreted as you passing a channel number. That means the first part of your formula (2/6374) is evaluated to zero. So the remainder of the print ‘-(1/6557.5)’ is output to screen.

You can fix this by putting a 0+ in front of the formula:

->? 0+(2/6374)-(1/6557.5)   
 .1612775690797811975E-03   

Chris Kukuchka
Sequoia Group, Inc.

Marty

  • Member
  • **
  • Posts: 5
    • View Profile
Re: Rounding
« Reply #2 on: September 23, 2018, 11:16:04 PM »
Thanks Chris.