0 Replies - 457 Views - Last Post: 08 December 2018 - 01:10 PM

#1 B-ing   User is offline

  • New D.I.C Head

Reputation: 0
  • View blog
  • Posts: 3
  • Joined: 07-December 18

IEEE-754 and machine numbers

Posted 08 December 2018 - 01:10 PM

I've been trying to wrap my head around machine numbers like the unit roundoff (u) and epsilon (e) in combination with the IEEE 754 standard. My textbook states some things that don't really make sense to me.

Unit roundoff according to my textbook is:

  • for single precision (mantissa is 23 bit): u = 6e-8
  • for double precision (mantissa is 52 bit): u = 2e-16


I've been trying to derive a formula for these results with two relations:

  • my textbook states: "In binary arithmetic with rounding we usually have e = 2*u"
  • e = 2^-n, n being the amount of mantissa bits


These combined results would then give: u = 2^-(n+1), again with n being the amount of mantissa bits. Checking this formule with the given results of u for different precisions:

for single: u = 2^-(23+1) = 5.96e-8, this result checks out.
for double: u = 2^-(52+1) = 1.11e-16, this result doesn't check out.

Could someone please help me derive a correct formule for the unit roundoff, or point me to some mistakes I have been making? All help is appreciated.

Is This A Good Question/Topic? 0
  • +

Page 1 of 1