Can't quite get fractional part of double precision float

[rwosdev]

I wrote an assembly function to get the fractional part of a double (e.g. the '72345' of 20.72345).
It works fine but not with values as precise as 20.723456 where it just ends up in a loop.
It's stdcall. It can be called like so and supplied to printf

Code: Select all

long long i = ffractionalpart(20.72345)

Any suggestions?

Code: Select all

FLOAT_TEN dd 10.0

DLLEXPORT ffractionalpart
.value EQU 0 +8
	push ebp
	mov ebp, esp
	
	sub esp, 16
.outVal EQU 8
.remainder EQU 16
		
	fld QWORD [ebp+.value]
	fisttp DWORD [ebp-.outVal] ; outVal = integer part
	
	fld QWORD [ebp+.value]
	fisub DWORD [ebp-.outVal] ; outVal -= integer part
	fstp DWORD [ebp-.outVal] ; outVal = 0.xxxxx
	; Multiply by 10 & mod by 1 until remainder == 0
	
.loop:
	fld1
	fld DWORD [ebp-.outVal] ; st0 = outVal, st1 = 1.0
	fprem ; st0 = remainder == st0 % st1
	
	fldz ; st1 = remainder, st0 = 0.0
	fcomp
	fstsw ax
	fstp
	and ah, 01000111b
	cmp ah, 01000000b
	je .ok
	
.next:
	fld DWORD [ebp-.outVal]
	fmul DWORD [FLOAT_TEN]
	fstp DWORD [ebp-.outVal]
	jmp .loop
	
.ok:
	fld DWORD [ebp-.outVal]
	fabs
	fisttp QWORD [ebp-.outVal]
	
	mov edx, DWORD [ebp-.outVal+4]
	mov eax, DWORD [ebp-.outVal]
	
.done:
	add esp, 16
	
	pop ebp
	add esp, 8+4
	jmp DWORD [esp-(8+4)]

[iansjack]

I suspect that your error is when you compare the remainder to 0. Since floating point numbers can't be represented exactly, this is very likely to always fail. You should really be looking at whether the remainder is less than a suitably small number (i.e. whether it is "near enough" 0).

If you were to run the routine under a debugger this might become clearer.

[rwosdev]

Yeah that seems likely as when I change FLOAT_TEN to FLOAT_HUNDRED (100.00) it works but gives me some trailing 0s in cases like 20.7 as expected, gonna keep trying and I'll post the results if I get it working

[Brendan]

Hi,

I wrote an assembly function to get the fractional part of a double (e.g. the '72345' of 20.72345).
It works fine but not with values as precise as 20.723456 where it just ends up in a loop.

It also won't work for values too large to fit in a 64-bit signed integer.

Any suggestions?

I'd make sure it's not a special value (infinity, NaN), then use "FABS" to make sure it's positive, then use "FXTRACT" to split the number into a significand and an exponent and store these separately as integers to get an unsigned 64-bit integer significand and signed 32-bit integer exponent.

Then I'd shift the significant left by "exponent + 32" (with a bunch of tests to make sure the shift count is in range, and using "right shift by negated value" if its negative; and using SHLD or SHRD if it's 32-bit code where you need to use a pair of registers for a 64-bit integer) to convert the value into fixed point format where the lowest 32 bits are the most significant 32 fractional bits (and anything else is discarded).

After that, it becomes a relatively simple loop (e.g. containing something like "mov ebx,10; mul ebx; add edx,'0'") to get each digit in ASCII.


Cheers,

Brendan

[rwosdev]

Probably try this over the weekend, thanks Brendan I'll write back here if I get it working