Scripting HelpersARCHIVE
A little piece of Roblox history. Preserved for everyone, one question at a time.Read about the archive →
COMMUNITY QUESTION

how come the decimals in roblox are not perfect for number Values?

when You insert a "number Value" into your roblox game and put the value as .7, it turns out as like: 69999999999999995559 for some reason...

though when I print "what is .8 - .1?" , it simply prints "0.7"

why does this happen?

is there a way to get a perfect decimal number from a number value without getting thousands of digests !?

7 comments

0

You can round the value either up, or down using math.floor or math.ceil. ^

0

Or you can use both, using if then statements checking if the value is >= 0.5, then round up, if it's < 0.5, round down.

0

no I mean I want the number value to actually say .7

0

https://scriptinghelpers.org/questions/31645/rounding-to-2-decimal-places, this will cut off anything off in the thousandths place and beyond.

0

Number Values are double-precision floating point number, or double. https://developer.roblox.com/en-us/api-reference/class/NumberValue

2 answers

Roblox uses doubles, which are 64-bit floating point values. Floating point values compromise precision for the space it takes up, meaning that when you have a fractional value, you (usually) won't get the exact value you wanted. This will, however, not affect any integer with an absolute value less than 2^53. However, Lua manages to hide these floating point errors, but I couldn't find any documentation of the process. If you want to find more information on this, look at this Wikipedia article. Hope this helps!

What you're seeing is roundoff error.

.69, with the 9 repeating like that, is really close to .7, but it is not .7.

There is a simple way to get your desired 0.7, and that is to use the Lua rounding formula:

roundedNumber = math.floor(numberToRound + 0.5)

If you use this on .69, you may have to actually multiply the number by 10 then floor it.

Rounding .69 to 0.7:

local num = 0.69
function RoundDecimal(givenNum)
	if tostring(givenNum):find(".") and (tostring(givenNum):find(".") - 1) == "0" then
		local newNum = givenNum * 10
		local flooredNum = math.floor(newNum + 0.5)
		local finalNum = flooredNum / 10
		return finalNum
	end
end
print(RoundDecimal(num))

The output:

0.7

But what's going on with the NumberValue?

Well, the number you insert into a NumberValue is not taken as a literal double or float. This means that the compiler can throw a binary-stored number at you instead of the decimal inside of the NumberValue.

A similar dilemma occurs with the tostring() function. When tostring() converts a number to a string, it suspends the floating point of the number, making the number more vulnerable to being binary-stored instead of being literal (however this usually only occurs with decimals because they have floating points). If you run a loop like this...

for i = 100, 0, -0.1 do
	wait(0.1)
	print(tostring(i))
end

.. nothing appears to be off. Around 91.4, however, you'll start to see this:

91.4
91.3
91.20000000001
91.10000000001
91.00000000001
-- and so on

This is the type of thing that baffles programmers but is simple to work around.

4 comments

0

Rounding the number would only work if you know what the result should be to see what place you should round to. Otherwise, you're likely to lose more precision rather than gaining precision by rounding.

0

Also, I'm not sure what you mean by saying tostring makes a number more vulnerable to being binary-storedwhich makes it loose precision. With some testing, it appears that the for loop makes the floating point errors by running "for i = 1000, 0, -1 do print(tostring(i/10)) end"

0

That's just tostring() doing it because it's suspending the floating point. If you do it without tostring(), the numbers are literal and are printed as is.

0

print calls tostring on its parameters, so I don't see how that makes a difference. I tried running the code you made without tostring (and without the wait) on Lua 5.1 and 5.3 and both still had floating point errors.

This discussion is preserved as it was. New answers, comments, and votes are closed.