Scripting Helpers is winding down operations and is now read-only. More info→
Ad
Log in to vote
0

How to round a number help?

Asked by 10 years ago

How do I round a number to nearest tenths. I made a script and whenever it runs it doesn't up the value by 0.1 it does it by 1.0000002123 or something like that. Is there a way to fix this?

2 answers

Log in to vote
2
Answered by
BlueTaslem 18071 Moderation Voter Administrator Community Moderator Super Administrator
10 years ago

Consider this little snippet.

Instance.new("NumberValue", workspace).Value = 0.1

If you go and look at the NumberValue in the Properties tab, you'll see something like 0.10000000000000001.

That's odd. Let's look at this:

print(0.10000000000000001 == 0.1)
-- true

When you see that it's that close, it actually is 0.1. The floating-pointer number system (the way computers store decimal numbers) isn't capable of representing 0.1 exactly, so it approximates it (to the nearest 10^-16ish)

For math (arithmetic / computation) purposes, you won't get anything better.

If you're using == with decimals, that's usually a bad idea, because of issues like this. Instead, a small range (e.g. 1.09 <= x and x <= 1.11) is usually better.

And if you're displaying a number, there's a few options. The simplest would just be to use :sub on a tostring representation of the number, e.g., tostring(num):sub(1, 4), though, this will eliminate large numbers, too.

A fuller solution would be something like:

function stringify(x, maxdec, mindec)
    x = tostring(x)
    local decimal = x:find("%.") -- Where the . is
    if not decimal then
        x = x .. "."
        decimal = #x
    end
    local digitsAfter = #x - decimal
    x = x .. ("0"):rep(math.max(0, (mindec or 0) - digitsAfter))
    x = x:sub(1, decimal + maxdec)
    if maxdec == 0 then
        x = x:sub(1, -2)
    end
    return x
end
-- (Not tested)
-- Note that this TRUNCATES as opposed to ROUNDS
0
Can I make it a 'script.Parent.Value.Changed:connect(function(x, madec, mindec)'? raystriker6707 30 — 10y
Ad
Log in to vote
-3
Answered by
Qorm 100
10 years ago

well instead of putting 0.1, put 1.0000000000 - 0.0000002123 (0.9999997877) to get exactly 1?

Answer this question