I'm trying to compress some numbers in a table before I convert it to JSON. This is easy enough:

function compressValue(value, level)
	if level == nil then
		level = compressQuality
	end
	return math.floor(value * 10^level + 0.5) * 10^(-level)
end

Though, after encoding, floating point errors occur:

print(HTTPService:JSONEncode({["value"]=31.73}))

Would print:

{"value":31.7300000000000004263256414561}

This kind of completely defeats the purpose of compressing it in the first place.

I could turn the number into a string but I'm not sure how I would turn it back into a number after encoding it.

#Any ideas?

AnonymousEdited

4 comments

0

You can't prevent the float errors so converting to string is good idea, you can turn it into a number with tonumber in Lua, or it's more complicated and i am not getting something?

Anonymous#
0

@imKirda I'm converting it to JSON, so unless I have a JSON parser I cant modify the values after encoding it. To clarify, the floating point errors happen in the process of JSONEncode.

Anonymous#
0

oh yeah i see the edit, then turn it into the string as you thought, floating point errors are not caused by Lua but by the computer itself so there probably is not a better option, you can also multiply the number by 100 and then divide by 100 when getting the values back.

Anonymous#
0

I'm gonna go with multiplying before I encode and dividing after I get the value back. I'll leave this marked as unaswered for now, in case anyone has a better idea.

Anonymous#