How to optimize?

Optimization is a very difficult skill to learn, and really not in the scope of a single answer.

A basic methodology you can attempt is to never ever ever restart from scratch. That is the worst way to optimize code, as you'll make the same pitfalls as before, or worse as you seem to have run in to here.

Optimizing for speed, which is what you seem to be doing, is all about benchmarks.

Figure out multiple methods by which to calculate something in your code, and then run those calculations 1,000, 10,000, 100,000 or even 1,000,000 times each. Using tick(), measure how long each calculation batch took, and (using a randomized set of inputs with known 'correct' outputs) the 'accuracy' of the calculation.

To explain with a concrete example:

local f1 = function(a)
	return a*a*a*a*a
end

local f2 = function(a)
	return a^5	
end

--Both of these functions do exactly the same thing, but which is faster?

local t = tick()
for i = 1, 10000 do
	f1(i)
end
print("function 1 took: ", tick() - t)

local t = tick()
for i = 1, 10000 do
	f2(i)
end
print("function 2 took: ", tick() - t)

Obviously, in your case, the calculations will be a lot more complex. It's up to you to figure out how they might be made more efficient, either by eliminating extraneous function calls, or simply reducing the amount called per second, without sacrificing accuracy.

A somewhat unrelated topic you should look into, which will help with optimization as well to a lesser extent, is refactoring. This is the process by which you take a badly-made or otherwise unwieldy codebase and turn it into something better.