Is there a problem with this function?

You should tab & space your code properly

function teams()
    local balanceTeams = function(players, teams, randomize, callback)
        for key = 1, #players do
            local value = table.remove(players, randomize and math.random(#players) or 1);
            if (not callback) or callback(key, value) then
                value.TeamColor = teams[(key % (#teams + 1)) + 1];
            end
        end
    end;

    balanceTeams(
    players:GetPlayers(), -- List of all players
    {BrickColor.new("Bright red"), BrickColor.new("Bright blue")}, -- List of all teams
    true -- Should it be random or not?
    );

You will see you are missing a final end and that teams() is never called.

There is only one more trouble with this script:

teams[(key % (#teams + 1)) + 1];

#teams + 1 should be replaced with just #teams.

When you do mod, you exclude the top value (#teams) -- however, you do include 0. So mod #teams + 1 actually allows #teams + 1 values.

(Just as the correct % #teams allows for #teams different values, [0, 1, 2, .., #teams - 1])