Trying to compare items in a table, can't seem to get it right. Any help?

I've been working on my own GUI Module so that I can edit GUI a bit faster in-game. I've made this function inside a module script to change the color of the player's interface. Here it is:

--//Module
function GuiModule.ChangeColor(guiTable, ignoreTable, guiColor)
	local defaultColor = Color3.fromRGB(255, 255, 255)
	for i, v in pairs (guiTable) do
		if v:IsA("GuiObject") then
			if v ~= ignoreTable[i] then --if v (current object) is not in the ignore table then:
				v.BackgroundColor3 = guiColor or defaultColor
				v.BorderColor3 = guiColor or defaultColor
				if v:IsA("ImageButton") or v:IsA("ImageLabel") then
					v.ImageColor3 = guiColor or defaultColor
				end
			end
		end
	end
end

--//LocalScript

local colorButtons = colorButtons:GetChildren()
local interfaceTable = screenGui:GetDescendants()

--Add connection for every button

for _, button in pairs(colorButtons:GetChildren()) do
    if button:IsA("ImageButton") then
        button.MouseButton1Down:Connect(function()
			GuiModule.ChangeColor(interfaceTable, colorButtons, button.ImageColor3)
		end)
    end
end

The basic idea is to have a table of GUI I do want to change, a table that I dont want to ignore, and a color to change them to.

However, when I compare the iteration and value to the ignore table, it still changes the colors of the ones I dont want to change. Its like 5am and Im not too sure what im doing wrong, or if im just really tired. Heres a gif of whats going on; and a gif of whats supposed to happen.

Whats happening:

https://gyazo.com/a91067c7fcd42f974d034806bccbb187

Whats supposed to happen (I removed the :IsA("ImageButton") check:

https://gyazo.com/dd3dbd06ac26c7a7fc7d522653d2c3c4

The reason I have the image button check in the first place, is because this is modularized, its going to be able to work in pretty much every situation. The only items I want to ignore are the ones in the ignore table, the only items I want to change are the ones in the guiTable, you get the idea.

Thanks for any help.