How can i change the a player's chat color without using teams?

You can do this through the Lua Chat System.

In the Chat you have to setup a few things. First of all a folder called ChatModules. In there you need to make a BoolValue called InsertDefaultModules and set it's value to true (check the mark). Lastly you insert a modulescript.

Here is an image for reference: link

In this modulescript you have to put the following code:

local Players = game:GetService("Players")
local MarketPlaceService = game:GetService("MarketplaceService")

local GamePassId = 1 -- change this ID
local VipChatColor = Color3.fromRGB(0, 0, 0) -- Change this to the wanted RGB value

local function Run(ChatService)
	local function OnSpeakerAdded(name)
		local speaker = ChatService:GetSpeaker(name)
		local player = speaker:GetPlayer()
		if player and MarketPlaceService:UserOwnsGamePassAsync(player.UserId, GamePassId) then
			speaker:SetExtraData("ChatColor", VipChatColor)
		end
	end
	
	ChatService.SpeakerAdded:Connect(OnSpeakerAdded)
end

return Run

What does this is everytime a speaker (or basically a player) is added it checks if the speaker is a player and whether or not he has the given gamepass. After that you can assign a new ChatColor to the given speaker.