How to check if player recieves a new chat message with a local script?

You can use a Serverscript and Localscript and communicate through a RemoteEvent.

The serverscript will listen to all players when they chat and make sure that the message is going to show up for the player, then communicate to this player's client script.

Then you can do whatever you want to happen through that localscript

ServerScript

local ReplicatedStorage = game:FindService("ReplicatedStorage") 
local RemoteEvent = ReplicatedStorage.Chat -- assuming you have a remoteevent called "Chat" under "ReplicatedStorage" 

game.Players.PlayerAdded:Connect(function(player)
    player.Chatted:Connect(function(msg, intendedPlr)
        if not intendedPlr or intendedPlr == player then -- check if plr is receiving a normal chat or whisper
            RemoteEvent:FireClient(player)
        end
    end)
end)

LocalScript

local ReplicatedStorage = game:FindService("ReplicatedStorage") 
local RemoteEvent = ReplicatedStorage.Chat

RemoteEvent.OnClientEvent:Connect(function()
    -- plug in what you want to happen on the client side whenever the player gets a new chat message
end)

Hope I helped, if you have any questions please let me know!