Charles, who played Sarah Beaumont in the soap for two stints in the 1990s and 2010s, alleges she was targeted for "taking a job from an Australian".
Charles made the claim in her forthcoming memoir, Soap Star, which has been previewed in The Herald Sun.
Fremantle Media, which produces the long-running Australian soap, has been contacted by the BBC for a comment.
In her book, Charles alleges that she found a handwritten letter sent to the immigration department shortly after her six-month probation period on Neighbours ended in 1996, just as her contract was about to be extended.
A day later, former Neighbours actor Meyne Wyatt said he too had experienced racism on set in a tweet.
"Neighbours strives to be a platform for diversity and inclusion on-screen and off-screen," a spokesperson for Fremantle Media told the BBC.
Floyd didn't die from heart attack, says expert.
'I feel just as Indian as I feel black' Video'I feel just as Indian as I feel black'
Neighbours star says cast mates wanted her deported6