In response, Trump earlier this week declared himself "your president of law and order" and vowed to return order to American streets using the military if widespread violence isn't quelled.

The comments from Mattis are a significant moment for a man who has kept mostly silent since leaving the administration.

Federal law enforcement officers violently broke up peaceful protests in front of the White House on Monday, apparently so Trump could stage a photo-op at a church across the street from Lafayette Square, where protesters had gathered.

The former secretary, who resigned from Trump's Cabinet, also indirectly criticized current Defense Secretary Mark Esper's use of the word "battlespace" in reference to American cities.

"We must reject any thinking of our cities as a 'battlespace' that our uniformed military is called upon to 'dominate,'" Mattis said.

Esper also specifically rejected the use of active duty forces in a law enforcement role at this time -- comments that put him on shaky ground with the White House.

White House press secretary Kayleigh McEnany would not directly answer Wednesday whether Trump still has confidence in Esper, saying instead, "as of right now Secretary Esper is still Secretary Esper."

"With regard to whether the President has confidence, I would say if he loses confidence in Secretary Esper, I'm sure you all will be the first to know," McEnany said during Wednesday's press briefing.

"Should the President lose faith, we will all learn about that in the future," she added.

But Mattis directed most of ire at Trump saying "Never did I dream that troops taking that same oath would be ordered under any circumstance to violate the Constitutional rights of their fellow citizens—much less to provide a bizarre photo op for the elected commander-in-chief, with military leadership standing alongside."

Trump on Wednesday evening softened his tone around sending the military into American cities, saying, "I don't think we'll have to," before reiterating that he has "very strong powers to do it" in an interview with his former press secretary Sean Spicer.

Still, Mattis made clear that his blistering assessment of the President extends beyond any one issue.