First lady-to-be Melania Trump likely won't be wearing designer Tom Ford's clothes in the White House.

Some of the designers who clamored to dress first lady Michelle Obama have been mixed about dressing Mrs. Trump, a former model. Ford became the latest to weigh in on the matter during an appearance this week on "The View."

The designer was asked to dress Melania Trump "quite a few years ago:" and declined because "she's not necessarily my image," he said on the ABC chat show Wednesday.

(Read also: Fidel Castro laid to rest in private ceremony in east Cuba)

Ford is a Democrat and voted for Hillary Clinton in the presidential election, he said, before adding that even if Clinton had won, she shouldn't have worn his clothes because "they're too expensive." His clothes cost a lot to make and the president needs to "relate to everybody," he explained.

Asked if Melania Trump should wear expensive clothes, Ford replied: "I'm going to leave that to Melania."

Ford was willing to dress first lady Michelle Obama for a London visit with Queen Elizabeth II in 2011 and isn't the first to be less enthusiastic about outfitting Mrs. Trump.

Sophie Theallat, whose designs have been worn by Mrs. Obama on several occasions, wrote in an open letter last month that she would not dress Mrs. Trump specifically brought up politics in urging her colleagues to take the same tact.

"The rhetoric of racism, sexism, and xenophobia unleashed by her husband's presidential campaign are incompatible with the shared values we live by," Theallat wrote.

(Read also: Victoria’s Secret models strut catwalk wearing Indonesian designers’ creations)

The letter prompted Women's Wear Daily , which covers the fashion industry, to survey other prominent designers on the question of working with Mrs. Trump, who began modeling in Paris and Milan at age 16 before settling in New York 20 years ago.

While Marc Jacobs said that he has "no interest whatsoever" in dressing Mrs. Trump, Tommy Hilfiger told the trade magazine he was willing, calling her "a beautiful woman who would make any designer's clothes look great."

Cynthia Rowley told WWD the question was "somewhat irrelevant," since Mrs. Trump can simply buy whatever she wants to wear.

"Checking someone's ethical beliefs before they're allowed to purchase, sets up an exclusionary dynamic that feeds into the exact mentality that is preventing us from moving forward in a positive direction," Rowley said.