By Ian Dunt

Some of the people who raise concerns about immigration are “frankly racist”, Anna Soubry has said.

The defence minister said she regularly dealt with people expressing discomfort with the level of immigration into the UK and tried to convince them the situation was more complicated than they might realise, but that there was a strain of racism behind some of the criticism.

“When you make the case with people who come and see me in my constituency surgery who say ‘I’m really worried about immigration’ you say: ‘Really, why? This is Broxtowe. We don’t have a problem with immigrants’,” she told BBC1′s The Andrew Marr Programme.

“When you explain that to them they get it. Not all of it. Some people have prejudices, some people are frankly racist, but there are many who just don’t know the argument.”

Downing Street did not reference the comments directly but said: “We understand people have legitimate concerns about immigration.”

Politicians have been resistant to labelling anyone expressing criticism of immigration racist since Gordon Brown’s disastrous exchange with Gillian Duffy during the 2010 general election campaign.

But polling suggests there is a hardcore rump of around 25% who are comfortable expressing prejudice.

A YouGov poll conducted just before the European elections showed 26% of people thought the government should encourage immigrants and their families to leave Britain, even if they were born in the UK.

Soubry added: “Undoubtedly there are certain parts of the country where there has been a huge influx of people. If you don’t put the right sort of infrastructure, you don’t put the schools and hospitals, people get fed up with it.

“But the principle of immigration, we have got to be up front about it. Immigrants have played a hugely important role in our society. They come over here overwhelmingly to work, they do not come here to scrounge.

“There are far fewer immigrants claiming benefit than there are people who have been born and bred in our country. That’s the debate we need to have.”

Soubry’s comments come as the three main parties are convulsed by debate over how to respond to Ukip’s victory in the European election.

Ed Miliband has so far refused to toughen up his approach to the subject, despite demands for a harsher stance from shadow chancellor Ed Balls, although he shocked some Labour figures by inserting a passage about west Africans into a speech in Thurrock last week.

“There is a contradiction in telling Ukip voters in Thurrock that you share their pain about west Africans and expecting those same west Africans to vote for the Labour party elsewhere in the country,” former Labour minister Diane Abbott responded.

Seven Labour MPs, including Frank Field and Kate Hoey, wrote to Miliband demanding a more critical approach to the impact of immigration on local services.