While there is some truth about immigration being seen as the Tories trump card, (their only one) its also not about anyone living next door to a foreigner, that just debases the issue to one of racism, which it is not.
We have numerous services that are under funded, NHS, Schools, councils, roads, etc etc, and just allowing free movement, unchecked of people into a country (any country) just continues the cost to the treasury of funding those services.
The overriding question is, if these people are in genuine fear of their lives, why didn't they plead asylum in France?
Under EU law (a treaty we are still bound by) anyone seeking asylum should do so at the first safe haven they come to. That way their numbers are allocated to member countries as per previous existing agreements.
People seeking assylum don't get to choose which country they fancy living in.