He said his Socialist government had to act and speak "differently" to counter the threat from FN.
That, in itself, is a victory for FN, as the government tries to implement some of their proposed reforms.
His comments came after an opinion poll showed that FN leader Marine Le Pen would beat incumbent Francois Hollande in presidential elections in 2017.
|Marine Le Pen's popularity among voters is rising|
Large numbers of migrants have arrived there in recent months. Many of them have been trying to cross illegally into the UK, a country they see as providing generous benefits to refugees.
"In France, the extreme right of Marine Le Pen is at the gates of power," Mr Valls said at a gathering of fellow left-wing politicians in Bologna, Italy.
"We have to act differently. We have to speak differently. In order to be listened to and to be heard.
"And I, as a man of the left, will never be able to resign myself to that because it will be the weakest who will be the first to suffer. And it will also be a terrible, perhaps fatal, blow to Europe," Valls added.
His warning comes as President Francois Hollande's ratings remain at an all-time low, and the government faces a confidence vote later this month.
|The number of people looking for a job in France has climbed to more than 3.4 million|
Mr Hollande replaced them and named a new cabinet in what was described as an emergency reshuffle.
In July, the number of people looking for work in France rose to a new record of 3,424,400.
FN - The Colourful, Inclusive, Far Right of France
Black and brown faces are not a rarity at events of France's National Front (FN) these days. Their presence may seem odd amid chants of "On est chez NOUS" ("This is OUR place").
Why would these people support a right-wing, anti-immigration group like the FN?
To hear them, it is the most natural thing in the world.
"You can be French by birth or French at heart," said a man marching with his Taiwanese wife at the front's May Day rally, their young mixed-raced daughter waving a tricolour on his shoulders.
The defence of national identity, one black FN supporter shrugs, has nothing to do with race.
Jean-Marie may have started the party for racial reasons, or, perhaps he was visionary enough to see the day coming when France would become part of Islam simply through population growth and immigration. Regardless, the party now has momentum, not just from the working class, but also from those who don't want to be under Shariah law within a generation or two. The left wing parties are completely oblivious to the threat.
Over the past two or three decades, however, the front has sought to shed its xenophobic image and turn itself into a mass party. "De-demonisation" has been the watchword under the current leader, Mr Le Pen's daughter Marine.
She remains virulently opposed to immigration, but has directed her ire at Europe's open borders. Her message that the influx of Romanians or Poles is destroying jobs has resonated with French workers of all races.
Tough on crime
The same is true of her claim to represent the struggling people of France betrayed by "globalised elites".
Jean-Philippe Virapin, a Paris municipal employee from the French Caribbean island of Martinique, says that what drew him to the front was "the defence of the modest, forgotten people".
Another FN member, Elie Chabot, a 22-year-old of Aramaean (Syrian) origin, says: "People of foreign origin suffer all the ills highlighted by Marine Le Pen - insecurity, mass migration and poverty."
1972: founded by Jean-Marie Le Pen
1983: First big election success in local poll
1986: Enters parliament
1980s: Gains votes from both nationalist right and hard left
2002: Le Pen comes shock second in presidential poll, beating socialist
2011: Marine Le Pen succeeds her father at head of the party
Key policies: Opposed to mass immigration and free trade; seeks exit from euro
How FN won in socialist heartland
Far right or hard left?
Mr Chabot says that in the multi-ethnic high rises of Bondy, the Paris suburb where he lives, he "would not be surprised if there were more immigrants voting for the National Front than whites".
The FN's emphasis on crime is striking a chord among middle-aged people. Mohammed Ayad-Zeddan, 40, lives in Henin-Beaumont, a northern town captured by the National Front in local elections in March.
He "does not care too much" about the front's stance on immigration but supports the new mayor's pledge to strengthen the local police force. "There is more and more crime here," he says.
Karim, 37, another Henin-Beaumont resident of North African origin, likes the idea of more security cameras. "That's good for everyone," he says.
The FN's increasing appeal among minorities is part of a deliberate strategy, according to blogger Octave Nitkowski, who has written a book about the FN.
In the old days, he notes, the French far-right used to frown upon any group that deviated from country's white, Christian, patriarchal tradition. Religious, sexual, or ethnic minorities were treated as suspect.
Now, Mr Nitkowski says, the FN has only one scapegoat: radical Islam. "They champion the rights of Jews, women, homosexuals and others by presenting them as victims of Muslim fundamentalists," he says. "The image of a single mother at its head is also a powerful symbol of openness." Republicans, are you reading this?
The FN's ethnic constituency projects a benevolent face, to be contrasted with the intolerance attributed to Islamism.
It may also reflect a strong desire to assimilate on the part of older immigrants, and a rejection of identity politics that is rising among the young.
"We will neither ban pork nor impose halal food” Laurent Brice, FN councillor, Henin-Beaumont
"It does not shock me that they should vote for the National Front," says Ahmed, a 30-year-old social worker from Henin-Beaumont.
"They have done everything to make sure their children are French, and the kids reject France!" he says.
The appeal of the National Front among people of foreign origin, of course, remains marginal. Many interpret its strong anti-Islamist stance as sign of hostility to immigrants themselves.
Last month, following its breakthrough at local elections, Ms Le Pen controversially said that FN municipalities "would not accept religious demands over school meals".
Ines Balia, a Muslim 19-year-old who goes to school in Henin-Beaumont, worries that students might be forced to eat pork in canteens.
More generally, Ms Balia says she has increasingly been hearing racist remarks around town in the two months since the FN won. "It looks as though it has become acceptable not to like Arabs," she says.
|Under Marine Le Pen, the FN has projected a more inclusive image|
This is fiercely denied by the FN. Laurent Brice, a senior figure in the new council, insists it is working for all residents regardless of religion or ethnicity.
Regarding school meals, he says, the front's stress on secularism simply means canteens will offer a choice of dishes. "We will neither ban pork nor impose halal food," he says. "This is not a problem."
In Henin-Beaumont, as elsewhere, politics is driving the FN towards a moderate stance. As one African-born resident puts it: "The front cannot afford to antagonise immigrants. If it wants a majority, it has to be inclusive."
France's National Front will never be as multi-ethnic as its national football team.
But as it seeks to establish itself as the true voice of the working class, you should expect to see more non-white faces at its rallies.