Kosovo Serbs are adamantly opposed to independence
In a statement after talks at the UN Security Council failed to break the impasse, they said the EU would take the lead in implementing a settlement.
Backed by the US and EU members, the Kosovo Albanians are expected to declare independence from Serbia.
Serbia, and its ally in the council, Russia, said that such a move would be illegal and urged further negotiations.
Last week, the EU said it was prepared to send 1,800 police officers and administrators to Kosovo.
Legal dispute looms
Following a closed debate described as tense, in which the Security Council heard from the Serbian prime minister and Kosovo's president, representatives from the US and EU stood together and said the two sides were irreconcilable.
"It's clear in our view that more negotiations in this or any other format will not make a difference," said Belgium's permanent representative, Johan Verbeke.
UK representative to the UN
"We therefore endorse the view of the European Union and US negotiators that the potential for a negotiated solution is now exhausted."
The statement said the EU stood "ready to play a leading role in implementing a settlement defining Kosovo's future status".
In April, UN special envoy Martti Ahtisaari put forward a plan offering Kosovo "supervised independence".
Under the proposal, international agencies would gradually steer Kosovo towards full independence and membership of the UN. But they would also prevent it from merging with Albania, or having its Serb areas split off to become part of Serbia.
Both the US and UK representatives said Security Council resolution 1244, which was passed after Nato threw Serbia out of Kosovo in 1999, allowed for the implementation of Mr Ahtisaari's plan.
"We would have preferred to do that through the Security Council, but we are entirely confident that resolution 1244 provides a sufficient legal base to move forward to a final settlement and to establish the necessary authorities needed to achieve that," said the UK's envoy, Sir John Sawers.
'Null and void'
But the joint US-EU statement drew a sharp reaction from Russia.
Its representative at the UN, Vitaly Churkin, insisted there was still "ample ground" for negotiations between Belgrade and Pristina to continue.
"Any move towards unilateral independence would clearly be outside the limits of international law and outside the limits of resolution 1244," he told reporters after the meeting.
Serbian Prime Minister Vojislav Kostunica said his country would declare "all unilateral acts of Albanian separatists null and void" - Kosovo would remain and integral and inalienable part of Serbia forever.The BBC's Laura Trevelyan in New York says Western diplomats expect Kosovo to declare its intention to become independent early next year and for the EU to take up the issue once Serbian elections have taken place in February.