US officials had been under the impression that the deal was in place, the source said, adding that when Brunson was not released, Pence spoke with Trump and the two agreed harsh new policy measures were needed to force the issue.
Trump has repeatedly demanded Brunson's release and said on Twitter last week that the pastor's detention was "a total disgrace".
Trump's threat came moments after Vice President Pence said the administration would impose sanctions on Turkey if Brunson was not released. "It would be a farce if it weren't so serious", he added.
US and Turkish officials had been working on a deal that would lead to Brunson's release, and Washington had expected him to be freed at his trial last week, a source in the United States familiar with developments told Reuters on Wednesday.
Brunson was arrested during this time on what his supporters say are fabricated charges. "This can not be allowed to continue", he continued. The Turkish government released Brunson from prison this week, but has forced him to stay under house arrest. He has denied the charges but faces up to 35 years in jail if found guilty.
The remarks follow similar efforts by Secretary of State Mike Pompeo and the State Department to highlight anti-government dissent in Iran.
Brunson's detention deepened a rift between North Atlantic Treaty Organisation allies Washington and Ankara - also at odds over the Syrian war and Turkey's plan to buy missile defences from Russian Federation - and financial markets took his transfer order as a positive sign. One of Brunson's attorneys is Jay Sekulow, who also represents Mr. Trump in the Russian Federation investigation.
Brunson, 50, has been detained in Turkey since 2016 after he was blamed for assisting rebels in a coup attempt against Turkish President Recep Tayyip Erdogan, one many have speculated was of Erdogan's doing to tighten his grip on power in the country.
He is on trial over allegations of espionage on behalf of Kurdish insurgents and the Fethullah Gülen network, a movement that is widely believed to have orchestrated a coup attempt in 2016. The Turkish government finally indicted him.
Prosecutors say Mr Brunson worked with a group led by Mr Gulen, who is living in exile in Pennsylvania, as well as with the Kurdistan Workers' Party (PKK).
On Wednesday, a Turkish court ordered that Brunson be freed from prison and placed under house arrest while his trial continues, according to Brunson's lawyer. "I forgive those who lie and bear false witness against me". The case was adjourned until October 12.