Investors also braced for possible hints on when the next interest rate hike is coming, ahead of the Fed's rate-setting two-day committee meet starting on Tuesday.
Spot gold edged down 0.04 percent to $1,243.80 per ounce at 0049 GMT after touching a three-week high on $1,247.48 an ounce in the previous session. It has risen about 1.3 percent for the week so far.
The ECB is expected to lay the groundwork for an autumn policy shift when it meets, emphasising improved growth while tempering expectations after previously setting off a mini tantrum in financial markets.
Brent crude futures , the international benchmark for oil prices, were at $48.73 per barrel at 0128 GMT, down 11 cents, or 0.2 percent, from their last close.
Spot gold was up about 0.1 percent at $1,242.86 per ounce at 0047 GMT. In the previous session, it hit its highest since June 30 at $1,244.56.
The U.S. government reported on Friday that consumer prices were unchanged in June and retail sales declined for a second straight month.
Spot gold rose 0.3 percent to $1,223.11 per ounce at 0351 GMT. U.S. gold futures for August delivery rose 0.3 percent to $1,222.60 per ounce.
Most regions also expect the middling expansion to continue in the coming months, but businesses said they continue to feel pressure to offer higher pay and benefits to attract qualified workers, according to the Fed's report on economic conditions.
In what may be one of her last appearances before Congress, Yellen depicted an economy that, while growing slowly, continued to add jobs, benefited from steady household consumption and a recent jump in business investment, and was now being supported by stronger economic conditions abroad.
"This judgement reflects our view that monetary policy remains accommodative. Ongoing job gains should continue to support the growth of incomes and, therefore, consumer spending.
Brent crude futures rose 65 cents, or 1.4 percent to $48.17 per barrel by 0155 GMT, while U.S. West Texas Intermediate (WTI) crude futures were at $45.77 per barrel, up 73 cents, or 1.6 percent.
Spot gold rose 0.1 percent to $1,218.60 per ounce at 0046 GMT, adding to a 0.3 percent gain in the previous session, its biggest rise since June 23.
Politico said sources in the White House, the Treasury Department and on Capitol Hill said that if Cohn decides he wants the job, he is likely to get it.
Spot gold fell 0.2 percent to $1,212.01 per ounce at 0043 GMT. It hit a low of $1,204.45 in the previous session, its lowest since March 15.
"The dollar's strength was definitely a negative for gold, as was the fact that yields on both sides of the Atlantic remained elevated," said INTL FCStone analyst Edward Meir.
Private employers added 158,000 jobs in June, the ADP National Employment Report showed, coming in below the estimated gain of 185,000 and suggesting cooling in the U.S. labour market as it nears full employment.
The gains reflected firm fuel demand in the United States, where data from the American Petroleum Institute (API) on Wednesday showed that U.S. crude inventories fell by 5.8 million barrels in the week to June 30 to 503.7 million.
Spot gold was nearly flat at $1,226.70 per ounce at 0421 GMT. It touched $1,217.14 in the previous session, its lowest since May 10.
The details of the meeting, at which the U.S. central bank voted to raise interest rates, also showed that several officials wanted to announce a start to the process of reducing the Fed's large portfolio of Treasury bonds and mortgage-backed securities by the end of August but others wanted to wait until later in the year.
Yellen, who is 70 years old and the most powerful figure in world finance, had a health scare in 2015, receiving emergency medical attention following a speech during which she appeared to lose concentration. Following that episode, the Fed said Yellen felt dehydrated after speaking for nearly an hour.
Brent crude futures had climbed 13 cents, or 0.3 percent, to $48.90 per barrel by 0643 GMT, after jumping 5.2 percent last week in their first weekly gain in six weeks.
Speaking to reporters on the sidelines of an event in London, Bullard added that it was "disconcerting" that market measures of inflation in the world's largest economy had fallen despite the central bank raising interest rates twice so far this year.
Spot gold was nearly flat at $1,245.08 per ounce at 0104 GMT. It was set to end about 1 percent lower for the week, about 2 percent lower for month. For the quarter, gold is down 0.3 percent.
Gold held firm on Thursday as the U.S. dollar hovered near 10-month lows on bets that central banks in Europe and Britain are preparing to scale back monetary stimulus.