The government successfully weathered the high current account deficit (CAD) scare in the just-concluded financial year.
"I think what we have to do is slowly and steadily take actions to remove some of these curbs (on gold imports)," Rajan told analysts at the post-policy concall with researchers and analysts on Wednesday.
He, however, said the timing on relaxation on gold imports needs to be discussed with the government.
"It would be useful for some of the big uncertainties facing us to be behind us rather than still in front of us before major actions are taken up in this regard, but I don't rule it out," Rajan said.
Gold imports has been the second biggest import bill after crude, leading to higher CAD, which had touched 4.8 per cent of GDP in FY13.
The government had raised raised customs duty to 10 per cent on gold from 2 per cent earlier.
According to Finance Minister P Chidambaram, with the curbs in place, CAD was brought to about $35 billion in FY14.
Earlier this week, Chidambaram had indicated that the government might further ease restrictions on gold imports after the monetary policy.
"Some relaxations were made a few days ago when more banks were allowed to import gold. We could consider some more relaxations in consultation with RBI," he had said.