Rosenstein took charge of overseeing Mueller's Russian Federation probe after former Attorney General Jeff Sessions recused himself from the investigation.
In September the New York Times reported, citing anonymous sources, that Mr Rosenstein had discussed recruiting cabinet members to invoke the 25th amendment of the U.S. constitution, which provides for the removal of a president if he is deemed unfit for office.
Citing anonymous sources, ABC claims that Rosenstein will leave his position as the nation's number two law enforcement officer around the time William Barr, President Trump's nominee for Attorney General, takes office following a Senate confirmation.
Graham, whom Republicans have selected for the Judiciary chairmanship, told reporters he was very quick to ask Barr about the special counsel's work. Barr and Mueller worked together when Barr was Bush's attorney general between 1991 and 1993 and Mueller oversaw the department's criminal division. Lindsey Graham, R-S.C., on Wednesday, ahead of next week's confirmation hearing.
"Mr. Barr is highly capable, highly respected, and will provide new and much-needed leadership for the Department of Justice", Graham said.
A source close to Rosenstein told NBC that the deputy attorney general's departure hinges specifically around the report Mueller will soon file on his sprawling investigation into Russia's interference in the election.
If Mr Barr is confirmed, he is likely to take over the role of overseeing the Russian Federation investigation.
Rosenstein reportedly assured them that he would assist Barr with the transition and help him find a replacement deputy.
Trump has denounced the investigation as a witch hunt.
"I'm going to do a deep dive into the FISA issue, I think he'll be part of it", Graham said of Barr.
Barr has made critical comments of the Mueller investigation in the past, including an unsolicited memo he sent the Justice Department a year ago critiquing Mueller's investigation into whether the president had sought to obstruct justice by firing Comey.
Mr Rosenstein has yet to comment on the reports.
Graham said Mueller and Barr are "best friends" and have known each other for over 20 years, their wives have been in a Bible study group together and Mueller has attended the weddings of two of Barr's daughters.