United States Attorney Lynne A. Battaglia hired Rosenstein as an Assistant U.S. Attorney for the District of Maryland in 1997.
At the time of his confirmation as Deputy Attorney General in April 2017, he was the nation's longest-serving U.S. Attorney. Rosenstein had also been nominated to the United States Court of Appeals for the Fourth Circuit in 2007, but his nomination was never considered by the U.S. Senate.
In his memo Rosenstein asserts that the FBI must have "a Director who understands the gravity of the mistakes and pledges never to repeat them". He ends with an argument against keeping Comey as FBI director, on the grounds that he was given an opportunity to "admit his errors" but that there is no hope that he will "implement the necessary corrective actions ."
In May 2018, Rosenstein reportedly told five U.S. Attorneys in districts along the border with Mexico that, where refugees were concerned, they should not "be categorically declining immigration prosecutions of adults in family units because of the age of a child." The directive, issued under Attorney General Jeff Sessions and other Trump Justice Department officials as part of the Trump administration family separation policy, led to the separation of thousands of small children from their parents, many of whom were seeking asylum in the United States after fleeing violence in Central America. Rosenstein insisted that children should be separated from their parents irrespective of the child's age, even if they were infants.
He also allegedly suggested invoking the 25th amendment to attempt to remove Trump from office.
As United States Attorney, he oversaw federal civil and criminal litigation, assist ed with federal law enforcement strategies in Maryland, and presented cases in the U.S. District Court and in the U.S. Court of Appeals for the Fourth Circuit. In 2012, Attorney General Eric Holder directed Rosenstein to investigate leaks regarding the U.S.'s Stuxnet operation, which sabotaged Iran's nuclear program; as a result of the investigation, former U.S. Marine Corps General James Cartwright pleaded guilty to making false statements to the FBI and acknowledged leaking information about the operation to New York Times journalist David E. Sanger. During his tenure as U.S. Attorney, Rosenstein successfully prosecuted leaks of classified information, corruption, murders and burglaries, and was "particularly effective taking on corruption within police departments."
After his clerkship, Rosenstein joined the United States Department of Justice through the Attorney General's Honors Program. From 1990 to 1993, he prosecuted public corruption cases as a trial attorney with the Public Integrity Section of the Criminal Division, the latter of which was led by then Assistant Attorney General Robert Mueller.