Tsarnaev could be charged as early as Sunday, although it was not clear what those charges would be. The twin bombings killed three people and wounded more than 180.
The most serious charge available to federal prosecutors would be the use of a weapon of mass destruction to kill people, which carries a possible death sentence. Massachusetts does not have the death penalty.
Across the rattled city, churches opened their doors to remember the dead and ease the grief of the living.
At the Cathedral of the Holy Cross in South Boston, photographs of the three people killed in the attack and an MIT police officer slain Thursday were displayed on the altar, the faces illuminated by glowing white pillar candles, one for each person lost.
"I hope we can all heal and move forward," said Kelly McKernan, who was crying as she left the service. "And obviously, the Mass today was a first step for us in that direction."
A six-block swath of Boylston Street, where the bombs were detonated, remained closed Sunday, though police at the scene told pedestrians it was expected to reopen before Monday morning.
Boston's historic Trinity Church could not host services Sunday because it was within the crime scene, but the congregation was invited to worship at the Temple Israel synagogue instead. The FBI allowed church officials a half-hour Saturday to go inside to gather the priests' robes, the wine and bread for Sunday's service.
Trinity's Rev. Samuel T. Lloyd III offered a prayer for those who were slain "and for those who must rebuild their lives without the legs that they ran and walked on last week."
"So where is God when the terrorists do their work?" Lloyd asked. "God is there, holding us and sustaining us. God is in the pain the victims are suffering, and the healing that will go on. God is with us as we try still to build a just world, a world where there will not be terrorists doing their terrible damage."