Preheat oven to 350°. Break up crackers and whirl half at a time in a food processor until finely ground. Spread evenly on a rimmed baking sheet. Bake, stirring every 4 to 5 minutes, until golden brown throughout, 8 to 10 minutes total. Pour into a bowl.
Meanwhile, heat butter in a medium saucepan over medium-low heat until melted and deep golden, swirling pan often, 12 to 15 minutes. Drizzle crumbs with butter and toss to coat. Let cool.
Spray a 9-in. round, 2 1/2- to 3-in.-deep springform pan with cooking-oil spray. Line pan bottom with a 9-in. circle of parchment paper. Line sides with 3-in.-wide strips of parchment; spray again.
Pour cream into a large bowl. Beating with a mixer on medium speed, slowly drizzle in sweetened condensed milk, then beat to stiff peaks. (Beating on medium speed takes longer, but helps build a stable structure.) Set aside, chilled.
Cut cheeks from mangoes parallel to center pits. Scoop out flesh from cheeks with a spoon and slice flesh from pits. Coarsely purée fruit in a clean food processor. Measure 2 cups (save extra for other uses). If you like, add lime juice so purée tastes sweet-tart.
Evenly sprinkle bottom of lined pan with 1/2 cup graham-cracker crumbs and spread in an even layer using a small offset spatula. Dollop half of whipped cream on top. Carefully spread cream level without stirring up crumbs. Spoon 1 cup mango purée on top and spread level. Sprinkle 1 cup crumbs on top. Repeat cream and mango layers. Sprinkle top with remaining crumbs but don't smooth down.
Wrap dessert with plastic wrap and freeze until firm but still sliceable, about 6 hours. Remove pan rim and parchment and cut into wedges.
*Also called Champagne mangoes, aromatic, satiny Ataulfo mangoes have almost no fibers. Find at well-stocked grocery stores, Asian markets, and melissas.com; or use another variety.
Make ahead: Up to 2 weeks, frozen; to serve, let stand at room temperature until just soft enough to slice, 45 minutes to 1 hour.