Mix oil, lemongrass, and ginger; let sit overnight to develop flavors. Strain oil; discard lemongrass and ginger.
Set a medium bowl on a wet kitchen towel. Add yolks and 1 tsp. water and whisk to blend. Pour in oil, drop by drop at first and whisking constantly, until thickened; then pour in rest of oil in a thin stream and whisk until very thick (add 1 tsp. water if needed to thin). Put 1/3 cup aioli in a serving bowl and set aside.
Add crab, panko, onion, herbs, and salt to aioli in mixing bowl and stir to combine. Form a thick patty with 1/3 to 1/2 cup crab mixture. Repeat to form 11 more patties.
Heat 2 tbsp. oil in a large frying pan. Cook patties in batches, turning once and adding more oil as needed, until golden brown, about 8 minutes per batch. Serve with reserved aioli.
*Peel off the 2 or 3 tough outer layers and chop the tender inner core.