Heat 3 tbsp. oil in a large frying pan over medium heat. Add onion, 1 tbsp. oregano, and bay leaves and cook, stirring occasionally, until onion is softened, about 5 minutes. Meanwhile, mince garlic, roughly chop half of olives, and thinly slice chile; set chile aside.
To pan, add garlic, olives, capers, juices from tomato can, sugar, cinnamon, and wine. Simmer sauce, uncovered and stirring occasionally, until reduced slightly, 3 minutes. Stir in tomatoes, remaining 1 tbsp. oregano, and chile.
Slip fish into sauce and spoon some sauce on top. Reduce heat to medium-low, cover pan, and cook just until fish is opaque throughout, 8 to 10 minutes. Drizzle lime juice over fish and sauce.
Serve fish over rice. Spoon on sauce, drizzle with remaining oil, sprinkle with parsley, and add a lime slice.