Make sauce: Combine ingredients in a bowl, adding hot sauce, salt, anchovies, and olives to taste. Set aside 30 to 60 minutes for flavors to come together.
Heat a grill to high (about 450°). Prepare fish: In a large, shallow bowl, coat fish with oil. Combine pepper, kosher salt, and celery salt; rub all over fish. Oil cooking grate, using a wad of oiled paper towels and tongs. Grill fish, turning once, until marks appear and fish is just cooked through, 5 to 6 minutes total. Serve with sauce.
*To pit olives, smash one at a time with the flat side of a chef's knife, and cut out pit.