Heat olive oil in a large skillet over high heat.
Just as the pan starts to smoke, tip in the shishito peppers and kumquats. Let them sit in the pan for a moment before giving the pan a quick shake or jostling them a bit with a spatula or tongs.
Continue to sauté for a few moments more, until there is a good bit of charring on the peppers and caramelization on the kumquats.
Remove the peppers and kumquats from the pan, then finish with a sprinkling of flake salt. Serve right away.