In a large glass or stainless steel bowl, combine pluots, sugar, and lemon juice. Cover and let stand until mixture is very soupy and sugar has dissolved, 1 1/2 to 4 hours, stirring occasionally.
Enclose peppercorns, bay leaf, orange zest, and vanilla bean in a 6- by 6-in. square of cheesecloth; tie tight with kitchen twine.
Pour pluot mixture into a 5- to 6-qt. pot and add spice bundle and butter. Bring to a boil over medium-high heat, stirring occasionally. Meanwhile, set a small plate in freezer and sterilize jars.
Reduce heat to medium and cook jam at a low boil, stirring occasionally, until jam thickens somewhat and a small spoonful put on a cold plate thickens a bit, 40 to 60 minutes. Discard spice bundle, pressing juices into pot.
Ladle jam into sterilized jars, filling to 1/4 in. from tops of jars. Wipe rims clean. Attach rings and new lids, tightening firmly but not forcing. Process jars in a boiling water bath for 5 minutes (boil for 10 minutes at altitudes of 1,001 to 6,000 ft., 15 minutes above 6,000 ft.). Or let jars cool, then chill.
Make ahead: Up to 1 year at room temperature, processed, or 1 month, chilled.