In a 1-gallon jar, stir wine, brandy, 1 cup sugar, oranges, and lemon until sugar is dissolved. Seal airtight (if seal is rubber or synthetic rubber, cover jar top with plastic wrap before screwing on top). Let stand in a cool, dark place at least 1 week or up to 3 weeks, stirring occasionally.
Pour mixture through a strainer set over a pitcher (at least 2 1/2 qt.); save fruit for other uses (see notes) or discard. Taste, and add more sugar if desired. Chill until cold, at least 2 hours and up to 1 day.