Make filling: Whirl all ingredients in a food processor until fairly smooth, about 3 minutes.
Preheat oven to 350°. Make dough: In a large bowl, beat sugar and butter with a mixer until smooth. Beat in egg, vanilla, and milk. Add salt, baking powder, and flour and beat on low speed until well blended, scraping bowl as needed. Dough will be soft.
Divide dough in thirds. Roll each portion between sheets of parchment paper into a 5- by 15-in. rectangle. As made, stack and freeze rectangles on a baking sheet until dough is firm enough to hold its shape, about 15 minutes.
Spread 2/3 cup filling down center of a dough rectangle, leaving 1 1/2 in. uncovered on each long side. Using parchment, fold 1 long side of dough over filling, then fold other long side on top. Press lightly to seal; turn packet over. Cut on a diagonal to make 1-in.-wide pieces. Transfer to ungreased baking sheets, setting slightly apart. Repeat to make remaining cookies.
Bake until cookies are lightly browned, about 25 minutes, switching pan positions halfway through baking. Transfer to racks to cool.
Make icing: Stir powdered sugar, lemon juice, and about 1 tbsp. water (enough for a fairly thick consistency) in a bowl until smooth. Spread or drizzle tops of cookies with icing.
Make ahead: Up to 3 days, stored airtight.
Note: Nutritional analysis is per cookie.