In a bowl, whisk together the flour, baking powder, and salt. Set aside.
In the bowl of a stand mixer, beat together the butter with 1 1/2 cups of the peanut butter until smooth. Add the sugar and brown sugar and beat until smooth. This should take 4-5 minutes. Then, beat in the eggs and vanilla until incorporated. Stir in the reserved flour mixture until just combined.
Scoop about 1 1/2 tablespoons of dough (with a spoon or small ice cream scoop) onto an ungreased cookie sheet. Continue scooping balls of dough, spacing them about 2 inches apart. Gently press your thumb into the center of each ball to make an indentation. Then spoon about 1 teaspoon of jelly into each indent. Bake 12-13 minutes until lightly golden. Remove to a rack to cool.
In a small bowl, beat together the remaining 1/2 cup of peanut butter and confectioners' sugar. Whisk in the milk, a tablespoon at a time, until the frosting reaches a good consistency for piping. Add this mixture to a piping bag or plastic baggie. Snip off the tip of one corner of the baggie and pipe the frosting over the cookies.
Serve immediately or pack and store in an airtight container.