Preheat oven to 400F. Allow puff pastry to thaw if frozen and place on a baking sheet lined with parchment. Toss nectarine slices in a bowl with ginger and brown sugar. Arrange slices on top of puff pastry, leaving about a 1 1/2 inch edge.
Fold edges in around the fruit to form a square. If desired, dot fruit with butter.
Make an egg wash (optional): Whisk together egg and milk. Brush the edges of the pastry with egg wash. Bake for 25-30 minutes, until pastry is golden and fruit is bubbly. Enjoy with whipped cream or vanilla ice cream while warm!