Preheat oven to 325°F. Place 4 tablespoons butter in a 9-inch round cake pan, and melt in the oven. Remove pan from oven and sprinkle the brown sugar evenly over butter.
Arrange the nectarines in an even layer in pan.
In a medium bowl, mix together flours, baking powder and baking soda. In a bowl attached to a stand mixer, beat 6 tablespoons butter on high until light and fluffy. Add the white sugar and beat until well combined. Beat in eggs, one at a time, then beat in the vanilla. With mixer on low, add flour mixture, alternating with the yogurt, and beat to combine.
With a spatula, spread the batter over nectarines. Bake until cake is dark golden brown and a toothpick inserted in center comes out clean, about 1 hour 10 minutes. Let cake cool in pan on a wire rack, for 15 minutes. Run a knife around edge of pan and invert cake onto a serving plate. Serve warm or at room temperature.