Preheat the oven to 350°F. Line a muffin pan with paper liners and place an oreo at the bottom of each. Set aside.
In a small bowl, mix together the milk and vinegar. Set aside.
In a medium bowl, mix together the flour, baking powder, baking soda and salt.
In a large bowl, beat together the eggs. Add in the sugar, vanilla and oil, and mix until fully incorporated.
Slowly add in the dry ingredients to the wet ingredients, mixing until fully combined.
Scoop the batter into the prepared muffin pan so that each cup is 2/3 cup full. Bake for 12-15 minutes, or until a toothpick inserted comes out clean.
To make the frosting: mix together all of the frosting ingredients in a large bowl. Add in more or less milk, depending on your desired consistency.
Once the cupcakes are cool, place the frosting in a ziplock bag with a corner clipped, or a piping bag. Swirl the frosting on each individual cupcake.