In the bowl of a stand mixer, beat together the peanut butter and sugar until smooth. Mix in the milk, heavy whipping cream and vanilla until the sugar is dissolved.
Pour the mixture unto your ice cream maker and process according to manufacturer's instructions. About 5 minutes before the ice cream is done, add in the peanut butter cups. When the ice cream is done, pour into an airtight container. Freeze for at least 2 hours.