Preheat the oven to 400°F. Line a 12-cup muffin pan with paper liners.
In a medium bowl, whisk together the flour, baking powder, baking soda, salt, cinnamon, and nutmeg.
In a large bowl, whisk together the melted butter, granulated sugar, and brown sugar until combined.
Add the eggs, sour cream, milk, and vanilla extract. Whisk until smooth.
Add the dry ingredients to the wet ingredients and stir just until combined. Do not overmix.
In a small bowl, combine the brown sugar and cinnamon for the swirl.
In another bowl, combine the flour, brown sugar, granulated sugar, cinnamon, and salt for the streusel. Cut in the cold butter with a pastry cutter or fork until coarse crumbs form.
Fill each muffin liner halfway with batter.
Sprinkle the cinnamon-sugar mixture evenly over the batter.
Top with the remaining batter until the liners are about ¾ full.
Sprinkle generously with the streusel topping.
Bake at 400°F for 5 minutes. Without opening the oven door, reduce the oven temperature to 375°F and continue baking for 13–15 minutes, or until a toothpick inserted into the center comes out clean or with a few moist crumbs attached.
Cool the muffins in the pan for 5 minutes before transferring them to a wire rack.
Whisk together the powdered sugar, milk, and vanilla extract until smooth. Drizzle over the cooled muffins before serving.