Preheat your oven to 350°F and line a 12-count muffin pan with cupcake liners.
In a medium bowl, whisk the flour, baking powder, salt, cinnamon, and nutmeg.
In a larger bowl, whisk the oil, sugar, room temperature eggs, rum extract, eggnog, and sour cream until smooth. Add the dry ingredients and whisk until smooth.
Pour the batter evenly into the cupcake liners, filling about ¾ of the way up. Gently tap the pan on the counter to release any trapped air bubbles and bake for 16-18 minutes. A toothpick inserted into the center should come out clean.
Make the eggnog buttercream frosting: In a large bowl, use an electric mixer to beat the softened butter, powdered sugar, rum extract, eggnog, cinnamon, and nutmeg until smooth and fluffy. Scrape down the sides of the bowl as needed. If the frosting is too thin you can add more powdered sugar, or if it's too thick you can add more eggnog.
Pipe the frosting onto the cooled cupcakes. I used a Wilton 1M piping tip.