Yep - you got it, kind of like the kind you get at the grocery store, but better because you're controlling the ingredients which means nothing you can't pronounce. Only downside, since you can pronounce all the ingredients these have a limited shelf life - upside, they taste so good who would keep them around long enough to worry about the shelf life?
I made these for the kids third birthday and they were a big hit with both the other kids and the adults at the party. I had extra filling and used that as icing, then sprinkled cocoa powder on the top. For best results, if you use filling as icing, place in fridge after icing to let them set up a bit before serving.
2 tsp. water
7oz (1 jar) marshmallow cream (for you Right Coasters, not sure if Fluff would work for this)
2/3c powdered sugar
1 tsp vanilla
Whip marshmallow cream, shortening, powdered sugar, and vanilla together until mixture gets fluffy. Spoon into a piping bag with a long tip and squeeze into the cupcake flavor of your choice. I choose chocolate for this.