Stir together milk, chai tea bags, pumpkin puree, dark brown sugar, and pumpkin pie spice in a saucepan.
Heat the mixture over medium-low heat for 5-6 minutes. Discard the tea bags and add vanilla extract and stir well.
Froth the mixture using an electric frother, a handheld electric or manual frother, or a French press.
Pour into two cups and serve hot.