Rotherham took the lead in the 19th minute when captain Ollie Rathbone struck a low effort into the bottom corner from the edge of the box.
The hosts then added a second when Shane Ferguson broke down the left before converting another low effort.
Leeds loanee Joe Gelhardt then scored his first Sunderland goal just after the hour mark, but it was too little too late.
Here’s how Phil Smith rated the Sunderland players against Rotherham:
1. Anthony Patterson - 5
Nothing he could do about either goal and his distribution was good in the main, creating some openings on a night when his team were generally short of their best. 6 Photo: Martin Swinney
2. Luke O’Nien - 5
Poor in possession during the first half and not able to combine with Amad at all. Delivered a couple of excellent crosses in the first half that were converted by Gelhardt, but the first was ruled out for offside. 5 Photo: Frank Reid
3. Daniel Ballard - 5
Struggled a little to get his team going as he usually does in the closing stages, but the mitigation was that he could barely move through exhaustion. Recent game time is clearly catching up with a few players. 5 Photo: Frank Reid
4. Danny Batth - 5
Largely won his headers and kept Rotherham quiet in terms of clear chances, but brought off in the second as Mowbray tried to increase the energy in his side. 5 Photo: Frank Reid