Nathan Coulter-Nile top-scored for the Aussies with a stunning 92 from 60 balls.
It means that Australia have won 10 of the last 11 meetings between the teams, have been victorious in their last 10 ODIs and remain unbeaten in the early stages of this tournament.
But in a tight finish, Starc dismissed Carlos Brathwaite and Holder within four balls - claiming his 150th ODI wicket in the process - before knocking over Shane Cottrell to earn his sixth five-wicket haul and inflict defeat on Windies on the occasion of their 800th recorded ODI.
Oshane Thomas had Australia captain Aaron Finch (six) caught before opening partner David Warner (three) was held at backward point off Sheldon Cottrell - a wicket that saw the left-arm quick, still a member of the Jamaica Defence Force, celebrate with his trademark salute.
Conscious that his momentum would carry him over, Cottrell threw the ball into the air and returned to the playing area to take the catch.
Former Sri Lanka player Russel Arnold was in agreement with Vaughan, saying: 'Amazing Sheldon Cottrell simply awesome. way better than the Stokes catch for me'.
China's Huawei to sell undersea cable business, buyer's exchange filing shows
The ban on US tech companies doing business with Huawei is expected to drastically curtail demand for Huawei phones in the future. A Huawei logo is seen on the side of a building at the headquarters in Shenzhen, Guangdong province, China May 30, 2019.
The former skipper's resistance allowed Coulter-Nile to pepper the leg side with some clean hitting.
The 39-year-old posted his 1,000th World Cup run with a driven four but was not to win a third review when given out lbw to Starc - the decision staying with umpire Chris Gaffaney's call.
Before his dismissal, Gayle had used DRS to survive twice off back-to-back deliveries in Starc' previous over.
All-rounders Glenn Maxwell and Marcus Stoinis follow at number five and six, with wicketkeeper-batsman Alex Carey at seven. Two balls later Gayle successfully reviewed after being given out lbw to a Starc delivery drifting down leg.
But as long as Holder was there the West Indies still had a chance.
"You have to keep trying to get wickets, once the West Indies batsmen get set they are so destructive and can hit it to all parts", Australian captain Aaron Finch said.
Starc was also involved in one of the game's more controversial moments, which ultimately led commentator and West Indies great Michael Holding to describe the umpires Chris Gaffaney and Ruchira Palliyaguruge as "weak" and "atrocious".