With the match goalless, Inter's Kwadwo Asamoah and Sampdoria's Gregoire Defrel had efforts disallowed.
Inter had also had Radja Nainggolan's first-half strike ruled out by VAR.
Croatia midfielder Brozovic struck in the dying moments to earn Inter a second Serie A win of the season.
Nainggolan's goal was ruled out because of an offside in the build-up while Asamoah had a strike disallowed because the ball had earlier gone out of play.
VAR spotted an offside before what Defrel thought was a late winner and to compound matters the French forward was booked for removing his shirt to celebrate the disallowed goal.
The win lifts Inter to seventh in the table, one place below Sampdoria. Both teams have seven points after five games.
Tags:
Serie A