Desperate for a win in St Petersburg after being held to a 1-1 draw by Switzerland in their opening fixture, Brazil looked like missing out on the three points when the decision to award Neymar a spot-kick was overturned in the 79th minute.
The former Barcelona star went down softly under the challenge of Giancarlo Gonzalez, with referee Bjorn Kuipers initially pointing to the spot.
However, VAR intervention led to a reversal of the decision, as replays showed minimal contact between Gonzalez and Neymar, who had fallen backwards as if pulled down.
The incident came late in a match Brazil dominated but in which for long spells they struggled to break down a determined Costa Rica defence.
However, with the clock having ticked past 90 minutes, Coutinho and Neymar stepped forward to break Costa Rican hearts and spare Brazilian blushes, sending their side top of Group E ahead of the clash between Serbia and Switzerland.
Neymar dropped to his knees in tears on the final whistle after the tournament's most successful side came through a real test of character against a Costa Rica side whose hopes of progression are now over.
— FIFA World Cup (@FIFAWorldCup) 22 June 2018
Costa Rica – rightly anticipating that Brazil would enjoy the lion's share of possession – sat deep but looked dangerous when they were given the rare opportunity to counter.
Celso Borges should have done better when he dragged an effort wide of the target from a Cristian Gamboa cutback after 13 minutes and, for all of their time on the ball, Brazil found it difficult to find a way through the Costa Rica defence.
Gabriel Jesus hammered a shot into the back of the net from an offside position in the 25th minute, rightly prompting a flag from the linesman, but the incident sparked a period of Brazil pressure, with Marcelo and Coutinho trying their luck from distance and a Neymar cross necessitating a desperate defensive header over the crossbar.
Marcelo had looked most likely to make something happen for Brazil but found Keylor Navas equal to his low drive five minutes from the break, the Real Madrid goalkeeper stooping to halt the game's first shot on target from his clubmate.
Neymar – named in the starting XI despite struggling with an ankle injury this week - confronted the referee on the pitch and in the tunnel at half-time, presumably taking objection to Costa Rica's rough-and-ready approach to dealing with him.
And Brazil returned to the field with added purpose after the break, with Jesus heading against the crossbar before Navas diverted Coutinho's follow-up wide.
Navas – a growing thorn in Brazil's side – pulled off even greater heroics to deny Neymar with a fingertip save soon after, while another Coutinho strike was directed straight at the goalkeeper.
Neymar curled around the right-hand post from 18 yards after the defence opened up for him in the 72nd minute, before the officials arrived at the correct decision via VAR to deny him a penalty.
That incident threatened to dominate the headlines until Coutinho prodded Brazil in front in the 91st minute, getting on the end of a Roberto Firmino nod down and a helpful touch from Jesus.
And Neymar was on hand to tap home Douglas Costa's delicate chipped cross even deeper into stoppage time to wrap up three points that had appeared like slipping through Brazilian fingers.
The relief was clear as the tears rolled down the face of the world's most expensive player, who surpassed Romario in third on the country's all-time list of goalscorers.
— OptaJoe (@OptaJoe) 22 June 2018
Photo Getty Images. Caption: Brazil forward Neymar.