Emmanuel Ledesma was prominent in all of Boro's creative early work, hitting the crossbar with a tremendous strike from 22 yards out in the third minute.
In an open game where Newcastle showed up well, Birger Meling, who has impressed in his short time with the club, defended well to deny Jonathyn Quinn.
There was danger for Boro in the 12th mintute when they conceded a free-kick just outside the box. Adam Armstrong's effort hit the wall and Boro broke with Curtis Main forcing a great save from Newcastle keeper Jak Alnwick with a volley from the edge of the area.
United made a change after 25 minutes when Kevin Mbabu was forced off injured. It didn't interrupt the Magpies' rhythm and determined work from Armstrong won a corner which Boro dealt with comfortably.
With just over half-an-hour played Boro upped the pressure on the Newcastle goal.
A sharp shot from Curtis Main on 34 minutes close to the edge of the area forced the first in a quick succession of saves from Alnwick.
Next to trouble Alnwick was Luke Williams, who's shot from 20 yards forced yet another brilliant save from the young keeper. Adam Jackson and Ledesma efforts from 18 and 19 yards respectively continued the pressure on the Newcastle goal.
There was nothing Alnwick could do as Boro took the lead with a stunning Emmanuel Ledesma free-kick from just outside the box on 39 minutes.
Boro continued to trouble the Newcastle goal, with Alnwick yet again being forced to make a save after a Ledesma knockdown played in Luke Williams.
United equalised in first half injury-time when a goalmouth scramble resulted in Jonathyn Quinn turning in from 10 yards.
Boro were quick to respond and just 30 seconds later were back putting pressure on the Newcastle goal as Ledesma, who was impressive in the first half, hitting the post from 22 yards.
Half-time: Middlesbrough 1 Newcastle 1
Thunder, lightning and heavy rain swept over the Riverside during the break and it hadn't abated when the second half got underway.
Boro started the second half brightly, taking the lead after just 90 seconds when Ryan Brobbel got the final touch after Main's effort appeared to be heading in.
Immediately after the goal United were forced into a second substitution, having to replace the outstanding Alnwick with Jonathan Mitchell.
Boro's lead lasted less than a minute, Adam Armstrong slotting home from close range to pull the visitors back level.
Jackson was disappointed not to be on target when meeting a left-wing cross with a powerful header as Boro set about regaining a lead already held twice.
With just over an hour played Ledesma toe-poked a shot towards goal which was deflected out for a corner. Adam Jackson made up for his earlier disappointment as he scored with another powerful header from eight yards out to put Boro 3-2 up.
Newcastle didn't have time to respond as Boro extended their lead to 4-2 with a low strike from Curtis Main as he met a cross from the right on 63 minutes.
After the flurry of activity on the pitch and in the skies above, things began to calm down.
With ten minutes left and the rain returning, Ledesma brought a good save at his near post from Mitchell with a low, left-footed shot from an angled eight yards.
Main forced a flying save from Mitchell after 86 minutes with a shot from just inside the area. He followed this up 30 seconds later with another shot across goal that flashed wide.
Williams continued to mount pressure on United just 60 seconds later by firing another shot just wide.
Deep into injury-time a header from Lubomir Satka too unfortunate deflection off young Boro midfielder Bradley Halliday made the score 4-3. That shouldn't have spoiled his night after an encouraging display.
Boro: Meling (56)
Newcastle: Sterry (85)
Boro: Konstantopoulos; Bennett, Burn, Jackson (Capt), Meling, B Halliday, Brobbel, Morris, Main, Ledesma, L Williams. Unused subs: Coddington (gk); Kneeshaw, Sirrell, Waters, Weledji.
Newcastle United: Alnwick (Mitchell 46); Mbabu (Sterry 25), Good (Capt), Booth (Gillesphey 46), Streete, Satka, Quinn, Olley, Armstrong, Amalfitano, Gilliead. Unused subs. Aird, Heardman.
Boro Website Man of the Match: Emmanuel Ledesma