The track became one of Justin Bieber's biggest hits, spending six weeks at number one in the UK.
"That was a song I had written for '÷'. It just wouldn't have made it," Sheeran said in an American radio interview on Monday.
He said Justin's input had made the Grammy-nominated song "what it is".
"So going from a song that would have never been released to [being] the biggest song of last year - it just became Billboard's #1 of 2016 of the whole year, and nominated for Song of the Year at the Grammys - it just shows you that you shouldn't always write stuff off," he said.
The 25 year old has just released two new singles, Castle On The Hill and Shape Of You.
Sheeran also wrote Cold Water for Justin, Major Lazer, and MØ, but says at the time he had forgotten about it.
"'Cold Water' was a weird one because I didn't even know that song existed.
"It was a song I must've half-written somewhere at a time where I was writing a lot of songs," he told host Carson Daly.
"And [Diplo] emailed and was like, 'Yo, Justin [Bieber] wants to hop on the 'Cold Water' song. Is that cool?'
"And I'm just like, 'What are you talking about?'... And then the song came out and it was number one and I was like 'Oh, that song!' I remember doing it, but it was really, really slow.
"I remember hearing it and being like 'That kind of sounds like me. Oh wait, it was me.'"
The song spent five weeks at number one in summer 2016, ending Drake's fifteen-week run at the top of the charts with One Dance.