Don Johnson Recalls Meeting Ex Patti D'Arbanville 'Stark Naked' on a Table (Exclusive)

Don Johnson is widely known for having married his ex-wife Melanie Griffith not once but twice.

But between those unions he was partnered up with Andy Warhol muse, model and actress Patti D'Arbanville, with whom he shares son Jesse, 44.

He tells PEOPLE the way the two first met was somewhat unconventional. "The wife of the mother of my first son, I met her because I was working on a movie in New York, and I went down to Andy Warhol's factory," the Doctor Odyssey star, 74, says.

"I walked in, and standing on the table stark naked was the mother of my first son. She was doing what I think was the very first cover photo shoot for Interview magazine," he says. "She was just so beautiful and captivating."

The two had undeniable chemistry, but they didn't become an official couple until 1981 when they ran into each other at a restaurant in Los Angeles.

According to a 1984 PEOPLE profile on the actor, Johnson arrived at the restaurant with his then-girlfriend Tanya Tucker but left with D'Arbanville. "She moved in that night," he told PEOPLE at the time. "And we stayed in bed for about eight days. The houseboy kept bringing food and water."

When Patti got pregnant with Jesse, Johnson told PEOPLE they decided not to get married: "I can't commit to marrying right now," he told her.

"I'm not into that either," replied D'Arbanville.

But their union and fatherhood soon became something Johnson embraced and credited with his sobriety.

"I walked into the breakfast room one morning," he said in the 1984 PEOPLE , of deciding to quit alcohol and drugs. "The sun was shining, and the birds were singing, and Patti was feeding the baby. I came staggering in and sat down and looked at them, and she looked at me, and I knew that if I didn't do something, she was going to leave. So I said, 'Patti, I'm a drunk and a drug addict, and I'm going to do something about it.' "

He beat his addictions, got back into shape, then rose to nationwide fame after he was cast as Sonny Crockett on Miami Vice, which ran from 1985 to 1990. He and D'Arbanville stayed together until 1986 when they split up for good.

Never miss a story — sign up for PEOPLE's free daily newsletter to stay up-to-date on the best of what PEOPLE has to offer​​, from celebrity news to compelling human interest stories.

D'Arbanville, who later married a New York City firefighter with whom she shares three children, also continued acting, most recently starring as Bobby Axelrod's (Damian Lewis) mother on the Showtime series Billions.

Johnson wed Kelley Phleger in 1999, and they've also welcomed three children together.

Before their breakup in 1986, Johnson gave another interview to PEOPLE, in which he explained that he and Patti enjoyed their unconventional and long-distance relationship — and that she wasn't worried about his intense Miami Vice groupies.