Stephen Colbert and wife Evelyn “Evie” McGee Colbert did not experience love at first sight.

During an appearance on SiriusXM’s Andy Cohen Live, the couple recalled their first impressions of one another after being married for over 30 years. Both Stephen and Evie grew up in Charleston, South Carolina and met when they were young, but they didn’t immediately start dating.

“We met several times,” Evie, 61, said.

“We did, yeah. The first couple times, it did not take,” Stephen, 60, jokingly replied. “We first met in college. A friend introduced each other. A friend from Charleston introduced us at a party and I was chemically impaired and Evie was a distant frost queen on a mountain top.”

Stephen Colbert and Evie McGee-Colbert.ROBYN BECK/AFP/Getty

When host Andy Cohen asked if Stephen liked Evie’s cold energy, he said no.

“Not the first time,” Stephen admitted. “The first time, I was like, ‘This is going nowhere.’ Literally, I think it was a 30-second conversation.”

After Stephen declined to say what “chemically impaired” him, Cohen questioned what Evie thought about her hubby when she first met him.

“An immature, skinny boy,” she put simply, laughing.

In the past, Stephen and Evie shared that their true first introduction was in the lobby of a theater, where neither of them realized they had grown up together. In 2016, Stephen described that special moment.

“Her. That’s your wife. You’re going to marry her,” he said he thought to himself.

They wed in October 1993 and went on to have three children together: daughter Madeline and sons Peter and John.

“Stephen and his family are very happily entrenched,” a local New Jersey neighborhood source told PEOPLE of the Colbert family in 2015. “You can always see him with his family. Family and community have always been important to him.”

Stephen Colbert and Evie McGee-Colbert.Nathan Howard/Getty

During COVID, when restrictions were put on filming his show, The Late Show with Stephen Colbert, Stephen kept it running with the help of Evie. In 2021, he was honored at the Emmy Awards for his 2020 special, “Stephen Colbert’s Election Night 2020,” and he made sure to thank his wife.

“I’m going to call my wife to celebrate as soon as I get through all this because she was a huge part of this past year and this night, too,” he said. “You know, she’s an absolute rock and I couldn’t do this without her and I’m so glad — the only good thing that came out of COVID is that she got to be part of the show.”

More recently, the couple released their joint cookbook Does This Taste Funny?, which came out in September 2024.

“It’s a little peek into our family,” Evelyn told PEOPLE at the time. “It’s just a teeny little bit of hanging out with us in the kitchen.”

The cookbook celebrates the tastes of their hometown in coastal South Carolina and includes recipes for dishes such as red rice and crab cakes.

SiriusXM’s Andy Cohen Live airs Monday through Friday at 10 a.m. ET on Radio Andy (Ch. 102) and anytime on the SiriusXM app.