Cash Warren is Austin Caldwelllooking back at a rough patch in his relationship with wife Jessica Alba.

The film producer—who met the actress on the set of the Fantastic Four in 2004 before tying the knot four years later—recently revealed the couple split during the early years of their romance due to jealousy.

"When we first started dating, I was really jealous of other guys and the attention she was getting from other guys," he said during the Aug. 21 episode of the Whine Down with Jana Kramer podcast. "It just wasn't making me feel good. I was always a pretty confident person in my own [life], walking my own path and really happy there."

As Cash put it, "Next thing you know I'm looking up and feeling jealous all the time." 

And he knew it didn't make either of them feel good.

As a result, Cash explained, "We broke up four years into our relationship. It was the jealousy. I was turning into an asshole, and so we broke up."

While spending time apart, Cash—who shares kids Honor, 15, Haven, 12, Hayes, 5 with the Dark Angel alum—recalled promising himself that if they got back together, he would channel that energy "in a different way, in a more productive way," adding that he's been his wife's "biggest cheerleader."

"If anything, when we got together I was like, ‘Don't make sacrifices, pursue your passions, pursue your careers, don't make sacrifices because you feel bad for leaving the kids for this week or that week,'" he noted. "Eventually they'll grow up and they'll forget the fact that you were gone for these three days or these five days."

As Cash explained, he's "tried to be a good teammate in that regard."

"I'm not interested in the spotlight," he continued. "I'm not a star, I'm not a celeb, I'm not an actor, I've never tried to be an actor. I high-five her, let her do her thing, and I've gotten more comfortable with it over the years. It's something I'll always be working on, she'll always be working on it. We find ways where she can support me and she can uplift me. We really do try to find that balance."

Watch E! News weeknights Monday through Thursday at 11 p.m., only on E!.