Paul Newman is considered to be one of the great sex symbols of all time, but in his posthumous memoir, the actor admitted that women didn't always think so highly of him.

In the mid-'80s, the legendary actor began compiling an oral history about his life and career, conducting interviews with friends, family, and himself where his only rule of thumb was they had to be “completely honest.” Those long-last recordings were eventually recovered and compiled in a new book, The Extraordinary Life of An Ordinary Man: A Memoir, due out later this month. In the memoir, per People, Newman examines his relationship to his own sexuality and his feeling that, despite being viewed as a heartthrob by women everywhere thanks to films like Cool Hand Luke, The Hustler, and Butch Cassidy and the Sundance Kid, he didn't actually feel sexy until he met his wife, Joanne Woodward. He explained, “Joanne gave birth to a sexual creature. We left a trail of lust all over the place. Hotels and public parks and Hertz Rent-A-Cars.”

The Oscar winner goes on to explain how insecure he was growing up in Shaker Heights, Ohio, especially when it came to girls. Newman was so petite as an adolescent that he had to get permission to play on the high school football team, an experience that shook his confidence. “Girls thought I was a joke. A happy buffoon,” he said. But everything changed once he met Woodward. He explained, “I went from being not much of a sexual threat to something else entirely.”

Newman and Woodward first met in 1953 when they were both understudies in the Broadway play Picnic. The Sting star was married at the time to his first wife, Jackie Witte, with whom he had three young kids, Scott, Susan, and Stephanie. But he was unable to deny his attraction to Woodward and the pair began a tumultuous affair that he describes in the book as "brutal in my detachment from my family." Eventually, he would divorce Witte in 1958 and quickly wed Woodward. In the memoir, he describes returning to their new Beverly Hills home one night to discover that she had fixed up a room off the master bedroom with a double bed she bought from a thrift shop and a fresh coat of paint. “'I call it the Fuck Hut,' she said proudly. It had been done with such affection and delight. Even if my kids came over, we'd go into the Fuck Hut several nights a week and just be intimate and noisy and ribald,” Newman wrote. The couple would go on to have three daughters of their own, Nell, Melissa, and Claire, and move to Westport, Connecticut.

But while their romance was one for the history books, Newman and Woodward still faced plenty of ups and downs in their relationship, often due to the actor's heavy drinking. He recalls in the book, “Joanne and I still drive each other crazy in different ways. But all the misdemeanors, the betrayals, the difficulties have kind of evened themselves out over the years.” Their daughter Claire also told the outlet that it's true her parents “fought and it could be dramatic, but they also fought really hard to stay together. They didn't walk. There were times it was pretty close but they worked hard at it. Ultimately they came together.” She added that this memoir gives her late father the opportunity to let others see him how he saw himself, concluding, “He was doing this for us so he could clear up the fairy tale and tell the real story.”