I’m old enough to remember getting turned on by Diana Rigg, he says. And this means nothing to her, which was the problem in the first place