I wouldn't ask your husband to intervene - tempting as that is.
How confident are you is my question? Australians don't get subtlety. If you are going to respond it has to be loud, decisive and obvious.
Next time he says it, I would reply, in a very loud voice,
"You're not one of those sexual deviants are you George? I've heard about them, but I've never seen one before, although I've heard they hand around in the sand dunes. Just in case you were wondering, if I was going to have an extra marital affair, it wouldn't be with an old ugly fucker like you"
Take it as far as you feel confident, but the brasher the better.
Or - report it if you don't feel confident handling it yourself.