Am I selfish that I don't want to have sex till I'm cleared?

No it's not selfish, you should definitely wait until you have the sperm count done and it is negative. As other's have suggested do other things..you need to ejaculate as much as you can anyway to clear the stored sperm.