We don't. Or, at least, most of us don't.
Rape is a little tricky, because often the *only* evidence that what happened was rape is the testimony of one of the two participants in the act, the other of whom is just as vehemently denying that it was rape. With what could be considered "classic" stranger rape (eg a man attacking a woman in a dark alley), the material evidence that sex occurred is usually enough to reasonably prove that the victim was raped, but with date rape, the material evidence cannot distinguish between voluntary (if perhaps rough) sex and rape. And if it was rape by coersion, threats of force, or intoxication, there often aren't even any signs that whatever sex occurred wasn't as gentle as normal sex.
I agree that it's a bad thing when someone is falsely convicted of rape. Can you also agree that it's a bad thing when someone gets away with rape, without being punished? If so, then do you agree that perhaps the standards for convicting someone of rape need to be at least a little different, in some ways, than the standards for convicting someone of, say, theft? I'm not saying abandon the due process of law, just... adjust things a little bit, to account for the nature of the crime.