Only sometimes. As an American who's not especially well-traveled, I can easily understand the accents I hear on TV and in many movies, which I gather is mostly upper class educated people. I can understand every word of The Crown, for instance.
But there are accents that might as well be another language, in which I catch only the occasional word. For those, I turn on subtitles, and if there aren't any, I stop watching.