Your recluse neighbor will care the minute he sees that Bobcat delivered and ready to almost destroy his fence. Get your own survey and put it with the paperwork you will need when you sell your property. Send Mr. Recluse a certified letter (sounds stupid I know, he IS right next door) to prove he got it, outlining your problem and requesting his help to solve it. If he ignores it in 30 days, start work. If you need permits, get your local municipality involved. Maybe he'll open the door for them.
We have the same problem. Years ago when the neighbors moved in, they put up a stockade fence a few inches away and parallel to our chain link, then planted Rose O'Sharon trees along their side of the fence. Decades later, the tree roots spread to between the fences and is now destroying their fence. They're mad that we 'allowed' the trees on our side of their fence (their side of our fence) to get big.
Our garden is pretty thick on our side so herbicides are out of the question. When we told them they were welcome to come in our yard to cut down the trees between the fences they said they hoped we would take care of that or they'd have to take down their fence to do it properly on their side. The only way to totally remove them is to dig out the roots and the way they spread, we will be weeding out tiny Rose O'Sharons forever. Besides, they're pretty and the hummingbirds love them.