First, here, it would need to be posted - No Trespassing . . . . . then take pic's of the signage. Then talk to party and give them x days to remove. Then involve Sheriff. That's the way it would have to go down, here. I too would get it surveyed and fenced off. In the grand seam of things, it doesn't cost that much.

ETA: and yes, as suggested, move and widen the driveway and farm that strip of land.