If permission is granted, then adverse possession does not apply.

But a prescriptive easement, as mentioned by Russ above, would be a concern.

Suggestion: Rent the access, annually, for "$1 and other considerations," and have it in writing.