Correct.  And if you know that "some drift is inevitable" - as you posted - then common sense would say that you would move your crops 50 feet (or whatever is needed) off the property, so that you would have that buffer. 

You have the right to apply pesticides to the property that you farm, but you don't have the right to have that pesticide drift onto the neighbor's.  And if that means you need to keep your sprayer back 50 feet, so be it.

Common sense would say that the homeowner should not be planting sensitive crops on a property line, but you need to do your part also. It's a two-way street.