VFR pilots must maintain 1000' of clearance (1000' above, or 1' above the ground and 1000' horizontal distance) from a densely populated area.  In a sparsely populated area or over water it is 500' of distance from any structure, person, vehicle or vessel.  The FAA is vague about the definitions of "densely" and "sparsely".  I have read of guys getting into trouble for being closer than 1000' to 2 people in a canoe, or a few cars on a highway.  Not exactly what I think of as "densely populated".  These rules don't apply when taking off or landing or during an emergency.  I'm pretty sure drone pilots fall under the same rules.