Perhaps annoying, but it wasn't as though you were still sleeping and they awakened you.  Turn the internal dead bolt and flip over the little metal retainer before you go to sleep at nite and they can knock till they're blue in the face, can't get into the room.  Ignore the knock, and you won't have to lambast anyone.