A friend of mine used to do that.  Downside is reaching in a hole and getting a snapping turtle.  He got some that were over 30 pounds and they're quite a tussle.  You aren't going to be taken down the river with a 30 or 40 pound catfish but you will have a wrestling match on your hands.

He did this on the Mississippi in the Davenport, Iowa area.