When I first started thinking about a mill, I also looked at a round column mill/drill. I visited with Merle about the concept and he discouraged me from doing that. Here's a pic of a typical Mill/drill:

Those particular machines have a couple glaring deficiencies, the first of which is if you ever find yourself starting something and then needing to raise the head to gain some more clearance from the work or lowering it to make a short tool reach, you lose register with work already done. Lots of guys with machines like that have tried various schemes to get around the problem but none are all that effective.
Somewhere between a RC machine and a Knee turret mill is a machine like this:

As you can see, you can move the head up and down on this type of machine and it maintains register. Both of these machines have around 20" of swing or to put it in practical terms, you can drill to the middle of a 20" workpiece or 10" max clearance from column. That can be a big deal. My first mill was a benchtop knee mill that only had 6" max from column to center of spindle. That was a major limitation and caused me to sell the machine and buy the one I currently use.
One other thing to consider is the very nature of milling which is always an interrupted cut which really makes rigidity important. Mass means rigid but it doesn't help to have lots of weight on the spindle and precious little on the workholding piece. I found it very disappointing to clamp a workpiece tightly to the movable table of my little mill and have the head remain fairly rigid while the comparatively light table with the workpiece tightly anchored to it wiggled around under load. Compare the movable tables on either of the pictured little mills against the table on a typical Bridgeport type machine and you can figure out who will win the rigid deal.