Punch cards were used as both an input and output/storage media, whereas mag tape was a output/storage media only.

In the days of punch cards, when you wanted to enter a program into a mainframe, you sat down at Ye Olde 029 Punch, put in a deck of blank cards and started typing. Here's a pic of a 029 punch:

http://www.lunch.org.uk/wiki/_media/mainframes:11_ibm-029-keypunch.jpg

Once you were done punching a deck, you'd drop it into the card reader:

http://www.columbia.edu/acis/history/2501.html

That reader, BTW, was the "user-available" card reader. In many academic environments, there was one 25xx reader like this one available to students in the same room as the punches, and then for more complicated jobs, you'd hand your deck (with several rubber bands wrapping it, or in a tray if it was a big job) to the system operator, who would put it on the stack with all other incoming jobs.

Magnetic tape when through several iterations before becoming the 9-track stuff most people have seen in movies or films. The earliest were 7-track and much less dense than the final iterations of reel-to-reel tapes. Here's the one for the IBM 701 machines from the 50's:

http://www-03.ibm.com/ibm/history/exhibits/701/701_1415bx27.html

Punched cards were how we humans put data/programs into the machines. There was also punched tape, from KSR/ASR machines, like the ASR-33 that was a popular unit on minicomputers in the 70's:

http://www.vintage-computer.com/asr33.shtml

See that part of the unit on the left? That's where the paper tape would spool off the back. The ASR usually had a paper tape punch and reader, so it was a true "all in one" terminal. You could just type onto the 9" wide yellow toilet-like paper coming off the roll, or punch a paper tape or read one. The paper tape readers were very slow by comparison to card readers - like 110 baud. Punched card readers could read a deck of cards 2' high in less than a minute. 

Paper tape was more convenient than punched cards because it was more dense - ie, you could put up a pretty large program in less mass of paper than the 029 cards. You also didn't need to worry about dropping your deck, as with punched cards. If you dropped a deck of punch cards, you had to find a card sorter - IF you had put sequence numbers on your deck. If you've ever programmed in COBOL or FORTRAN, you'll remember that columns 72 through 80 were not read by the compiler. That's because the cards would be sequence-numbered in those columns - so that if you dropped your deck, the card sorter could re-sort your deck.

Cards allowed you to do inserts into the deck. With punched paper tape, you could not edit your tape easily - a few of us because proficient at cutting the tape with an eXacto knife, punching a new bit of tape and splicing it together, but most people didn't do that.

I don't miss any of them, really. I spent only one semester in 1980 dealing with punched cards, years dealing with punched tape in the 70's and most of my career dealing with 9-track mag tapes on mini's and mainframes. The one thing I can say for 9-track mag tape is that it is drop-dead reliable, even after tapes have been in storage for 10 to 25 years.

If given a choice between "slab of rock, hammer+chisel" and "029 card punch" -- I'd have to think for a bit to make my choice. If the choice was for punched paper tape, I'd hesitate if given an air chisel and the slab of stone.