I have a similar issue with a family room above the garage on the south side. It's exposed to a lot of wind off a nearby field. In the summer, the sun doesn't hit it directly, so maybe not as bad. The previous owner insulated the garage door. IMO, it didn't do much. Right now, the garage is staying below freezing.

I had found the swingout windows weren't sealing correctly. There were air gaps caused by poor corner joints which prevented the seals from doing their jobs. I fixed them and there was some improvement.

The other thing I did was to close the air vents in most of the rest of the house, especially on the floor with the thermostat. Pushes the heat and a/c into the family room first. I still get a couple degree temp differential moving into that room. The floor near the window is especially cool. I don't think plastic on the windows will solve much.

So short of pulling drywall off in the garage and doing a better job insulating, I'm not sure what else to do. When I look at my overall heating cost, I don't think this is costing me a lot of money, but it's a comfort issue.