Got the 4640 back today and all is fixed. Oil leak was the mounting stud on the oil filter housing, didn't have an sealer on the threads and in goes into the oil gallery.
The rough idling was solved by adding a fuel check valve in the return line coming out of the pump. Seems the higher serial number tractors (engines) have this from factory, earlier ones didn't. Neither service manager or pump shop mechanic had ever ran into this problem, or if they did they didn't fix it. I'm guessing the reman pump is a later one and requires some back pressure on the return. Whatever it is the tractor idles as smooth as any I've ever been around at all speeds.
Thanks to all those who replied to my previous posts.