Quick recap of a normal 120V 1-ph circuit: Black is hot; white is neutral; green is ground. Neutral is bonded to ground at the service drop only. When a circuit is in use, the neutral carries current back to the source (transformer). The grounding circuit is to protect the user, providing a low-impedance path back to the source through the earth. In normal use, the grounding conductors should register 0 Volts to ground.

It is possible to use the grounding conductors as ad hoc neutrals, though this is a very bad idea. 240V motors will run without a neutral...but if the appliance/application also has a 120V requirement (light, control panel, etc.) it also needs a neutral wire. Do not use a ground as a neutral.

My guess is that somewhere, someone has done something similar at your place. The metal receptacle box that you are using is getting voltage on the grounding wire. This is unsafe and going to PVC would only mask a serious problem. Tracking these things down requires some sleuthing, a voltmeter, and maybe another person to flip breakers while you watch the voltage.