It depends upon the check valve design. 

Check valves like those with just a ball and a spring, all totally enclosed in the housing, will always cause the opening pressure drop while there is flow through the valve (plus whatever friction loss due to the fluid passing through the valve passages) because it takes the differential pressure on the inlet and outlet sides of the ball to keep the spring deflected.

Other check valves like diaphragm valves in no-drip sprayer nozzles which have the back side of the diaphragm vented to the outside of the body will stay open once the pressure on the fluid side of the diaphragm is high enough to move overcome the spring closing pressure.  As long as there is enough back pressure from beyond the valve on the fluid side of the diapgragm compared to the atmospheric pressure on the other side of the diaphragm, the diaphragm will stay off the seat so the only additional back pressure would be due to friction loss due to flow through the valve body.