Horse with tongue over bit?

If you are not showing pleasure there is no reason you have to use a bit I guess. One thing to check though is if your bit is tight enough. There should be one wrinkle above the bit on each side of his mouth.