I had this same problem with a few necklaces I made with flat rectangular stones. My only suggestion is to make it a choker style, and then maybe it'll be flat? I'm not sure...I wear the one that seems to twist a bit often, and it doesn't bother me.
The on;y other thing I can think is maybe using a heavy pendant, and then maybe the other stones will fall into line on the sides?