Here's a suggestion:

Try positioning the tom reversed. That is, swap the l-arm/clip around. Instead of having the l-arm/clip enter from the left, have it enter from the right. That will push the center of balance more towards the center of the bass drum. It will also give you a better positioning of the tom off the snare. It looks odd, but it will play a bit better for you. This will definitely reduce the amount of mass pushing to the left and will keep the bass more centered.