JSimon you pretty much nailed all the things I've adjusted.
The only thing I've not done - yet - was get rid of those shoulder straps. As much as I dislike them, I haven't quite found the piece that I want to use instead of them, so they stay on. Wrestling with them in the dark is a good way to get my blood going for the walk.
I don't use the lineman's belt, I made my own which is significantly lighter and easier to adjust (for me anyway). I changed the connection point also, and it works in tandum with my climbing foot platform. I like mine better becuase the connection point is much lower, but still secure to the tree, so I can lift my GM (62AMO) over the point and still shoot easily on the other side.
It's still a work in progress, but like you said, the moment you start "changing" things, there's no going back. So far I keep finding little things to adjust, but like any experiment, I only adjust one at a time until I find what works for me.