Removing leaves

I hedge my bets and remove some in total that are shading later on in the year but say halve them early on, I look at how big the plant is and judge (rightly or wronly) whether it will suffer by being shorn. Not saying that's the right way though. I have the same dilemma with taking off the lower leaves.