Susan Campbell talks about staying on “your side of the net” – meaning, “I feel angry” rather than “you make me angry.”
In that simple example she captures an essential to self-empowerment and I believe also releasing “who’s to blame?” as how we approach we-space emotions.
Humans are not certainly Newtonian objects acted upon without agency. Cause/effect as you note is an incredibly crude way of being with emotional complexity, even hurts and harms that seem on the surface to be “caused” by another person’s actions and attitudes.
I like your example of the same words and energy directed at us at different times, or directed at other people, evoking a different reaction. On the face of it that takes it out of simple cause/effect.
Rick