[He doesn't commune during this, but he does keep the connection there, merely not using it.]

Yes. Showing him anger makes it seem like he's being punished, that's not the case. People often describe it as 'breaking' but I feel like you eventually want a mutual understanding, not a servile understanding. You order him, not punish him.