grammar - ～られる connotations

I was looking through A Dictionary of Basic Japanese Grammar recently and found this as an example under ni (3):

一男{かずお}は友達に手紙を読まれた。
Kazuo's friend read his (= Kazuo's) letter (and Kazuo was unhappy).

And another under rareru (1):

私は弟にケーキを食べられた。
Lit. I got my cake eaten by my younger brother. (= My younger brother ate my cake (and I was unhappy).)

I found this to be a bit strange, because I can't see why the subjects necessarily become unhappy even if the events were not of their control.

After looking around a bit, it appears some categorise one of the uses of ～られる as a "suffering passive" form, with some adverse implication for the subject unto which the action has been performed.

However there are others that say that such a thing doesn't really exist and that it's context specific. This is also explained partially in the Dictionary under rareru (1), note 7.

What connotations are there with ～られる? And are there ways to discern them without the use of context?

There has been an almost identical question posed on Reddit, but I feel that the answers didn't really explain where this general view comes from.

Answer

subject unto which the action has been performed

I think this is a useful way to look at it. Why is the sentence even structured so that the action is performed unto another person?

Why am I the subject?

If it's just that my friend read my letter and I have no problem with it, the natural way of stating that is with my friend as the subject:

友達が手紙を読んだ

Only when I perceive that to be done unto me, like some kind of encroachment, will I structure the sentence with me as the subject:

(私が) 友達に手紙を読まれた

Same goes for the cake example. The neutral statement, naturally, takes my brother as the subject (弟がケーキを食べた). When I feel intrusion, I take over as the subject ((私が)弟にケーキを食べられた). (Whether it's my cake or some communal cake.)

Sometimes I have to be the subject

Then, there are many situations when it is perfectly natural for me to be the subject, even in the most neutral case. My sister questioned me (妹に質問された), or my husband asked me for help (夫に手伝いを頼まれた). But it could still mean that I feel burdened. You can't tell from just the sentence. This is probably why it seems context specific.

---

There is no sure-fire way to discern between them in every case, but a good way to tell is when the subject is someone that doesn't have to be.