It is clear that many things are not properly known respect joints.
It is also clear that if you design your 4 stories tall building as a sway frame (we dimension that way RC frames in Spain) but you infill the building with attached masonries the building in the infilled stories won't be working as a sway frame at least for wind, but contrarily as a braced one.
In any case the sway frame design will ensure that strength and serviceability (take here lateral displacement) will be warranted till the conditions implied the design.
Proper detail of joints is better made at some works and countries than others.
It is clear that many of the details being used at least here in Spain maybe won't be warranting even the design assumptions, i.e., significant numbers of joints are being badly detailed.
I have a good photo of a exterior joint failed by the sway movement supported in one earthquake. External joints fail, for sure, in sway mode, so proper anchor of the rebar in the beams unto the edge columns is required whenever the area is assigned a high probability of having to stand a big earthquake.
Why are the joints not being detailed so exactingly? Well, in Spain there are scarce sites where the probability of a big earthquake is great, so most are familiar only with the far more ordinary ways of the less engineered details. Also, in fact the detail exigence is different of course for different seismic zones.
So if your work is in a high risk seismic zone, yes, you design for the earthquake and within it detail properly for the alternate sway movement, with proper anchors in the columns.
It is clear that damage to the building is to be expected under such earthquake.
So there may be more effective ways to deal with earthquakes, and specially for low structures, shearwalls for example. In any case, the choice may be not yours, sometimes happens.