Sorting and grouping mail by Contact

I’ve been trying to sort my mail into groups by Contact so that I can see how much communication I have had with specific people over a course of time.

However, when I set my view to sort and group by the From field, it still creates multiple groups per contact such as:

And sometimes even various other groups. This is anything but useful to me.

How can I really sort my emails from a single contact together?