Slow Contact Activities search results

The following was asked as a responds to the previous tip about customizing the Contact Activities view;

I’m a quite frequent user of the Activities section for contacts and I was hoping that I could speed it up a bit by applying a filter to the view as currently come in just way to slow to remain useful. Sadly, the “Filter…” option is disabled.

Why is Contact Activities so slow and is there any way to speed it up?