No, it hasn't to do with the cache, normally. The CPU is the one that processes, the RAM memory is the guy that holds the info that the CPU uses at some point, and the Hard Disk is the guy who holds the info permanently, as the RAM erases itself when the computer is off. The way between the CPU and RAM is expensive in time, and the cache is some sort of little ram that is close to the CPU. The most used things are left there to avoid to do al the way to the RAM, it's efficient.
Programs like defrag, solve fragmentation problems of the latter, the hard disk. Fragmentation problem is worst when out of 100 bits of Info, 90 occupied, and 10 are free. You have to place now 5 bits, but that 10 free bits are scattered, and there are not 5 in a row. So to place and later read that 5 bits, you have to jump to get the info, and that's slower.
In the RAM memory, you have the same problem. As time passes, you load things into the RAM which gets full. To make things worse this OS like Windows are not efficient, and after using the info, it doesn't bother to clean it. So the more time passes, the more degenerated a system is, inevitably.
This is just one of the houndred of reasons that can make a computer slower. Then you have a suspicious little program that loads in your favourite pórn site (which are crowded with those little programs), your protection software asks the program, who are you?, the little program answers whatever gibberish, the protection software asks again who are you again and again and you have a block. This is one of the million things that make that window not responding, and is so frustrating

So that's why knowing all this is useless woof

So you can live perfectly with a beer, the basic cleaning programs, and the turn off and on tactic. Also note that those websites with ring tones, tits, naked women etc, are loaded of these little bugger programs.