Debian officially provides two ARM architectures, armel and armhf.
The armel architecture supports the ARMv4 instruction set.
This architecture handles floating-point computation in a compatibility mode which it slows performance but allows compatibility with code written for processors without floating point units.
So you can use the armel architecture to build high compatible systems.
The armhf architecture supports ARMv7 platform, and more, it adds direct hardware floating-point support.
This means the armhf architecture is faster than the armel one, but it lacks the compatibility with the old architectures.
On memetic you can see some benchmark run on Raspberry Pi with the two architectures. Take a look to Raspbian Benchmarking – armel vs armhf.
If I want to build a system running on a big number of already installed board I have to consider using armel architecture (since some installation could use old ARM boards).
If I have to build a totally new system, working with audio and video, with high probability I'll choose the armhf architecture.