mempool – What makes it difficult to prevent transactions where the value transferred is less than the fee paid?

It’s hard to wrap your head around the fact that someone could end up paying more in fees than the value transferred (although some wallets can handle it at that level), but this makes for a well-funded DDoS vector.

Is there a technical reason why Bitcoin doesn’t prevent something like this in Bitcoin Core at a consensus level or at least as one of its policy rules?

There is a minimal dust relay option with completely arbitrary defaults (very inefficient and inconvenient to use). Instead it may need to be dynamic with an on/off option.