Deal – did I mess it up? Do I have to pay the $2000 TX fee?

To add to the other answers, transaction size (and cost) depends on the number of inputs. Even if mining rewards are sent to the same address, those transactions each generate one UTXO (Unspent Transaction Output).

To use the funds, you must use those UTXOs as input to a new transaction and provide some kind of secret (permission to use) for each UTXO individually. This causes the spending transaction size to increase with the number of UTXOs, and the more UTXOs you spend, the higher the transaction cost as a result.

Some companies (and individuals) in the Bitcoin space often aggregate large numbers of UTXOs into combined UTXOs whenever the fee rate is low. (If you’re not in a hurry) you can also do the same thing and combine it with RBF. Ideas suggested in other answers.