Peter Griffin, who wrote "Theory of Blackjack", has a section titled 'World's Worst Blackjack Player' where he quantifies the penalty a player will face for certain ill-begotten plays. The penalty for never doubling down is 1.6%. So if you are begin as a perfect basic strategy player where the casino typically has a .5% edge over you, then by never doubling down, you have just quadrupled the casino's edge against you!

If you are counting cards, there are counts where it is correct to do as you say, to hit rather than double. But without this knowledge, your best chances are to follow Basic to the letter.