New to the Game. Should I move my BTC off Coinbase?

So I got convinced to buy into the BTC game a couple weeks back after regretting not getting into it earlier (like everyone here). My buddy suggested Coinbase and so far its been easy to use and I've had zero problems linking my bank, buying coins, converting them, etc.

However, I'm seeing some Coinbase hate on here and now wondering if I made an unwise move and if I should keep my small amount (around $500) of BTC on there or move it elsewhere? I plan to continue stacking sats and HODL'ing, just not sure if Coinbase is the best place to keep it.

I keep hearing about 'cold storage' which I'm assuming is one of those USB key devices and I've also heard about some 'better' exchanges such as Gemini. Any suggestions or should I just stick with Coinbase?

I've got much to learn on this subject and I am no expert by any means when it comes to cypto, just figured I get in now than later. Thanks for any pointers, this sub has been great the last couple weeks I've been here.