Your general health is ultimately more important than a big spell collection. Health potions are pretty rare, so concentrate on collecting them when you encounter a vendor. They'll come in handy when you're in big battles.
You have a limited inventory. Stockpile your excess inventory in places on the map, and be sure to mark them so you can come back later and replenish your supplies.