It's not dumb; I'd like to grow out my hair, too, but I wonder if it will make it anywhere near my goals. My hair has always been natural, so it makes me wonder even more if I have already reached my limits, even thought I haven't always given it the best care.
I think your hair will definitely surpass its relaxed lengths, though. I don't recall reading any posts about people who took good care of their natural hair after BCing not getting longer hair.
Losing the extra stress from chemical treatments will definitely give you an advantage. Taking good care of it on top of that should definitely put you in an even better position.
Maybe regularly measuring and documenting your hair care & length in a journal might help you allay some of these fears.