We live in 21st century. There are many good Dhamma books and suttas available some of which are for free. Why can't a person read a lot of good books and practice based on them? I understand that traditionally, centuries ago, there were NO books and no internet available so the only way one could learn was from a Teacher or a monk. Today the situation is different. It is possible to know more than a teacher by having read Nikayas, Abhidhamma, Visudhimagga, and many good meditation books. Regarding "personalized instructions". Typically on the interviews in popular methods the suggestions were all the same "just observe!" or "don't forget to label that". It seems rare to find someone who has different suggestions for different occasions and who can tweak instructions specifically for you rather than repeat the standard teaching in that Center. These are not hidden instructions and one can learn them, and more from books.
While it is true that routinely one might be told to "just keep going", the key thing about a competent real teacher is that s/he can tell when "just keep going" isn't going to work and give appropriate advice/correction.
I also have a hunch that progress in meditation does NOT equal knowledge of more theory. So the meditation master doesn't have to know more than a newbie, the difference is in the skill at applying the basics.
I don't think theory is so important in any case (if by theory you mean being able to quote lots of suttas and/or commentaries). It's experience, as you say below.
It is like in a case of a boxer. A beginner boxer who later becomes an advanced boxer can know exactly the same moves as before, but the advance boxer is better because of having practiced those moves hundreds of thousand times more than a beginner and can use them better.
And, again, the key thing that's obvious with competent teachers is that they have seen what the student is describing, both for themselves, and with a number of other students, and can therefore give good advice.
- I'm noticing X, that seems odd...
Do you notice Y as well?
Yes (Hmm, how did he know that? Does he read minds?).
OK, that's normal, try doing Z instead...
I don't have the experience to give advice about what is or isn't sensible to do without some backup from teachers (at least occasionally). Certainly, one can now get quite a lot of good advice by listening to retreat talks and discussions from the internet, where students ask typical questions, which may well turn out to be your question... I find that sort of discussion very helpful, but I'm not sure how helpful I'd find it if I hadn't had previous personal instruction...