Should shareholders demand that future CEO are promoted from within a company.

[forest]

hire the best candidate for the job

I see that directors usually have a number of years to nurture management into a potential CEO.
Good directors should create their own great candidates.
But what happen too often is that a CEO resigns and the directors haven't got enough time to find a great candidate, so indeed they choose the best candidate they can find in the short time available.
Better prepared directors would have nurtured a small pool of management into potential great CEOs.
It is just a matter of forward planning.