Should I not see the world at all?

Sri Ramana Maharshi

Why did the Self manifest as this miserable world?

In order that you might seek it. Your eyes cannot see themselves. Place a mirror before them and they see themselves. Similarly with the creation. "See yourself first and then see the whole world as the Self."

So it amounts to this - that I should always look within.

Yes.

Should I not see the world at all?

You are not instructed to shut your eyes from the world. You are only to "see yourself first and then see the whole world as the Self." If you consider yourself as the body the world appears to be external. If you are the Self the world appears as Brahman.