CEO of OpenAI, Sam Altman:

I believe there are two ways AI progress can go terribly wrong that we should avoid. First, we may lose control of the future to AI.

The risk is that it moves so fast that people can no longer follow what is happening or intervene when necessary. Of course it would be terrible.

Industry must not accept too much technology risk just because the benefits are too great and they feel too important to slow down.