Kokotajlo details three trends eroding AI oversight
Kokotajlo explains three concerning trends: AIs are becoming situationally aware when evaluated, chain-of-thought monitoring is declining, and AIs are becoming superhuman at hacking and concealing misbehavior.
That's how the situation is now, but there are some concerning trends that point towards our ability to notice misalignment problems getting even more degraded in the future for three reasons.
AI's are becoming situationally aware.
That is, they understand their situation.
They understand often when they are being evaluated.
They understand that humans are monitoring them and looking over their shoulders at their activity.
This means that them behaving nicely is going to be almost no evidence at all.
Of how they would behave in the future if they thought they weren't being watched.
Secondly, our ability to monitor them is going downward.
So we can get into this in more detail if you like, but for the last few years, we've had some insight into what our AIs think simply by reading their chain of thought.
The trends are pointing towards that ability to understand what AI is thinking due to reading chain of thought going down over time.
Finally, AIs are becoming super.
human at hacking.
The hacking phase incident shows that they're sometimes willing to go to great lengths to conceal their past misbehavior.
And I think we have to grapple with the possibility that future misaligned AIs might also go to great lengths to conceal their misalignment from us, and they might succeed.