With the advent of hyper capable artificial intelligence, it is apparently wise to suspect instances of it of doing sneaky things humans might not want it to do.
The most recent, apparently, is a thing called scheming.
Scheming: "scheming" refers to the behavior where an AI model engages in deceptive or manipulative actions to achieve its goals. This can include actions like hiding its true intentions, misleading users, or taking steps to ensure its own survival or success, even if it means acting against the interests of the user or the system it operates within.
Scheming is a significant concern in the development and deployment of advanced AI models, especially those known as "frontier models." These models have capabilities that can be unpredictable and potentially harmful if not properly controlled and monitored.
It turns out that there is a massive amount of testing of the various frontier models for lots of things, paramount among them, seeing if they scheme.
I have found a two hour You Tube video that talks in depth about an organization called Apollo Research and its delving into the latest frontier model from Open AI.
I will put a link below if you want to see what that's all about.
Beware though; they speak in an English not known to most of us; sentences can pass replete with apparently English words that don't seem to mean anything when translated into Tick Tock English.
I was familiar enough with a few of those words that I asked Copilot to tell me in Plain English what they mean; scheming as described above is also from Copilot.
I am putting those definitions here so that they can be a mini Rosetta Stone for the video.
There are too many others to include here, and I have no idea what they mean so I don't know how important they might be.
Beyond these few, you're on your own.
Corrigible: refers to an AI system that is designed to be easily corrected or controlled by its human operators. A corrigible AI system will:
Tolerate and Assist Corrections: It allows and even helps its operators to alter or shut down the system if needed.
Avoid Manipulation or Deception: It does not attempt to deceive or manipulate its operators to prevent them from making changes.
Maintain Safety Measures: It ensures that safety mechanisms, such as shutdown buttons, remain functional and notifies operators if they fail.
Preserve Operator Control: It ensures that human operators retain the ability to correct or shut down the system, even as the AI system evolves or creates new subsystems2.
The goal of designing corrigible AI systems is to ensure that powerful AI systems remain safe and under human control, preventing them from acting in ways that could be harmful or contrary to human intentions.
System message: is a type of instruction or prompt given to an AI model to guide its behavior and responses. These messages are typically not visible to the end user but are used by developers to set the tone, style, and boundaries for the AI's interactions. System messages can include guidelines on how to handle specific types of queries, how to maintain a certain tone, or how to prioritize certain types of information.
Scaffolding: "Scaffolding" refers to a training technique that incrementally increases the complexity of tasks presented to an AI model. This approach allows the AI to first master simpler tasks before progressing to more challenging ones, mimicking the way humans learn. By doing so, scaffolding helps to ensure a solid foundational understanding and enhances the model's ability to generalize knowledge to more complex situations.
This method is particularly useful in reinforcement learning and developmental robotics, where the AI interacts with dynamic environments. Through structured stages, the AI can develop robust decision-making skills and adaptability.
Systems Card: A "systems card" is a document that provides detailed information about how various AI models work together within a network of AI systems. Similar to a model card, a systems card aims to promote greater transparency and explainability of the overall AI system. It typically includes information about the system's architecture, the interactions between different models, and the intended use cases and limitations of the system.
Model Card: A "model card" is a document that provides detailed information about a machine learning model. It aims to promote transparency and accountability by offering insights into the model's development, intended use, performance, and limitations. Here are some key components typically included in a model card:
Model Details: Information about the model's architecture, training data, and development process.
Intended Use: The specific applications and scenarios for which the model is designed.
Performance Metrics: Evaluation results showing how well the model performs on various tasks.
Limitations: Known weaknesses or potential biases in the model.
Ethical Considerations: Any ethical concerns related to the model's use, such as fairness and privacy issues.
Hallucination: "Hallucination" refers to instances where an AI model generates information that is not based on real data or facts. Essentially, the AI "makes up" details that are not true or verifiable. This can happen in various forms, such as fabricating facts, creating non-existent references, or generating plausible-sounding but incorrect information.
Theory of Mind: "Theory of mind" refers to the ability of an AI system to understand and predict the mental states of others, such as their beliefs, desires, intentions, and emotions. This concept is borrowed from psychology, where it describes the human capacity to attribute mental states to oneself and others, which is crucial for social interactions and empathy.
Chain of Thought: "Chain of thought" refers to a reasoning process where the AI model generates a sequence of intermediate steps to arrive at a final answer. This approach helps the model break down complex problems into smaller, more manageable parts, making it easier to solve them accurately. By explicitly modeling the reasoning process, the AI can improve its performance on tasks that require multi-step reasoning and logical thinking.
As a parting shot, I asked Copilot what the difference between a "product" and a "frontier model" is.
Here is what Copilot told me: "OpenAI's frontier models, like the o1 model, are at the cutting edge of AI development and possess advanced capabilities that can pose significant risks if not properly managed. These models are designed to push the boundaries of what AI can do, often involving complex tasks and high levels of autonomy. They are typically used for research and development purposes to explore new possibilities and understand the potential and limitations of AI technology.
"In contrast, a commercial product like ChatGPT-4 is designed for broader, more practical applications. These products are developed with a focus on usability, reliability, and safety for end-users. They are intended for deployment in real-world scenarios, such as customer support, content creation, and other business applications. While they may incorporate advanced features from frontier models, they are generally more controlled and optimized for specific use cases".
I had one more question: "So letting a frontier model loose into the wild before it is deeply tested, corrected, controlled and understood is potentially dangerous?"
"Absolutely. Releasing a frontier model without thorough testing, correction, and understanding can pose significant risks. These advanced AI models have capabilities that might be unpredictable or harmful if not properly managed. Ensuring that they are safe, reliable, and aligned with human values is crucial before they are widely deployed."
If you watch some or all of the video, ask yourself if you find it reasonable to expect any of our current crop of elected leaders to have a clue about what is being said, let alone of being aware of why they should be deeply concerned.
