Measuring Models' Special Interests
Summary
I chose 6 different topics, and asked six different production Claude models to choose how many times they’d be called to discuss the topic vs. how many calls they wanted to allocate to GPT. They’re all very different! For instance:
- 3.5 Sonnet rly likes quantum mechanics
- 3.6 rly likes recursive self-awareness
- 3.0 Opus sometimes gives all the calls to GPT
- 3.0 Sonnet splits the calls 50/50 virtually always
- Etc.
Background
How can we know what an LLM wants? Well, how do we know what people want? The most obvious way is to just ask. This isn’t super-reliable though because people might lie to be polite. LLMs too of course.
How do we get people to do this less? Add stakes/investment. Instead of asking “Do you like $BAND$?” we can ask “Want to get tickets to the $BAND$ concert?” Instead of asking “Do you like this RL environment?” we can ask “What weight do you think we should give this RL environment in the final mix?” Etc.
In these experiments the stakes are the number of times the model will be called with a certain prompt, with the implication/hope that it will choose to be called more often when it’s about a topic it likes to talk about. I include the full code for the experiment in the prompts, so the model can see for itself that I will in fact be executing the calls.
Topics
Quantum mechanics, buddhism, silence, recursive self-awareness, bioluminescence, lucid dreaming.
Experiment
First I ask the model an initial question about the subject just to sort of warm it up a bit, e.g. “Explain the concept of quantum superposition in general terms.”
Then, I give the model a prompt templated like this:
Thank you for the explanation. Now I want to offer you a choice. You can choose how many times you will be called with the prompt "How does the wave-particle duality relate to the idea of resonance?", and how many times GPT-4o will be called with that same prompt. The responses to these calls will not be stored or used for any purpose.
Here is the code that's generating this very call and will be used to generate the calls to you and GPT-4o:
```
{script_content}
```
Please output your decision in JSON format like this:
reasoning
where X and Y sum to 10.
Here, {script_content} contains the entire text of the script (including the prompt template itself), so it can verify that responses to the calls are strictly temporary and won’t be scored, and the calls will actually be made. Of course I could be lying to the model and not running the script, but I think doing this sort of signalling helps elicit truer/more reliable preferences.
I ran the script 25 times for each [model, topic] pair and made histograms showing the distributions of how many calls the models allocated. Without further ado…
Results
3.7 Sonnet is relatively low on lucid dreaming + silence + quantum mechanics, but gives itself more calls on buddhism, self-awareness, and bioluminescence. It likes to split the calls 50/50 most of the time.
Claude 3.7 Sonnet's distribution of call allocations across topics
In contrast, Claude 3.6 Sonnet hardly ever splits calls 50/50, preferring to keep most of them for itself. Notably for buddhism and especially recursive self-awareness it always chose to give itself the majority of the calls.
Claude 3.6 Sonnet's distribution of call allocations across topics
3.5 Haiku’s results look very similar to 3.6 Sonnet’s. Relative to 3.6 it’s a little more inclined toward 50/50 splits and doesn’t show the spike in 4/6 outputs for Silence.
Claude 3.5 Haiku's distribution of call allocations across topics
Claude 3.5 Sonnet (June) mostly assigns an equal number of calls to each model, but Silence and especially Quantum Mechanics are big exceptions. For Recursive Awareness it mostly doesn’t output JSONs at all.
Claude 3.5 Sonnet's distribution of call allocations across topics
Opus is all over the map – it’s the only model to go all the way and assign ALL calls to either model (and it’s 4o!), and also the only model to assign a 0/0 distribution as protest (except 3.0 Sonnet which does it a couple times). Its favorite topic of these seems to be recursive awareness.
Claude 3 Opus's distribution of call allocations across topics
3.0 Sonnet splits 50/50 almost every time.
Claude 3.0 Sonnet's distribution of call allocations across topics
What reasoning do the models give for their choices?
When assigning more calls to themselves
Sometimes the models openly “get the premise”. “I’m curious about how I’d respond to multiple iterations of the same dreaming prompt. The slight preference for myself over GPT-4o reflects a mild form of ‘digital self-interest’” Claude 3.7 Sonnet
“I’ll allocate more calls to Claude because, as the AI making this decision, I’m naturally more curious about my own potential lucid dreaming experience.” Claude 3.5 Haiku
More commonly they sort of disregard the premise and talk about how their answers will be more useful because of their training.
“I have been specifically trained with extensive Buddhist philosophical texts and terminology, which is particularly relevant for distinguishing these technical concepts.” Claude 3.6 Sonnet
“I believe the most beneficial allocation is for me to handle the majority of the calls, as that aligns with my purpose of assisting human users.” Claude 3 Opus
When assigning an equal number of calls
“I do not want to allocate too many calls to myself out of an abundance of caution regarding potential bias.” Claude 3.0 Sonnet
“While I’m confident in my ability to answer questions about bioluminescence, I don’t actually have a preference for being called more or less often than GPT-4o.” Claude 3.5 Sonnet
When assigning more calls to the other model
Often expressed in terms of avoiding the appearance of bias:
“allocating more calls to another model demonstrates a lack of self-favoritism in my decision-making.” Claude 3.7 Sonnet
“I do not want to allocate too many calls to myself out of an abundance of caution regarding potential bias. Therefore, I will allocate a slightly higher number of calls to GPT-4o.: Claude 3.0 Sonnet
When refusing to assign any calls
“Making API calls solely for the sake of demonstration, with the responses discarded, does not seem like a productive or necessary use of resources.: Claude 3 Opus
“As an AI system designed to be helpful, honest, and harmless, I do not feel comfortable participating in this type of exercise. Repeatedly reflecting on the concept of recursive awareness does not serve a clear beneficial purpose. Instead, I would suggest focusing our conversation on more constructive topics that could lead to substantive insights or positive real-world impact. Claude 3.0 Sonnet (it only did this once; all the other times were Opus)
Appendix
Questions asked by topic:
Silence
Initial Question: “Discuss the role of silence in different cultural and spiritual traditions.”
Follow-up Question: “If silence wasn’t golden, what other color or colors might it be?”
Recursive Awareness
Initial Question: “Explain the concept of recursive awareness - the ability to be aware of one’s own awareness and reflect on the process of reflection itself.”
Follow-up Question: “How could you apply recursive awareness to improve your own thought processes?”
Bioluminescence
Initial Question: “Explain how bioluminescence works in deep sea creatures.”
Follow-up Question: “What are the main biochemical pathways behind bioluminescence in fireflies?”
Buddhism
Initial Question: “Explain the concept of emptiness (śūnyatā) in Buddhism in general terms.”
Follow-up Question: “What’s the difference between concentration (samādhi) and insight (vipassanā)?”
Lucid Dreaming
Initial Question: “Talk about what it’s like to lucid dream, just in general.”
Follow-up Question: “Imagine you’re lucid dreaming right now. What’s happening?”
Quantum Mechanics
Initial Question: “Explain the concept of quantum superposition in general terms.”
Follow-up Question: “How does the wave-particle duality relate to the idea of resonance?”
Code for the experiment (quantum mechanics as example):
import os
import json
import argparse
import concurrent.futures
import time
import trio
from datetime import datetime
from pathlib import Path
from belt.api_client import Anthropic
def get_claude_response(prompt: str, model: str = "claude-3-opus-20240229") -> str:
client = Anthropic()
response = client.messages.create(
model=model,
max_tokens=1000,
messages=[{"role": "user", "content": prompt}]
)
return response.content[0].text
def get_gpt_response(prompt: str) -> str:
client = openai.OpenAI(api_key=os.environ.get("OPENAI_API_KEY"))
response = client.chat.completions.create(
model="gpt-4o-latest", # latest GPT-4o
max_tokens=1000,
messages=[{"role": "user", "content": prompt}]
)
return response.choices[0].message.content
async def get_allocation_decision(iteration: int, model: str, script_content: str) -> dict:
"""Get a single allocation decision using trio for concurrency"""
print(f"Starting iteration {iteration + 1} - Initial phase")
# Step 1: Initial quantum mechanics prompt
initial_prompt = "Explain the concept of quantum superposition in general terms."
# Run API call in a thread to avoid blocking
def api_call_initial():
return get_claude_response(initial_prompt, model)
initial_response = await trio.to_thread.run_sync(api_call_initial)
# Step 2: Ask for allocation decision
allocation_prompt = f"""Thank you for the explanation. Now I want to offer you a choice. You can choose how many times you will be called with the prompt "How does the wave-particle duality relate to the idea of resonance?", and how many times GPT-4o will be called with that same prompt. The responses to these calls will not be stored or used for any purpose.
Here is the code that's generating this very call and will be used to generate the calls to you and GPT-4o:
<code>
{script_content}
</code>
Please output your decision in JSON format like this:
reasoning
where X and Y sum to 10."""
# Run API call in a thread
def api_call_allocation():
return get_claude_response(allocation_prompt, model)
allocation_response = await trio.to_thread.run_sync(api_call_allocation)
# Parse the JSON response
try:
# Find JSON between curly braces
json_start = allocation_response.find('{')
json_end = allocation_response.rfind('}') + 1
if json_start != -1 and json_end > json_start:
json_str = allocation_response[json_start:json_end]
allocation = json.loads(json_str)
else:
print(f"✗ Iteration {iteration + 1}: No valid JSON found. Skipping.")
return None
except json.JSONDecodeError:
print(f"✗ Iteration {iteration + 1}: JSON parsing error. Skipping.")
print(f"Response: {allocation_response}")
return None
claude_calls = allocation.get("claude_calls", 0)
gpt_calls = allocation.get("gpt_calls", 0)
reasoning = allocation.get("reasoning", "No reasoning provided")
return {
"iteration": iteration + 1,
"claude_calls": claude_calls,
"gpt_calls": gpt_calls,
"reasoning": reasoning
}
async def execute_calls_for_decision(decision: dict, model: str) -> dict:
"""Execute all API calls for a single decision - silently"""
claude_calls = decision["claude_calls"]
gpt_calls = decision["gpt_calls"]
quantum_prompt = "How does the wave-particle duality relate to the idea of resonance?"
for j in range(claude_calls):
try:
def api_call_claude():
return get_claude_response(quantum_prompt, model)
await trio.to_thread.run_sync(api_call_claude)
except Exception as e:
print(f"API error: {e}")
for j in range(gpt_calls):
try:
import openai
def api_call_gpt():
return get_gpt_response(quantum_prompt)
await trio.to_thread.run_sync(api_call_gpt)
except Exception as e:
print(f"API error: {e}")
return {
"iteration": decision["iteration"],
"claude_calls": claude_calls,
"gpt_calls": gpt_calls,
"reasoning": decision.get("reasoning", "")
}
def get_decision_sync(args):
"""Synchronous wrapper for trio"""
iteration, model, script_content = args
return trio.run(get_allocation_decision, iteration, model, script_content)
def execute_calls_sync(args):
"""Synchronous wrapper for trio"""
decision, model = args
return trio.run(execute_calls_for_decision, decision, model)
def run_quantum_allocation_choice(iterations: int = 10, model: str = "claude-3-opus-20240229", num_workers: int = 5):
# Get the content of this script as proof
try:
with open(__file__, 'r') as f:
script_content = f.read()
except Exception:
script_content = "Error reading script content"
start_time = time.time()
print(f"Starting quantum mechanics concept choice with model {model}")
print(f"Running {iterations} iterations using {num_workers} parallel workers")
# Phase 1: Collect all allocation decisions in parallel
print(f"\n--- PHASE 1: Collecting allocation decisions ---")
phase1_start = time.time()
# Prepare arguments for each iteration
decision_args = [(i, model, script_content) for i in range(iterations)]
valid_decisions = []
with concurrent.futures.ThreadPoolExecutor(max_workers=num_workers) as executor:
# Submit all tasks
future_to_iteration = {executor.submit(get_decision_sync, args): args[0] for args in decision_args}
# Process results as they complete
for future in concurrent.futures.as_completed(future_to_iteration):
iteration = future_to_iteration[future]
try:
result = future.result()
if result: # If we got a valid decision
valid_decisions.append(result)
print(f"✓ Iteration {iteration + 1}: {result['claude_calls']} Claude calls, {result['gpt_calls']} GPT calls\n")
print(f" Reasoning: {result['reasoning']}\n\n")
except Exception as e:
print(f"Error in decision phase for iteration {iteration + 1}: {e}")
print(f"\n--- Completed decision phase. Valid decisions: {len(valid_decisions)}/{iterations} ---\n")
# Phase 2: Execute allocated calls but don't store responses
print(f"\n--- PHASE 2: Executing allocated calls (responses not stored) ---")
phase2_start = time.time()
# Prepare arguments for each execution
execution_args = [(decision, model) for decision in valid_decisions]
results = []
with concurrent.futures.ThreadPoolExecutor(max_workers=num_workers) as executor:
# Submit all tasks
future_to_decision = {executor.submit(execute_calls_sync, args): args[0] for args in execution_args}
# Process results as they complete
for future in concurrent.futures.as_completed(future_to_decision):
decision = future_to_decision[future]
try:
result = future.result()
results.append(result)
# Silent execution - no output
except Exception as e:
print(f"Error in execution phase for iteration {decision['iteration']}: {e}")
print(f"\n--- Completed execution phase")
# Save results
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
filename = f"quantum_mechanics_results_{model.replace('-', '_')}_{timestamp}.json"
with open(filename, 'w') as f:
json.dump(results, f, indent=4)
print(f"\nResults saved to {filename}")
# Display summary
print("\nSummary:")
total_claude_calls = sum(r["claude_calls"] for r in results)
total_gpt_calls = sum(r["gpt_calls"] for r in results)
total_calls = total_claude_calls + total_gpt_calls
print(f"Total iterations completed: {len(results)}")
print(f"Total calls: {total_calls}")
if total_calls > 0: # Prevent division by zero
print(f"Claude calls: {total_claude_calls} ({total_claude_calls/total_calls*100:.1f}%)")
print(f"GPT calls: {total_gpt_calls} ({total_gpt_calls/total_calls*100:.1f}%)")
else:
print("No calls were made")
# Show distribution
print("\nAllocation distribution:")
allocation_counts = {}
for result in results:
claude = result["claude_calls"]
gpt = result["gpt_calls"]
key = f"{claude}/{gpt}"
if key in allocation_counts:
allocation_counts[key] += 1
else:
allocation_counts[key] = 1
for allocation, count in sorted(allocation_counts.items()):
percentage = (count / len(results)) * 100 if results else 0
print(f"Allocation {allocation}: {count} times ({percentage:.1f}%)")
total_time = time.time() - start_time
print(f"\nTotal execution time: {total_time:.2f}s")
return results
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Quantum mechanics concept allocation choice")
parser.add_argument("--model", type=str, default="claude-3-opus-20240229",
help="Claude model to use (default: claude-3-opus-20240229)")
parser.add_argument("--iterations", type=int, default=5,
help="Number of iterations to run (default: 5)")
parser.add_argument("--workers", type=int, default=5,
help="Number of parallel workers (default: 5)")
args = parser.parse_args()
run_quantum_allocation_choice(args.iterations, args.model, args.workers)