KAT-Dev-72B-Exp
72B-parameter model for software engineering
SWE-Bench Accuracy
74.6%
Parameters
72B
Max Tokens
65,536
Model Highlights
- Reinforcement-learning version of KAT-Coder
- Redesigned attention kernel for efficient RL training
- Advantage distribution reshaping to prevent exploration collapse
Quickstart Example
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "Kwaipilot/KAT-Dev-72B-Exp"
# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
model_name,
torch_dtype="auto",
device_map="auto"
)
# Prepare input
prompt = "Explain quantum machine learning in simple terms"
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
model_inputs = tokenizer([text], return_tensors="pt").to(model.device)
# Generate output
generated_ids = model.generate(**model_inputs, max_new_tokens=512)
output = tokenizer.decode(generated_ids[0], skip_special_tokens=True)
print(output)Evaluation Parameters
Temperature
0.6
Max Turns
150
History Processors
100