SWE-bench Multimodal: Do AI Systems Generalize to Visual Software Domains?
Paper • 2410.03859 • Published • 1
ProgramBench
university
AI & ML interests
None defined yet.
Recent Activity
View all activity
john-b-yangÂ
authored 4
papers about 20 hours ago
OpenThoughts: Data Recipes for Reasoning Models
Paper • 2506.04178 • Published • 55
LongCodeBench: Evaluating Coding LLMs at 1M Context Windows
Paper • 2505.07897 • Published
EnIGMA: Interactive Tools Substantially Assist LM Agents in Finding Security Vulnerabilities
Paper • 2409.16165 • Published
john-b-yangÂ
authored 5
papers about 21 hours ago
Collaborative Gym: A Framework for Enabling and Evaluating Human-Agent Collaboration
Paper • 2412.15701 • Published
Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces
Paper • 2601.11868 • Published • 36
SWE-Protégé: Learning to Selectively Collaborate With an Expert Unlocks Small Language Models as Software Engineering Agents
Paper • 2602.22124 • Published • 2
SWE-chat: Coding Agent Interactions From Real Users in the Wild
Paper • 2604.20779 • Published • 14
ProgramBench: Can Language Models Rebuild Programs From Scratch?
Paper • 2605.03546 • Published • 3
klieretÂ
updated a
dataset 8 days ago
klieretÂ
published a
dataset 10 days ago
john-b-yangÂ
updated a
Space 11 days ago
john-b-yangÂ
updated a
dataset 11 days ago
klieretÂ
published a
Space 11 days ago
john-b-yangÂ
authored a
paper 6 months ago
john-b-yangÂ
authored 5
papers about 1 year ago
SWE-smith: Scaling Data for Software Engineering Agents
Paper • 2504.21798 • Published • 15
SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
Paper • 2310.06770 • Published • 10
InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedback
Paper • 2306.14898 • Published
DevBench: A Comprehensive Benchmark for Software Development
Paper • 2403.08604 • Published • 2
WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents
Paper • 2207.01206 • Published • 3