Images, charts and screenshots
Working from images: answering questions that need a chart, diagram or photo, and fixing software bugs shown in screenshots. Good means the model gets the task right when it depends on what is in the image.
Sources last checked 2026-10-23.
We can't show scores for this kind of work yet. Vals AI publishes results from two tests. See who tests it
At a glance
Vendor claims have not been collected for any category yet.
ⓘ More about this category
- Reliability not yet measured
- 2 sources, linked results only
- Sources last checked 2026-10-28
- Ring 4: needs rings 1 to 3 first
- MMMU = MMMU Pro, SWMM = SWE-bench Multimodal
Tested by
MMMU Pro
Vals AI, checked 2026-10-28Graduate-level multiple-choice questions across 30 subjects where the answer depends on reading an image, such as a chart, diagram or photo, together with the text.
Vals AI reports that the leading models now slightly exceed the best reported human expert performance on this test.
Last updated 2026-09-01Covers 32 of the models we trackScores not shown: permission requested on 2026-10-24
See their resultsMore about this test: MMMU Pro
Reliability not measured by this source.
Technical details for MMMU Pro
Accuracy: Graduate-level multiple-choice questions across 30 subjects where the answer depends on reading an image, such as a chart, diagram or photo, together with the text.
License: Score reuse not stated. Checked 2026-10-28.
SWE-bench Multimodal
Vals AI, checked 2026-10-28Real bug reports from JavaScript projects that depend on a screenshot or recording; the model must read the image, find the faulty code and fix it so the project's tests pass.
Vals AI reports that the official fix for each task is easy to find, and agents with general network access use it.
Last updated 2026-10-25Covers 9 of the models we trackScores not shown: permission requested on 2026-10-24
See their resultsMore about this test: SWE-bench Multimodal
Reliability not measured by this source.
Technical details for SWE-bench Multimodal
Accuracy: Real bug reports from JavaScript projects that depend on a screenshot or recording; the model must read the image, find the faulty code and fix it so the project's tests pass.
License: Score reuse not stated. Checked 2026-10-28.
What this doesn't tell you
- None of these tests used your images or your charts.
- Making images is a different job, not covered here.
- Scores from different sources are not comparable, even when they share a unit.
- None of these tests were run on your own documents, code or data.
- No source for this task allows us to show its figures yet. Follow the links for their results.
Data version 2026-10-28+91ac3fc9d071 · Terms of use