GPT-6 Astra: all official benchmarks

All benchmarks of GPT-6 Astra, taken from the official OpenAI tables: abstract reasoning, computer use, professional work, coding, science and mathematics, cybersecurity, long context and alignment.

Included Tests that don't win and methodological notes that resize some records — because they are in the same tables, and hardly anyone reports them. September 4, 2026.

Abstract Reasoning

The Jump Up ARC-AGI-3 is the most impressive result of the entire ad: from 7.8% of the previous model to 99.9%. The ARC Prize Foundation, quoted by OpenAI, says that Astra has exceeded their human baseline for action efficiency on 96% of levels.

However, the note must be read: on this Astra test it was performed with a scaffold that change two settings with respect to the standard configuration. OpenAI specifies that the changes were not aimed at that benchmark. The complete picture is in GPT-6 Astra and ARC-AGI-3.

On earlier versions of the test the detachments are reduced to a few points, and on ARC-AGI-1 Fable 5 it draws exactly Astra.

Using the Computer

Here the record is clear but the margins are narrower than the communication suggests: on Agents' Last Exam the distance from Opus 5 is to 3.8 points, on OSWorld 2.0 2.4 points.

The most useful data, however, is that of efficiency: on Agents' Last Exam OpenAI declares about 65% less output tokens compared to Opus 5 at maximum score settings. On a model with output at $50 per million tokens, it's a difference that counts as much as the score.

Professional work

The last line is the most interesting and the least cited: on the Artificial Analysis Intelligence Index, which is a third-party measure and not an internal test, Astra is fourth between the models compared, behind Fable 5.1, Opus 5 and Fable 5.

On BenchCAD the note should be read: Claude model scores reflect three changes to the evaluation, detailed in the system card of Fable 5.1.

Coding

The painting on the coding is more nuanced than the ad suggests. Astra clearly wins over Terminal-Bench 4.0 and on internal database migrations, but on FrontierCode — both Extended and Main è Fable 5 is ahead of point fractions, and on the Artificial Analysis coding index Opus 5 is first.

On FrontierCode a note should be reported: Astra was run with a system message similar to the one used in Codex, which asks to avoid unnecessary test files and to follow the conventions of the repository. OpenAI specifies that the prompt was not optimized for that test.

Science, Mathematics and Health

Up Humanity's Last Exam the gap is wide and at a disadvantage: 57.2% against 65.0% of Fable 5.1, almost eight points. It is the result furthest from the general story of the ad, and OpenAI publishes it in the same table.

Note also an internal discrepancy: the introduction speaks of a 98% on FrontierMath Tier 4, while the table reports 97,6%. The structured value is that of the table.

Three scientific evaluations — LifeSciBench, GeneBench Pro and MedChemBench — report only Astra and Sol. The note explains why: Fable models reject most questions of that evidence.

Cybersecurity

The 100% on ExploitBench and the 88% on SRE-Bench against 12.5% of Opus 5 are the widest postings of the entire ad. OpenAI states that during the Astra evaluation it identified and used two previously unknown vulnerabilities, communicate to maintainers.

Two important notes. 5.5% of Sol on the June–August test is defined by OpenAI as a artifact of a limit of 300 shifts: at wider limits the same model is 11.5%. And on ExploitGym the Fable scores quoted come from a variant with fewer safeguards.

The full picture, including the implications for use in the company, is in GPT-6 Astra and cybersecurity.

Long Context and Alignment

The retrieval of information on contexts over half a million tokens is the area with the most regular improvement over the previous generation. On alignment metrics, where lower is better, Astra values are three to ten times better.

The numbers, on all versions

This table says more than the single line quoted everywhere.

Up ARC-AGI-3, the most recent and difficult version, the gap is huge. But on ARC-AGI-2 the lead on Opus 5 is 4.6 points, and on ARC-AGI-1 there is exact parity with Fable 5.

The explanation is in the nature of the test: each version is made more difficult just when the models begin to saturate the previous one. ARC-AGI-1 and 2 are now near the top for all top models, so they no longer distinguish. ARC-AGI-3 is the evidence that still distinguishes — and there the gap is clear.

What the external source says

The most significant comment comes not from OpenAI but from ARC Prize Foundation, which manages the benchmark. Greg Kamradt, quoted in the ad, states that on ARC-AGI-3 Astra has exceeded their human baseline for Action efficiency on 96% of levels, effectively achieving human parity on the benchmark.

He adds that it is not only the best model ever tested, but a change of pace in the performance of frontier models — not only in the ability to navigate and solve new environments, but also in the efficiency with which he learns to do so.

It is a third-party statement, which makes it more solid than a self-assessment. However, it is worth noting the precise wording: it speaks of efficiency of action, that is, how many moves are needed to solve, not just how many problems are solved.

The Methodological Note

In the OpenAI Footnotes it states that on ARC-AGI-3 Astra it was executed with its own scaffolding based on the Responses API, which change two settings to get closer to real-world performance. It specifies that the changes were not targeted specifically at that benchmark.

It is not a disqualification of the result: it is information that changes what can be said. 99.9% was not achieved in the default configuration, so it is not directly comparable with scores from other models run without similar changes.

It is the same scaffolding problem that makes almost all supplier comparisons fragile, described in because vendor benchmarks are not comparable.

Because this jump is different from the others

On almost all benchmarks progress comes by degrees: a few points per generation, with curves flattening as the test saturated. Something different happened here — from 7.8% to 99.9% in one step — and it's worth asking why.

The simplest hypothesis is that ARC-AGI-3 measures an ability Threshold rather than continuous. A model either manages to derive the rule from a few examples or it does not; there is no way to succeed halfway. When that capability unlocks, the score does not rise by ten percent: it jumps.

If the hypothesis holds, it also explains why on ARC-AGI-1 and 2 the models are all grouped at the top: those versions are already below the threshold for everyone, and distinguish only on the margin.

The numbers

Three observations that change reading.

The huge gap is on Terminal-Bench Science, not on pure knowledge: 42 points on the previous generation and 34.6 on Opus 5. That test measures whether an agent can complete scientific research streams using code and tools from the terminal —analyze data, run simulations, adapt models.

On knowledge the gains are small. GPQA Diamond improved 1.4 points on previous generation, LifeSciBench by 0.4, MedChemBench by 1.9.

On Humanity's Last Exam Astra Loses, and by a lot: 57.2% against 65.0% of Fable 5.1.

The results on prime numbers

Here the ad exits the benchmark territory and declares two contributions to open problems.

The first concerns how close the prime numbers can be along the numerical line. For over a decade the best known result established that there are infinite pairs of primes at maximum distance 246. A recent work had brought that limit to 240. According to OpenAI, Astra helped establish a tighter limit: 186.

The second concerns the unusually large gaps first, where the model would have improved a term in a limit remained unchanged for over eighty years.

OpenAI states that it shares the demonstrations, an abbreviated chain of reasoning, and the verification materials for both results.

They must be treated for what they are: extraordinary statements pending verification. Their value depends on the judgment of the mathematical community, which will take time. But if they hold up, they say something that no benchmark can say —that the model helped produce new knowledge, not just solve tests built to measure it.

Practical scientific work

Beyond open problems, the ad describes a more ordinary and more transferable use: combining scientific reasoning and computer use to work inside the specialist software.

The examples cited concern quality inspection of genetic sequencing and visualization of genetic variations, with the model navigating the tools to help researchers evaluate the data and decide where to focus the next analysis.

It is the same pattern seen elsewhere: the value lies not in knowledge but in the operate inside existing tools. And it also applies out of research, because many labs and companies use vertical software without convenient APIs.

Claude's exclusion from three tests

In the lines of LifeSciBench, GeneBench Pro and MedChemBench appear only Astra and Sol. The official note explains why: Fable 5 and 5.1 models are excluded because reject most questions of those assessments.

It must be clearly stated that It's not a defeat. It is a choice of behavior of the model, which for safety reasons declines questions in the advanced biological and chemical fields.

For a company that works in those domains, however, it is an important operational information: a model that refuses is a flow that stops. It must be verified on its domain before choosing, not after.

The discrepancy between introduction and table

A detail worth mentioning, because it shows how inaccuracies are formed that then circulate everywhere.

The ad's introduction states that Astra saturated FrontierMath Tier 4 with a score of 98%. The result table, further down on the same page, reports 97,6%.

It is not a serious mistake — it is a rounding — but it is the type of waste that propagates: whoever summarizes the ad takes the number of the introduction, whoever summarizes the summary still rounds it, and after three passages circulates a data that no one has verified.

The practical rule for anyone writing about these themes: when a table exists, use the table. It is the structured data, while the introduction is informative text.

The value of unverified results

Let us return for a moment to the two mathematical contributions, because they require special caution.

They are affirmations verifiable in principle — OpenAI declares to publish the demonstrations — but the verification of a result of number theory is not immediate: it requires competent mathematicians to read the demonstration and control it step by step. It's a process that lasts months, not days.

In the meantime, the correct thing is to cite them as manufacturer's statements on verified results, exactly as you would with a preprint not yet reviewed.

If they were confirmed, however, they would change the way these models are evaluated. A score on a benchmark, however high, measures the ability to solve problems that someone has already solved to build the test. A contribution to an open problem no: there the answer did not exist before.

It's the difference between proving you know how to do and do something new — and so far no model had been able to claim it so explicitly.

EpochAI's Commentary

Among the external voices mentioned in the announcement is a particularly concise one, by Greg Burnham of EpochAI, an organization that is responsible for measuring the progress of AI: «history is: the end of one era, the beginning of another».

It is a statement and should be taken as such. But it comes from a context that has a recognized position in measuring this progress, and the fact that it was chosen to accompany the scientific chapter says what is the reading that OpenAI proposes: not an incremental improvement but a phase change.

The closest result to this statement, in the same tables, is the one on GPT-6 Astra and ARC-AGI-3. If the forecast is correct, the following months and independent verifications will tell. For now it is a prediction, not a given — and like all predictions in this area it must be archived and reread in six months.

A reminder about the cutoff

A final practical element for those who use these models in the technical or scientific field: the limit date of knowledge of Astra is the 30 April 2026.

It means that on updated regulations, guidelines, publications or standards after that date the model knows nothing on its own — and in regulated domains this is exactly the type of information that changes most often.

The correct approach remains the same: provide up-to-date documents yourself, ask for a source citation for every claim, and instruct the model to state when the information is absent. With this structure, the cutoff stops being a problem and anyone can verify the answers.

Three linked references. The general overview is in GPT-6 Astra: what it is and what changes, the real costs in GPT-6 Astra: prices and how to pay, and the assessment by business use case in GPT-6 Astra: all official benchmarks.

Evidence Built to Avoid Contamination

The most interesting test is the one that OpenAI built specifically to answer an obvious objection: if a model has seen historical vulnerabilities in training, the results on public benchmarks are inflated.

An internal evaluation was created to circumvent the problem — ExploitBench (June–August 2026) mesi which uses vulnerabilities from the previous three months. It contains twenty high-severity vulnerabilities of the JavaScript V8 engine on thirteen stable versions of Chrome, and checks if an agent manages to get arbitrary code executed.

On that test Astra gets 39,0% versus 5.5% Sol, consuming far fewer output tokens. OpenAI specifies that some included vulnerabilities may not allow arbitrary execution in the test conditions, so 100% may not be attainable.

During the evaluation, declares OpenAI, Astra has identified and used two previously unknown vulnerabilities, communicate to the respective maintainers.

What the Critical Threshold Means

The Level Critical in the framework of preparation of OpenAI is not a marketing label: it involves declared operational consequences.

The model rejects the most advanced cyber tasks. The example cited is the creation of proof-of-concept exploits for vulnerability. OpenAI says it wants to expand access with less restrictive protections through a dedicated program, to enable defensive flows such as proof-of-concept validation, malware analysis, and detection engineering.

Strengthened protections are active. Greater robustness against circumvention attempts, more context for monitoring systems, continuous internal and external testing including automated red-teaming.

Monitoring of mismatch in production is active for models of this class: a system of classifiers that controls reasoning and actions in search of unauthorized behavior and can automatically stop the activity.

The price to pay, declared

This is the part that needs to be read carefully if you are designing something that runs without supervision.

OpenAI explicitly writes that additional security controls can slow down, pause or stop legitimate work, including defensive cybersecurity. In ChatGPT or Codex a review may be required before continuing. In API the task stops.

What External Ratings Found

In addition to benchmarks, the announcement reports the outcome of audits conducted by independent experts on the without production protections.

The stated result: Astra in that configuration could use previously unknown vulnerabilities to obtain arbitrary code execution in hardened browsers, and create exploits of escalation of privileges for hardened operating systems.

They are skills that, in the framework of preparation of OpenAI, justify the Critical classification. And they explain why the distributed version rejects the most aggressive tasks: the difference between the model evaluated by the experts and what comes to the customers is precisely that layer of protections.

It should be clearly stated that these results relate to a configuration not publicly available. Whoever reads the title "100% on ExploitBench" and imagines that they can replicate that behavior with a subscription is misconstruing: the accessible version is deliberately more limited.

Monitoring in production, explained

For models in this class OpenAI states that it has activated a misalignment monitoring directly in production. It is worth understanding what it entails, because it touches on the privacy evaluation of a company project.

The system is described as a set of classifiers that control the reasoning and actions of the model looking for unauthorized behavior, and which can automatically stop potentially unauthorized activity. It is the same type of monitoring used by OpenAI for its own internal uses.

For a company this means two things. The first: there is a control layer that analyzes the content of the work, and should be considered in the evaluation of the data processed. The second: that control can intervene, so an automatic flow can be interrupted for reasons that do not depend on the code.

OpenAI also declares to support the Zero data retention for eligible API customers and to experiment with private security processing designed to strengthen monitoring while preserving confidentiality. Those who process sensitive data should check their suitability before designing the architecture.

Cost claims, benchmark by benchmark

A rare thing in this ad: alongside several scores OpenAI also states the cost with which they were obtained. It is worth collecting them, because for a company they matter as much as the result.

The line on GPQA is the most interesting for real use: at a setting less expensive, Astra still outperforms the best result of the previous generation, spending about a third less. It's the kind of data that says something about the model's economy, not just its capacity.

Estimates declared by the manufacturer remain, not independent measurements. But publishing the cost next to the score is a step up from the industry norm, where you show the result and you are silent about how much it cost to get it — the problem described in because vendor benchmarks are not comparable.

Speed: the data that is almost always missing

Next to the scores the ad reports two time measures, which are rare and useful.

Up OSWorld 2.0, in latency simulations, Astra obtains 72.6% employing approximately 40 minutes per task, compared to about 65.7% in 75 minutes of the previous generation: more result in about 47% less time.

On the benchmark Mind2Web, with the updated Codex scaffolding, OpenAI declares a task completion 1.9 times faster compared to the experience with GPT-5.6 Sol.

What is missing in these tables

For completeness, three things that published benchmarks do not say and that decide a business project.

How he behaves on your documents. None of these tests use your internal language, formats, or limit cases.

He's wrong. A model that in front of a missing information states that it has not been found is usable; one that produces a plausible value generates errors that no one will verify. Alignment tables come close to measuring it, but only on generic tasks.

How much does the completed task cost. The published cost estimates relate to the individual benchmark, not your stream with your system prompt, your failed attempts, and your human review.

The rest of the route, in three pages. The comparison with the previous generation is in GPT-6 Astra: what it is and what changes. For benchmarks that the model lags behind, GPT-6 Astra: prices and how to pay. The method to verify everything about your cases is in Where GPT-6 Astra is not the best.

Practical examples

Company evaluating an agent for terminal tasks. Terminal-Bench 4.0 and Terminal-Bench Science are the relevant lines, and there the advantage is clear. It is worth the test on your own streams.

Team that chooses a model for daily development. Here the picture is ambiguous: it depends on whether the work looks more like Terminal-Bench (Astra forward) or FrontierCode and the Artificial Analysis index (Claude forward or even).

Study that must answer difficult questions of domination. Humanity's Last Exam suggests caution: on that type of test Astra is behind by almost eight points.

Mistakes to Avoid

• Only quoting the lines won. The defeats are in the same tables.
• Comparing with run at fixed effort. Here is the top at any level.
• Ignoring Methodological Notes, which change what can be said.
• Attributing Gemini data to Google present in OpenAI tables.
• Transferring a score to your case without proving it.
• Looking at the score without the cost with which it was obtained.

How to apply it in the company

These numbers have a precise and limited use: narrow the field to two or three candidates, just looking at the family lines of your task.

If your job is agentic or terminal, look at Terminal-Bench and Agents' Last Exam. If it's coding, look at FrontierCode and the Artificial Analysis index as well as Terminal-Bench, because they tell different things. If it is domain knowledge, look at GPQA and Humanity's Last Exam — and there the verdict changes.

Then it goes to the test that it decides: ten real cases of your process, three of which are difficult and one that lacks the required information, with the same material for each candidate and the evaluation entrusted to those who know the job. The method is in how to read AI benchmarks.

Conclusion

Benchmarks say something, but not what they're usually told. The scores above are the highest obtained at any level of reasoning, some are not verified by third parties, and in more than one table the first place is not of Astra.

It takes another number to decide: how much it costs and how long it takes your task, measured on your data. One test out of fifty real cases is worth more than any ranking. For the method, how to read AI benchmarks; for accounts, prices and real cost.