Results

Run: svc_121_b4bca3

Cost is short-context API-equivalent usage, not a subscription payment; full tool/response trace unavailable in CLI export.

Astra_standard_short_context_API_equivalent_not_subscription_bill. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"Astra_cost_excludes_per_request_long_context_fast_mode_and_tool_adjustments":50,"codex_model_response_count_and_full_tool_calls_unavailable":50}

Run: svc_59_708d1b

New no-preload Energy50 evaluation; preserve its distinct track and measurement contract when comparing earlier runs. Cost is the logged template estimate from final cumulative snapshots, not an invoice.

recorded_template_price_estimate. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_31_878995

Claude_standard_global_list_price_estimate_from_logged_usage. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_157_c2dbaf

Completed local evaluation 157; all 50 tasks. Single-action mode enforces one GUI action followed by a fresh screenshot before another action. Same model, effort and configured limits as the normal-I/O xhigh baseline; owner-approved current runtime and Codex subscription. Cost is the same historical standard API-equivalent estimate from logged usage, not subscription spending. The source link contains derived task metrics and source hashes; the trajectory archive has not been published to Hugging Face. One GUI action per request, with a fresh screenshot required before the next action.

Astra_standard_short_context_API_equivalent_not_subscription_bill. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"Astra_cost_excludes_per_request_long_context_fast_mode_and_tool_adjustments":50,"codex_model_response_count_and_full_tool_calls_unavailable":50}

Run: svc_61_1ec4be

New no-preload Energy50 evaluation; preserve its distinct track and measurement contract when comparing earlier runs. Cost is the logged template estimate from final cumulative snapshots, not an invoice.

recorded_template_price_estimate. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_151_181620

Added 2026-09-14 at owner request. Separate no-preload track/measurement contract; not a same-track ranking against older preload runs. Cost uses the prior September11 short-context API-equivalent pricing snapshot, not subscription billing. Full CLI model-response/tool trace remains unavailable. Final-task metrics exclude retained non-final attempts.

Astra_standard_short_context_API_equivalent_not_subscription_bill. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"Astra_cost_excludes_per_request_long_context_fast_mode_and_tool_adjustments":50,"codex_model_response_count_and_full_tool_calls_unavailable":50}

Run: svc_152_8d02f4

Added 2026-09-14 at owner request. Separate no-preload track/measurement contract; not a same-track ranking against older preload runs. Cost uses the prior September11 short-context API-equivalent pricing snapshot, not subscription billing. Full CLI model-response/tool trace remains unavailable. Final-task metrics exclude retained non-final attempts. One retained infrastructure attempt is represented separately; final-task API-equivalent total=31.818674 USD, extra attempt=0.263074 USD, all recorded sessions=32.081748 USD.

Astra_standard_short_context_API_equivalent_not_subscription_bill. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"Astra_cost_excludes_per_request_long_context_fast_mode_and_tool_adjustments":50,"codex_model_response_count_and_full_tool_calls_unavailable":50}

Run: svc_127_422895

Cost is short-context API-equivalent usage, not a subscription payment; full tool/response trace unavailable in CLI export.

Astra_standard_short_context_API_equivalent_not_subscription_bill. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"Astra_cost_excludes_per_request_long_context_fast_mode_and_tool_adjustments":50,"codex_model_response_count_and_full_tool_calls_unavailable":50}

Run: svc_24_ddcb3b

Claude_standard_global_list_price_estimate_from_logged_usage. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_85_948d7f

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_86_9e7a56

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_141_f28fed

recorded_provider_usage_cost. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_32_909005

Claude_standard_global_list_price_estimate_from_logged_usage. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"Claude_cost_assumes_standard_global_for_responses_with_missing_tier_geo":1}

Run: svc_122_93c9c5

Cost is short-context API-equivalent usage, not a subscription payment; full tool/response trace unavailable in CLI export.

Astra_standard_short_context_API_equivalent_not_subscription_bill. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"Astra_cost_excludes_per_request_long_context_fast_mode_and_tool_adjustments":50,"codex_model_response_count_and_full_tool_calls_unavailable":50}

Run: svc_87_735f9a

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_60_8db123

New no-preload Energy50 evaluation; preserve its distinct track and measurement contract when comparing earlier runs. Cost is the logged template estimate from final cumulative snapshots, not an invoice.

recorded_template_price_estimate. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_56_9e002a

Published Energy50 max-reasoning subset of Kimi evaluation56; not a separate execution. One model tool call per response; the code in that call can contain multiple GUI actions.

recorded_provider_usage_cost. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"Kimi_finish_rejected_response_usage_included_but_tool_bodies_unlogged":1,"Kimi_noncompletion_replies_have_no_logged_usage_or_body":2}

Run: svc_20_9456b8

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_23_753ef9

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_90_357e81

recorded_provider_usage_cost. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_22_3a244d

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_13_01885e

Published HF PR23, merged before inventory. All 50 final tasks complete; original no-preload contract and GUI-only tool configuration retained. Full response usage recovered from agent.stdout despite the published README marking usage unavailable; duplicated SDK step usage is not counted again. Cost is an API-rate estimate using logged input/output and billed cached input at publisher rates verified September 19, not a recorded charge. Reasoning-token breakdown is incomplete and remains unknown.

published_API_rate_estimate_from_logged_usage_and_billed_cache_not_recorded_charge. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"reasoning_token_breakdown_not_logged_for_all_responses":50}

Run: svc_155_7fbf62

Added 2026-09-16 at owner request. GPT-5.6 Luna via Codex, not the direct-API gpt54 template. All 50 original tasks included, including failures. No rerun or regrading. No-preload track/measurement contract; not a same-track ranking against older preload runs. Costs are standard short-context API-equivalent estimates using official 2026-09-16 Luna rates, not subscription charges. All 50 session token summaries complete; cached input is a subset of input, reasoning is a subset of output, recorded cache-write counters are zero. Full CLI model-response/tool trace remains unavailable; observed CLI tool items and harness steps are separate metrics. No additional retained attempts found. Submitted agent.py SHA-256=306fe8c02f9c85a5ff90be95d8efef897a3555ab37fe4f46e84dad68dad0058a. Full per-task normalized evidence and usage breakdowns are linked in companion_cost_usage_url.

Luna_standard_short_context_API_equivalent_not_subscription_bill. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges; assumes standard short-context rates; per-request long-context/fast-mode adjustments and tool charges cannot be reconstructed from session totals; output already includes reasoning

{"Luna_cost_excludes_per_request_long_context_fast_mode_and_tool_adjustments":50,"codex_model_response_count_and_full_tool_calls_unavailable":50}

Run: svc_144_1f26e4

recorded_provider_usage_cost. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_69_41f88a

Standard API price proxy, not the archived contributor-tier charge; historical contributor pricing is unverified. Recorded input/output tokens; no input caching assumed. Rates verified 2026-09-16; excludes other charges.

Run: svc_156_2bc119

Added 2026-09-16 at owner request. GPT-5.6 Luna via Codex, not the direct-API gpt54 template. All 50 original tasks included, including failures. No rerun or regrading. No-preload track/measurement contract; not a same-track ranking against older preload runs. Costs are standard short-context API-equivalent estimates using official 2026-09-16 Luna rates, not subscription charges. All 50 session token summaries complete; cached input is a subset of input, reasoning is a subset of output, recorded cache-write counters are zero. Full CLI model-response/tool trace remains unavailable; observed CLI tool items and harness steps are separate metrics. No additional retained attempts found. Submitted agent.py SHA-256=306fe8c02f9c85a5ff90be95d8efef897a3555ab37fe4f46e84dad68dad0058a. Full per-task normalized evidence and usage breakdowns are linked in companion_cost_usage_url.

Luna_standard_short_context_API_equivalent_not_subscription_bill. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges; assumes standard short-context rates; per-request long-context/fast-mode adjustments and tool charges cannot be reconstructed from session totals; output already includes reasoning

{"Luna_cost_excludes_per_request_long_context_fast_mode_and_tool_adjustments":50,"codex_model_response_count_and_full_tool_calls_unavailable":50}

Run: svc_11_0dddb1

Published HF PR23, merged before inventory. All 50 final tasks complete; original no-preload contract and GUI-only tool configuration retained. Full response usage recovered from agent.stdout despite the published README marking usage unavailable; duplicated SDK step usage is not counted again. Cost is an API-rate estimate using logged input/output and billed cached input at publisher rates verified September 19, not a recorded charge. Reasoning-token breakdown is incomplete and remains unknown.

published_API_rate_estimate_from_logged_usage_and_billed_cache_not_recorded_charge. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"reasoning_token_breakdown_not_logged_for_all_responses":50}

Run: svc_68 + rerun3_xhigh13

Published STITCH-musespark13-xhigh-68.txt: 47 original tasks plus three rerun replacements; overlaps both components and is not an independent run. Published 664.79 seconds/task is mean(agent_wall_sec + env_boot_sec)=664.78846; timed task clock=612.07638625182 seconds/task. Published as one combined 50-task result. Original tasks ran at concurrency 8; the three recovery tasks ran at concurrency 3. Both source archives are retained below.

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

svc_68_bca75f · rerun3_xhigh13

{"result_json_absent_metrics_reduced_from_original_runlog_events":47}

Run: svc_65_46a645

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_25_adbcee

Claude_standard_global_list_price_estimate_from_logged_usage. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_153_795ea7

Added 2026-09-16 at owner request. GPT-5.6 Luna via Codex, not the direct-API gpt54 template. All 50 original tasks included, including failures. No rerun or regrading. No-preload track/measurement contract; not a same-track ranking against older preload runs. Costs are standard short-context API-equivalent estimates using official 2026-09-16 Luna rates, not subscription charges. All 50 session token summaries complete; cached input is a subset of input, reasoning is a subset of output, recorded cache-write counters are zero. Full CLI model-response/tool trace remains unavailable; observed CLI tool items and harness steps are separate metrics. No additional retained attempts found. Submitted agent.py SHA-256=306fe8c02f9c85a5ff90be95d8efef897a3555ab37fe4f46e84dad68dad0058a. Full per-task normalized evidence and usage breakdowns are linked in companion_cost_usage_url.

Luna_standard_short_context_API_equivalent_not_subscription_bill. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges; assumes standard short-context rates; per-request long-context/fast-mode adjustments and tool charges cannot be reconstructed from session totals; output already includes reasoning

{"Luna_cost_excludes_per_request_long_context_fast_mode_and_tool_adjustments":50,"codex_model_response_count_and_full_tool_calls_unavailable":50}

Run: svc_51_121311

Standard API rates through 512K context; above 512K, input/output/cache rates double. Priority pricing differs. Archived responses do not retain token usage, so unit prices cannot yield a per-task total.

{"output_truncated_at_400_characters_and_usage_not_logged":50}

Run: svc_10_cd0ed2

Published HF PR23, merged before inventory. All 50 final tasks complete; original no-preload contract and GUI-only tool configuration retained. Full response usage recovered from agent.stdout despite the published README marking usage unavailable; duplicated SDK step usage is not counted again. Cost is an API-rate estimate using logged input/output and billed cached input at publisher rates verified September 19, not a recorded charge. Reasoning-token breakdown is incomplete and remains unknown.

published_API_rate_estimate_from_logged_usage_and_billed_cache_not_recorded_charge. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"reasoning_token_breakdown_not_logged_for_all_responses":50}

Run: svc_21_00be0a

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_143_e8d33d

recorded_provider_usage_cost. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_70_8f6cda

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_89_b4886a

recorded_provider_usage_cost. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_52_554434

Standard API rates through 512K context; above 512K, input/output/cache rates double. Priority pricing differs. Archived responses do not retain token usage, so unit prices cannot yield a per-task total.

{"output_truncated_at_400_characters_and_usage_not_logged":50}

Run: svc_59_d8a6d4

Standard API price proxy, not the archived contributor-tier charge; historical contributor pricing is unverified. Recorded input/output tokens; no input caching assumed. Rates verified 2026-09-16; excludes other charges.

Run: svc_58_38520a

Standard API price proxy, not the archived contributor-tier charge; historical contributor pricing is unverified. Recorded input/output tokens; no input caching assumed. Rates verified 2026-09-16; excludes other charges.

Run: svc_60_72500b

Standard API price proxy, not the archived contributor-tier charge; historical contributor pricing is unverified. Recorded input/output tokens; no input caching assumed. Rates verified 2026-09-16; excludes other charges.

Run: svc_39_025394

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_154_323b97

Added 2026-09-16 at owner request. GPT-5.6 Luna via Codex, not the direct-API gpt54 template. All 50 original tasks included, including failures. No rerun or regrading. No-preload track/measurement contract; not a same-track ranking against older preload runs. Costs are standard short-context API-equivalent estimates using official 2026-09-16 Luna rates, not subscription charges. All 50 session token summaries complete; cached input is a subset of input, reasoning is a subset of output, recorded cache-write counters are zero. Full CLI model-response/tool trace remains unavailable; observed CLI tool items and harness steps are separate metrics. No additional retained attempts found. Submitted agent.py SHA-256=306fe8c02f9c85a5ff90be95d8efef897a3555ab37fe4f46e84dad68dad0058a. One original agent_error on osworld_libreoffice_writer_adf5e2c3-64c7-4644-b7b6-d2f0167927e7/seed_1031319744: the environment HTTPS endpoint refused the /done connection and the gateway failed. Its zero score, timing and recorded usage remain included; not a clean failure-free comparison. Full per-task normalized evidence and usage breakdowns are linked in companion_cost_usage_url.

Luna_standard_short_context_API_equivalent_not_subscription_bill. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges; assumes standard short-context rates; per-request long-context/fast-mode adjustments and tool charges cannot be reconstructed from session totals; output already includes reasoning

{"Luna_cost_excludes_per_request_long_context_fast_mode_and_tool_adjustments":50,"codex_model_response_count_and_full_tool_calls_unavailable":50}

Run: svc_12_026ed7

Published HF PR23, merged before inventory. All 50 final tasks complete; original no-preload contract and GUI-only tool configuration retained. Full response usage recovered from agent.stdout despite the published README marking usage unavailable; duplicated SDK step usage is not counted again. Cost is an API-rate estimate using logged input/output and billed cached input at publisher rates verified September 19, not a recorded charge. Reasoning-token breakdown is incomplete and remains unknown.

published_API_rate_estimate_from_logged_usage_and_billed_cache_not_recorded_charge. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

{"reasoning_token_breakdown_not_logged_for_all_responses":50}

Run: svc_61_96ce3f

Standard API price proxy, not the archived contributor-tier charge; historical contributor pricing is unverified. Recorded input/output tokens; no input caching assumed. Rates verified 2026-09-16; excludes other charges.

Run: svc_66_660b4a

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_62_9b9e92

Added from JY's HF PR21 at immutable revision 5d04d3b4b91dceb4c90e8592ce3235c1965c9db2; PR open at inventory. All 50 final tasks completed. No-preload track and original run contract retained. Cost is the sum of rounded per-response template estimates from final-task logs, not an invoice. Published-rate reconstruction independently agrees within the saved rounding bound. One prior infrastructure attempt is retained separately; the publisher's $16.43 total includes it, while this final-task row excludes its cost and usage.

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_48_4e20fb

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_41_6ef6f1

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_63_ccb407

Added from JY's HF PR21 at immutable revision 5d04d3b4b91dceb4c90e8592ce3235c1965c9db2; PR open at inventory. All 50 final tasks completed. No-preload track and original run contract retained. Cost is the sum of rounded per-response template estimates from final-task logs, not an invoice. Published-rate reconstruction independently agrees within the saved rounding bound.

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_67_b7d034

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_40_c6a678

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_81_c8b73b

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_87_ed8093

Publisher-designated corrected thinking-off/Pillow-fixed rerun; Pillow 11.3.0 installation is present in init.log. Recorded dollar cost and reasoning/cache token breakdown are unavailable, not zero. Separate reference-cost bounds use the cited OpenRouter rate snapshot and assume all versus no input cached; they are not billing.

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_83_d83faa

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges

Run: svc_49_c74b13

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_47_1eed89

API-equivalent estimate using recorded tokens and no input caching; not historical billing. Current OpenRouter public API-equivalent token price interval, not historical billing; lower assumes all input cached, upper none; excludes other charges

Run: svc_88_da1388

sum_of_rounded_per_response_template_estimates. not an invoice; excludes unlogged requests, startup credential probes, infrastructure and separate grading charges