SIMPLIFYAITOOLS · ONE MESSAGE, ONE DECISION
Measured API run
Task: route the same customer message to Billing, Technical Support or Sales.
| Provider | Configured model | Accuracy | Median time | Est. USD / 1,000 decisions | Successful / attempted | Cost coverage |
|---|---|---|---|---|---|---|
| jev | jev-1.13.0 | 100.0% (60/60) | 326.42 ms | 0.017894 | 60/60 | 60/60 |
| openai | gpt-4.1-mini-2025-04-14 | 100.0% (60/60) | 814.80 ms | 0.086740 | 60/60 | 60/60 |
Input cost includes any reported cached input at its configured rate. Output tokens can be reported even when their price is zero: Jev output is free at the saved rate. OpenAI input and output are charged separately.
| Provider | Input cost for this run | Output cost for this run | Total cost for this run | Mean cost per request | Projected cost per 1,000 requests | Known cost / attempted |
|---|---|---|---|---|---|---|
| jev | $0.001073646 | $0.000000000 | $0.001073646 | $0.000017894 | $0.017894100 | 60/60 |
| openai | $0.004594800 | $0.000609600 | $0.005204400 | $0.000086740 | $0.086740000 | 60/60 |
| Provider | Input rate | Cached input rate | Output rate | Rate checked |
|---|---|---|---|---|
| jev | 0.042 | Not configured | FREE ($0) | 2026-09-27 |
| openai | 0.4 | 0.1 | 1.6 | 2026-09-27 |
Cost = (uncached input tokens × input rate + cached input tokens × cached rate + output tokens × output rate) / 1,000,000. Rates come from this run's manifest, not today's configuration. Estimates are not invoices.
Time: median response time for successful requests, in milliseconds (1,000 ms = 1 second). This includes network and SDK processing, not just model computation. Failures are counted separately above.
Answers are shown in repeat order. Time is the median of successful repeats; cost is the mean per request when every repeat has known cost.
| Case | Customer message | Expected | Jev answers | Jev median ms | Jev USD / request | OpenAI answers | OpenAI median ms | OpenAI USD / request |
|---|---|---|---|---|---|---|---|---|
| C01 | My card was charged twice for the same subscription. Please investigate the duplicate charge. | billing | billing / billing / billing | 322.62 | 0.00001785 | billing / billing / billing | 813.32 | 0.00008600 |
| C02 | The dashboard shows a blank screen whenever I sign in. Please fix it. | technical_support | technical_support / technical_support / technical_support | 315.88 | 0.00001781 | technical_support / technical_support / technical_support | 829.81 | 0.00008720 |
| C03 | We are considering your product for a team of 30. Can you arrange a demo? | sales | sales / sales / sales | 367.29 | 0.00001798 | sales / sales / sales | 748.57 | 0.00008680 |
| C04 | Please send a copy of the invoice for my payment last month. | billing | billing / billing / billing | 310.29 | 0.00001772 | billing / billing / billing | 1141.40 | 0.00008480 |
| C05 | The password reset link says it has expired, even when I open it immediately. | technical_support | technical_support / technical_support / technical_support | 325.72 | 0.00001785 | technical_support / technical_support / technical_support | 779.11 | 0.00008760 |
| C06 | Which plan should I buy if I need access for ten employees? | sales | sales / sales / sales | 314.56 | 0.00001772 | sales / sales / sales | 890.43 | 0.00008480 |
| C07 | I cancelled yesterday but was billed again today. Please refund that charge. | billing | billing / billing / billing | 329.30 | 0.00001777 | billing / billing / billing | 866.92 | 0.00008520 |
| C08 | Your integration returns error 500 every time it tries to sync a contact. | technical_support | technical_support / technical_support / technical_support | 321.63 | 0.00001789 | technical_support / technical_support / technical_support | 882.47 | 0.00008720 |
| C09 | Can you give us a quote for 100 new seats? | sales | sales / sales / sales | 336.34 | 0.00001777 | sales / sales / sales | 774.45 | 0.00008440 |
| C10 | Please cancel my paid subscription at the end of the current billing period. | billing | billing / billing / billing | 307.99 | 0.00001777 | billing / billing / billing | 801.70 | 0.00008520 |
| C11 | Exporting my report produces an empty file. I need help fixing the export. | technical_support | technical_support / technical_support / technical_support | 341.22 | 0.00001789 | technical_support / technical_support / technical_support | 848.25 | 0.00008760 |
| C12 | Before purchasing, I want to know whether your enterprise plan includes onboarding. | sales | sales / sales / sales | 340.69 | 0.00001789 | sales / sales / sales | 791.18 | 0.00008520 |
| C13 | I received a receipt for $80 but my agreed subscription price was $40. Please correct the amount. | billing | billing / billing / billing | 370.48 | 0.00001814 | billing / billing / billing | 853.85 | 0.00008800 |
| C14 | I can log in on desktop, but the mobile app crashes as soon as I open it. | technical_support | technical_support / technical_support / technical_support | 331.09 | 0.00001798 | technical_support / technical_support / technical_support | 813.85 | 0.00008880 |
| C15 | We have not purchased yet. Do you offer annual contracts or volume discounts? | sales | sales / sales / sales | 321.51 | 0.00001781 | sales / sales / sales | 742.88 | 0.00008560 |
| C16 | The app problem is already resolved. I only need a refund for the duplicate payment. | billing | billing / billing / billing | 307.61 | 0.00001789 | billing / billing / billing | 774.97 | 0.00008640 |
| C17 | The invoice amount is correct, but clicking Download Invoice crashes the page. Please fix that button. | technical_support | technical_support / technical_support / technical_support | 319.38 | 0.00001798 | technical_support / technical_support / technical_support | 815.75 | 0.00008880 |
| C18 | Our payment method works fine. We want to purchase 20 additional seats; please send a quote. | sales | sales / sales / sales | 328.72 | 0.00001806 | sales / sales / sales | 1036.34 | 0.00008760 |
| C19 | I do not need a demo. I need someone to explain an unfamiliar charge on my existing account. | billing | billing / billing / billing | 339.85 | 0.00001802 | billing / billing / billing | 901.68 | 0.00008760 |
| C20 | Sales already helped us choose a plan. Now our API requests fail with an authentication error despite a valid key. | technical_support | technical_support / technical_support / technical_support | 339.94 | 0.00001810 | technical_support / technical_support / technical_support | 767.99 | 0.00009000 |