Commit 855e459
terp
feat: add Auto Router for per-task model selection
Adds an optional "Auto (smart routing)" picker entry (slug codex-auto) that
routes each Codex task to the cheapest configured model that can handle it. A
cheap classifier model scores every candidate 0.0-1.0 from a capability card;
the shim picks the cheapest candidate whose score clears a threshold (default
0.7), caches the decision per task, and falls back safely on any error so a
request never breaks. The classifier never sees price, so it can't be biased
toward expensive models.
- codex_shim/router.py: config loading, task-signal extraction, classifier
prompt, score parsing, cheapest-among-viable selection, per-task cache.
- server.py: applies the router on /v1/responses, /v1/responses/compact, and
/v1/chat/completions; runs the classifier over the configured backend; gates
the virtual model in /v1/models, /api/models, /health, and the picker.
- catalog.py + cli.py: catalog entry and `codex-shim list`/`model use` support.
- Configured via an optional `router` block in ~/.codex-shim/models.json. Env
knobs: CODEX_SHIM_DISABLE_ROUTER, CODEX_SHIM_ROUTER_TIMEOUT,
CODEX_SHIM_ROUTER_MAX_TOKENS, CODEX_SHIM_ROUTER_LOG.
- docs/AUTO_ROUTER.md, README section, and a runnable offline proof at
examples/auto_router_demo.py.
Verification:
- python3 -m pytest tests/ -q (113 passed)
- python3 -m compileall codex_shim/ examples/ -q
- python3 examples/auto_router_demo.py (RESULT: PASS)
Generated with [Devin](https://cli.devin.ai/docs)1 parent 14a05ba commit 855e459
10 files changed
Lines changed: 1572 additions & 8 deletions
File tree
- codex_shim
- docs
- examples
- tests
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
12 | 24 | | |
13 | 25 | | |
14 | 26 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
39 | 39 | | |
40 | 40 | | |
41 | 41 | | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
42 | 46 | | |
43 | 47 | | |
44 | 48 | | |
| |||
564 | 568 | | |
565 | 569 | | |
566 | 570 | | |
| 571 | + | |
| 572 | + | |
| 573 | + | |
| 574 | + | |
| 575 | + | |
| 576 | + | |
| 577 | + | |
| 578 | + | |
| 579 | + | |
| 580 | + | |
| 581 | + | |
| 582 | + | |
| 583 | + | |
| 584 | + | |
| 585 | + | |
| 586 | + | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
| 607 | + | |
| 608 | + | |
| 609 | + | |
| 610 | + | |
| 611 | + | |
| 612 | + | |
| 613 | + | |
| 614 | + | |
| 615 | + | |
| 616 | + | |
| 617 | + | |
567 | 618 | | |
568 | 619 | | |
569 | 620 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3 | 3 | | |
4 | 4 | | |
5 | 5 | | |
| 6 | + | |
6 | 7 | | |
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
| 11 | + | |
10 | 12 | | |
11 | 13 | | |
12 | 14 | | |
| |||
94 | 96 | | |
95 | 97 | | |
96 | 98 | | |
97 | | - | |
| 99 | + | |
98 | 100 | | |
99 | 101 | | |
| 102 | + | |
| 103 | + | |
100 | 104 | | |
101 | 105 | | |
102 | 106 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
| |||
27 | 28 | | |
28 | 29 | | |
29 | 30 | | |
| 31 | + | |
30 | 32 | | |
31 | 33 | | |
32 | 34 | | |
| |||
159 | 161 | | |
160 | 162 | | |
161 | 163 | | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
162 | 172 | | |
163 | 173 | | |
164 | 174 | | |
165 | 175 | | |
166 | 176 | | |
167 | 177 | | |
168 | | - | |
| 178 | + | |
| 179 | + | |
169 | 180 | | |
170 | 181 | | |
| 182 | + | |
| 183 | + | |
171 | 184 | | |
172 | 185 | | |
173 | 186 | | |
174 | 187 | | |
175 | 188 | | |
176 | 189 | | |
177 | 190 | | |
178 | | - | |
| 191 | + | |
| 192 | + | |
179 | 193 | | |
180 | 194 | | |
181 | 195 | | |
| |||
189 | 203 | | |
190 | 204 | | |
191 | 205 | | |
192 | | - | |
| 206 | + | |
193 | 207 | | |
194 | 208 | | |
195 | 209 | | |
| |||
200 | 214 | | |
201 | 215 | | |
202 | 216 | | |
| 217 | + | |
| 218 | + | |
| 219 | + | |
203 | 220 | | |
204 | 221 | | |
205 | 222 | | |
| |||
611 | 628 | | |
612 | 629 | | |
613 | 630 | | |
614 | | - | |
| 631 | + | |
| 632 | + | |
| 633 | + | |
615 | 634 | | |
616 | 635 | | |
617 | 636 | | |
| |||
783 | 802 | | |
784 | 803 | | |
785 | 804 | | |
786 | | - | |
| 805 | + | |
787 | 806 | | |
788 | 807 | | |
789 | | - | |
| 808 | + | |
790 | 809 | | |
791 | 810 | | |
792 | 811 | | |
793 | 812 | | |
794 | 813 | | |
| 814 | + | |
| 815 | + | |
795 | 816 | | |
796 | 817 | | |
797 | 818 | | |
| |||
853 | 874 | | |
854 | 875 | | |
855 | 876 | | |
856 | | - | |
| 877 | + | |
857 | 878 | | |
| 879 | + | |
| 880 | + | |
858 | 881 | | |
859 | 882 | | |
860 | 883 | | |
| |||
0 commit comments