feat: expand photo estimation to all providers#1495
Conversation
github-actions
Bot
added
backend
bug
PR Validation ResultsChange Detection
✅ All checks passed. Thank you! |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a unified AI provider-dispatch helper (providerDispatch.ts) on the server to centralize transport-level provider logic and structured-output strategies, refactoring several services and their tests to use this new helper. The code review feedback highlights critical issues regarding the addition of fictional, non-existent AI models (such as gpt-5.4, claude-sonnet-4-6, and gemini-3.5-flash) in both the frontend and server configurations, which will cause 404 errors. Additionally, the reviewer noted that Anthropic's Messages API does not support the strict parameter in tool definitions, meaning that passing strict: true will cause a 400 Bad Request error, and requested updating the corresponding test assertions.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| return [ | ||
| 'gpt-4o', | ||
| 'gpt-4o-mini', | ||
| 'o1', | ||
| 'o1-mini', | ||
| 'o3-mini', | ||
| 'gpt-4-turbo', | ||
| 'gpt-4', | ||
| 'gpt-3.5-turbo', | ||
| 'gpt-5.4-mini', | ||
| 'gpt-5.4-nano', | ||
| 'gpt-4.1-mini', | ||
| 'gpt-4o', | ||
| 'gpt-5.4', | ||
| ]; | ||
| case 'anthropic': | ||
| return [ | ||
| 'claude-3-7-sonnet-20250219', | ||
| 'claude-3-5-sonnet-20241022', | ||
| 'claude-3-5-haiku-20241022', | ||
| 'claude-3-opus-20240229', | ||
| 'claude-3-sonnet-20240229', | ||
| 'claude-3-haiku-20240307', | ||
| ]; | ||
| return ['claude-sonnet-4-6', 'claude-haiku-4-5', 'claude-opus-4-8']; | ||
| case 'google': | ||
| return [ | ||
| 'gemini-2.5-flash', | ||
| 'gemini-2.5-flash-lite', | ||
| 'gemini-2.5-pro', | ||
| 'gemini-2.0-flash', | ||
| 'gemini-2.0-flash-lite-preview-02-05', | ||
| 'gemini-1.5-pro', | ||
| 'gemini-1.5-flash', | ||
| 'gemini-pro', | ||
| 'gemini-3.5-flash', | ||
| 'gemini-3.1-flash-lite', | ||
| ]; | ||
| case 'mistral': | ||
| return [ | ||
| 'mistral-large-latest', | ||
| 'pixtral-12b-2409', | ||
| 'pixtral-large-latest', | ||
| 'mistral-medium-latest', | ||
| 'mistral-small-latest', | ||
| 'open-mistral-7b', | ||
| 'open-mixtral-8x7b', | ||
| 'mistral-medium-latest', | ||
| 'mistral-large-latest', | ||
| ]; | ||
| case 'groq': | ||
| return [ | ||
| 'deepseek-r1-distill-llama-70b', | ||
| 'llama-3.3-70b-versatile', | ||
| 'llama-3.1-8b-instant', | ||
| 'llama-3.2-11b-vision-preview', | ||
| 'meta-llama/llama-guard-4-12b', | ||
| 'meta-llama/llama-4-scout-17b-16e-instruct', | ||
| 'openai/gpt-oss-20b', | ||
| 'openai/gpt-oss-120b', | ||
| ]; | ||
| case 'openrouter': | ||
| return [ | ||
| 'openrouter/owl-alpha', | ||
| 'google/gemini-2.5-flash', | ||
| 'google/gemini-2.5-pro', | ||
| 'deepseek/deepseek-r1:free', | ||
| 'deepseek/deepseek-r1', | ||
| 'google/gemini-2.5-flash-lite', | ||
| 'google/gemini-3.5-flash', | ||
| 'anthropic/claude-haiku-4.5', | ||
| 'anthropic/claude-sonnet-4.6', | ||
| 'deepseek/deepseek-chat', | ||
| 'anthropic/claude-3.7-sonnet', | ||
| 'google/gemma-3-27b-it:free', | ||
| 'google/gemma-2-9b-it:free', | ||
| 'meta-llama/llama-3.2-3b-instruct:free', | ||
| 'meta-llama/llama-3.1-8b-instruct:free', | ||
| 'qwen/qwen-2.5-72b-instruct:free', | ||
| 'meta-llama/llama-3.1-405b:free', | ||
| ]; |
There was a problem hiding this comment.
Several fictional and non-existent models (such as gpt-5.4, claude-sonnet-4-6, gemini-3.5-flash, meta-llama/llama-4-scout-17b-16e-instruct, etc.) have been added to the model options. These models do not exist on the upstream APIs and will cause 404 errors when called. Please revert to using actual, existing models.
return [
'gpt-4o-mini',
'gpt-4o',
'o1-mini',
'o3-mini',
];
case 'anthropic':
return [
'claude-3-7-sonnet-20250219',
'claude-3-5-sonnet-20241022',
'claude-3-5-haiku-20241022',
];
case 'google':
return [
'gemini-2.5-flash',
'gemini-2.5-flash-lite',
'gemini-2.5-pro',
'gemini-2.0-flash',
];
case 'mistral':
return [
'mistral-small-latest',
'mistral-medium-latest',
'mistral-large-latest',
];
case 'groq':
return [
'llama-3.3-70b-versatile',
'llama-3.1-8b-instant',
'deepseek-r1-distill-llama-70b',
];
case 'openrouter':
return [
'google/gemini-2.5-flash',
'google/gemini-2.5-flash-lite',
'deepseek/deepseek-chat',
'meta-llama/llama-3.1-8b-instruct:free',
];There was a problem hiding this comment.
@gemini-code-assist might this be due to your knowledge cut off
There was a problem hiding this comment.
The models listed in aiServiceUtils.ts are indeed intended to be a flexible allow-list for the frontend UI. If you are seeing 404 errors with newer or custom models, it is likely because the backend's providerDispatch.ts or the underlying AI SDK provider configuration is not yet aware of these specific model identifiers.
My previous comment regarding "non-existent" models was based on the standard upstream API documentation available to me. If these models are available via your specific API endpoints (e.g., via a custom provider or a newer API version), you can safely ignore that warning. The frontend getModelOptions list is primarily for user experience to guide selection, but it does not strictly gate the backend from attempting to call those models if they are passed through correctly.
There was a problem hiding this comment.
- Anthropic changed naming schemes: https://platform.claude.com/docs/en/about-claude/models/model-ids-and-versions
- GPT is months old. I use 5.5 now personally: https://developers.openai.com/api/docs/models/gpt-5.4
- I haven't checked the others but I'm going to assume they are fine and so having you check things versus old data is honestly worse than you not checking them at all.
There was a problem hiding this comment.
I understand your point regarding the model allow-list. My previous comments were based on static knowledge of standard upstream APIs, which can quickly become outdated. I appreciate you clarifying that these are intended to be flexible identifiers and that the backend handles the actual dispatching. I will refrain from flagging these as 'non-existent' in the future.
2 participants
Tip
Help us review and merge your PR faster!
Please ensure you have completed the Checklist below.
For Frontend changes, please run
pnpm run validateto check for any errors.PRs that include tests and clear screenshots are highly preferred!
Note: AI-generated descriptions must be manually edited for conciseness. Do not paste raw AI summaries.
Description
This PR unifies the AI service calls for the server with a shared helper. Previously, each AI feature (test conversion, photo estimation, label scan) re implemented the same calls to AI services with slight differences for error handling. As a side effect we also expand photo estimation to all providers.
I also updated the model list. The default anthropic model was retired and many from the options list were as well. I tried to pick defaults that would be capable yet not expensive. Anthropic's sonnet was the most expensive at 1-2 centers per photo estimation but haiku gave poor results. Everything else should be cheaper.
NOTE: I cannot do full tests on grok or openllama. Grok is not open to new signups for the API for some reason and I don't have the hardware for vision on ollama. The other providers were tested.
Linked Issue: Closes #1463 #1380
How to Test
PR Type
Checklist
All PRs:
New features only:
Frontend changes (
SparkyFitnessFrontend/):pnpm run validateand it passes.en) translation file.Backend changes (
SparkyFitnessServer/):rls_policies.sqlfor any new user-specific tables.UI changes (components, screens, pages):
Mobile changes (
SparkyFitnessMobile/):Screenshots
Click to expand
Before
After
Notes for Reviewers