feat: gpt caching - #80
Conversation
|
AI review: The selection logic seems quite complicated, as evidenced in the bullet points above... can we boil it down to easier criteria? |
|
Works for anthropic provider. For aicore provider, it works with gpt-5.5 and anthropic models. For other gpt models ( Also for aicore, I see these in the log: |
|
The mlflow errors are fixed in the other PR. gpt 5.2 is not available with version latest in AI core causing the error and got-5.6-sol did not accept temperature = 0. |
| options.params || (deepAgent ? { max_tokens: 4096, temperature: 0 } : cds.env.agents?.params) | ||
| const params = withPromptCachingParams(model, rawParams) | ||
| if (params.temperature === 0 && noTemp0Support(model)) { | ||
| params.temperature = 1 |
There was a problem hiding this comment.
Setting temperature to 1 ourselves is a bad idea. We should ideally not have to provide it at all
There was a problem hiding this comment.
Fair point. Deleting it should work as well and I switched to that now.
Prompt caching is now also applied to GPT models.
Have you...