2 Commits

Author	SHA1	Message	Date
jonathanortega2023	7a0c149d2e	fix: Use eval_duration for output TPS calculations in Ollama LLM provider (#4568) ... * fix: Use eval_duration for output TPS calculations and add as a metric field * refactor usage of eval_duration from ollama metrics * move eval_duration to usage * overwrite duration in ollama provider wip measureAsyncFunction optional param * allow for overloaded duration in measureAsyncFunction * simplify flow for duration tracking --------- Co-authored-by: shatfield4 <seanhatfield5@gmail.com> Co-authored-by: Timothy Carambat <rambat1010@gmail.com>	2025-11-20 13:02:47 -08:00
Timothy Carambat	dd7c4675d3	LLM performance metric tracking (#2825) ... * WIP performance metric tracking * fix: patch UI trying to .toFixed() null metric Anthropic tracking migraiton cleanup logs * Apipie implmentation, not tested * Cleanup Anthropic notes, Add support for AzureOpenAI tracking * bedrock token metric tracking * Cohere support * feat: improve default stream handler to track for provider who are actually OpenAI compliant in usage reporting add deepseek support * feat: Add FireworksAI tracking reporting fix: improve handler when usage:null is reported (why?) * Add token reporting for GenericOpenAI * token reporting for koboldcpp + lmstudio * lint * support Groq token tracking * HF token tracking * token tracking for togetherai * LiteLLM token tracking * linting + Mitral token tracking support * XAI token metric reporting * native provider runner * LocalAI token tracking * Novita token tracking * OpenRouter token tracking * Apipie stream metrics * textwebgenui token tracking * perplexity token reporting * ollama token reporting * lint * put back comment * Rip out LC ollama wrapper and use official library * patch images with new ollama lib * improve ollama offline message * fix image handling in ollama llm provider * lint * NVIDIA NIM token tracking * update openai compatbility responses * UI/UX show/hide metrics on click for user preference * update bedrock client --------- Co-authored-by: shatfield4 <seanhatfield5@gmail.com>	2024-12-16 14:31:17 -08:00