Monitoring ML Model and LLM Traffic
Last updated
Was this helpful?
Last updated
Was this helpful?
Traffic as a service metric gives you basic insights into the operational health of your model's service in production.
Traffic — The volume of traffic received by the model over time.
Traffic is a basic high-level metric that informs us of the model's output activity.
A dip or spike in traffic needs to be investigated. For example, a dip could be due to a production model server going down; a spike could be an adversarial attack.
