Model maintenance as a service
AI Solution Reliability Monitoring & Maintenance
Unattended models degrade over time. Continuous AI Solution Reliability monitoring and maintenance is a critical component to assure a consistent performance and quality over the lifetime of an AI model.
Continuous learning as insurance on your AI solution quality
When AI solutions are operationalized you risk that the reliability of your AI solution degrades over time. This is inherent in how AI works. Being able to implement continuous quality control in your AI lifecycle is critical in operationalizing AI solutions in production.
Either caused by human manipulation or model drift, Faktion’s Continuous Learning services are used by our customers to maintain the performance and quality of your solution at the highest standards.
This integrated maintenance approach can be considered as post-delivery quality assurance and is offered as a service level agreement.Â
Being an integral part of your overall AI lifecycle control strategy, Faktion will continuously monitor your solutions’ reliability and perform periodic or continuous model (re)training in order to ensure your AI solution to perform at the predefined quality KPI’s.
Benchmarking and model quality monitoring
Faktion offers both active and passive benchmarking and model quality monitoring, enabling either a continuous or a triggered approach. Both offerings come with a full chain-of-custody and traceability of change, but have each a different set of constraints and advantages.
Where a continuous approach (passive) provides a pervasive level of quality assurance, strict monitoring quality and roll-back controls are required. In case a triggered approach (active) is used, an efficient release management process is required.
Active
Passive
Active
The active benchmarking and model quality monitoring service, entails using both manual and automated solutions to continuously monitor the concept drift of your AI solution, but are triggered when explicit and significant changes are detected on the model.Â
Upon detection, newer data is verified, and used to refresh the model and substitute the old, and degraded, version.
Passive
In a passive context, the benchmarking and quality monitoring is performed continuously retraining the model on the most recently observed samples at regular intervals.Â
In this case, “circuit-breakers” and roll-back mechanisms are implemented in order to prevent malicious or deliberate model drift situations.
Integrating your model maintenance into your software development lifecycle
Predictive model maintenance is something that needs to be done and considered before you even build your models. Depending on your approach, it also has to be integrated into your software development lifecycle.
When developing predictive models, the entire development team’s involvement is required. As those who build the models may not be the same team who implements and maintains them. A fully integrated approach is therefore one of the key success criteria.
Finally, and as models are periodically refreshed new algorithms or a different set of features might be discovered, that provide improved accuracy, and could be used in reviewing or redesigning even large parts of the AI solution.
