As more and more predictive models are being developed and machine learning is added to the mix, a new management and governance issue is starting to show. Although many companies are starting to look into big data and analytics, at Kentivo we start already experiencing the challenges associated with managing it all. The challenges are bigger than the traditional master data management and data governance challenges. This is due to the fact that with predictive models and machine learning, new data gets created and this created data starts getting used in core business processes. Add time dependency and real-time adaptation in the mix and a really interesting challenge is apparent.
When a master data management system is off-line, the situation is less critical as copies of the data reside in the system. A predictive model creating a next-best offer for an online customer trying to order can have more impact.
Challenges in Predictive Model Curating
The challenges in curating predictive models are driven
- Business Processes & Impact: In which processes are the models used and what is the impact. What happens if a business process is changed? How do we handle business processes that transcend a department or even the own organisation
- Version Management: Over time, multiple versions of models can be in used as upgrade cycles of systems might differ and the same predictive models are used in different processes.
- Quality Management: The qualty of models in productions might different and over-time many assumptiosn made, might no longer be valid. Also, is a new model better or not than the current one and what is the impact of replacing it? Is the model robust enough for the purpose and should we upgrade.
- Trust Management: Models make predictions, but the accuracy of the predictions might change over time. Can a model still be used or does it need re-calibration? Should it be replaced because it starts predicting none-sense within month as it moves beyond its intended purposes.
- Upgrading & Patching: What do we do with upgrades and patches? Do we recalculate all values are do we calculate only new cases coming in. What do we do with past feedback generated by feedback loops?
These challenges are important when implementing predictive models and machine learning.
How KENTIVO helps
At KENTIVO we take a pragmatic approach to this and help our clients implement the governance processes. Our consultants help make the choices explicit and advice based on the impact it has on a clients situation.
However, we not only help the clients by providing advice, we also develop a predictive management platform. This development is an R&D project carried out in conjunction with the projects we do for our clients. Both we and our clients benefit as this workbench gets stronger over time. To clients where we have multi-year contracts, to support them in managing predictive models and machine learning, the platform is an integral part of the service.