ML View

The ML View of the konaAI application features Automated Machine Learning which allows you to train high-quality models without requiring an in-depth understanding of machine learning. It handles activities such as data preparation, feature selection, model building and model evaluation.

ML View consists of three main features:

• Active Model
• Training Questionnaire 
• Modelling Experiment 
• History

Active Model

This screen dives deep into the inner workings of our deployed model. Here, you will find valuable insights into the model's training process, its effectiveness, and how different factors contribute to its predictions. Model training evaluation matrix for details on training data and performance metrics like accuracy and F1 score can be found while feature importance sheds light on which factors have the most significant influence on the model's predictions. 

Steps to perform in the Active Model tab 

• Once you are on Active model tab in Model Training part, last hedging is Contribution to Risk Scoring.  
• After the development of the active model, the next step is prediction scoring. Once the scoring is completed, two columns—'AutoML Prediction' and 'Total Risk Scoring'—will be added.  
• The 'AutoML Prediction' column will show '1' for a hit.  
  • If the transaction has an 'AutoML Prediction' of 1, the 'Total Risk Score' should be updated based on the respective contribution of each risk score percentage, relative to the current risk score. 
• Set the “Contribution to Risk Score” by accounting for all applicable factors.  
• After desired risk contribution is selected, click on the update button. 

Training Questionnaire

This section is mainly used for gathering data for training the models through closed alerts. This function operates through a supervised machine-learning classification model in which the model tries to understand the relationship between the independent and dependent variables.

In our case independent variable are the various test scenario and dependent variable is the conclusion of the alert. As an example – the conclusion could be a concern or no concern.  

Steps to perform in the Training Questionnaire tab. 

• Alert Status should always be closed. 
• It is mandatory to select one question with its appropriate option for each "Concern" and "Non-concern" Transactions Section. 
  • The questions visible in the "Concern" and "Non-Concern" sections are from the entire alert questionnaire, which includes all questions, checkboxes, and radio buttons. 
• If the same question is selected for "Concern" and "Non-concern" Transactions Sections then the option selected should be different for both the section. 
• Once the question and option selections are done, then we should save it by clicking on the “SAVE” button on the top right. 
• After saving, you will move to the Modeling Experiment tab. 

Modelling Experiment

This section works as the training ground for building powerful classification models. Data from prior modules are leveraged to train the classification models. In addition, decision-based models like AdaBoost, Decision Tree, and Random Forest have been used to uncover patterns by splitting data into branches based on features.

Further, ensemble methods have been used to combine multiple weaker models into a single, highly accurate predictor like Gradient Boosting, XGBoost, and LightGBM. Logistic Regression has been used for binary classification, estimating the probability of an event occurrence.

Steps to perform in the Modelling Experiment tab 

• Select questions in the Questionnaire tab. 
  • The minimum data to set is 60 closed transactions, with at least 2 transactions in each of the 'Concern' and 'Non-Concern' sections. 
• Click on Train Model button to start the experiment. 
  • This will change the status to "In Progress" until the training is complete. Once completed, the status will be marked as either "Complete" or "Failed." 
    • If the status is "Failed," click on the icon to see the reason for failure. 
  • This action will trigger an automated email to all users whose emails have been configured with the application. 
• Once the experiment is completed it will be added to the experiment list.  
  • The module, submodule, date, and time will be shown in the name of the experiment. 
  • Users with configured email addresses will receive an email notification when the training is complete. 
• Select the experiment from the Select Experiment drop-down list. 
  • Once the experiment is selected, the experiment Summary table will be displayed. 

• The Experiment Summary provides a list of 7 classification models and with details pertaining to model accuracy, balanced Accuracy, F1, precision, recall, and seconds to Train for all 7 models. 
  • AdaBoost  
  • Decision Tree 
  • Random Forest 
  • Gradient Boosting 
  • XGBoost 
  • LightGBM 
  • Logistic Regression 
• Once you have selected the desired experiment and viewed the experiment summary, select the best machine learning model from the select model drop-down list.  
• Selecting the desired machine learning model, will display a matrix about the model.    
  • Model Training - this part deals with the matrix related to model training like model training date, total concern and non-concern records used for training, trained model accuracy, F1, Precision, Recall, etc.  

• • Confusion Matrix - A confusion matrix is a two-dimensional table that shows the performance of a machine learning model. It's also known as an error matrix and is used to evaluate the model's errors, weaknesses, and performance.  
    • The matrix has rows that represent actual classes, and columns that represent predicted classes 

• • Feature Importance – The number of features considered for building the selected model are displayed in a bar graph in a descending order of their importance. 

• Once you have selected the best ML model, deploy it by clicking on the deploy button, beside the select model drop-down.  

History 

You will now be able to view the history of each action performed on the model on this page. This tab reveals a drop-down grid containing the actions performed in active model, training questionnaire and modelling experiments tabs.  

The grid displays a list of who initiated the changes, when it was done, the sub-module the changes belong to etc. Similarly, for modelling experiments and training questionnaire, you will be able to view all information pertaining to the experiment including the status in a card view on your screen when you click on the respective drop-down option. 

At a given time, only 5 records are displayed on the screen, use the page navigation to access older modifications done to the model.