Chartered AI Engineer
Level 1
Chartered AI Engineers (Level 1) are working professionals who have the necessary technical skills to develop AI solutions. They are able to draw relationship between an AI project and its relevance to business, and design the solution to meet the deliverables.
Assessment Rubrics forChartered AI Engineer Level 1
1. Project Management Assessment
Project Due Diligence
Model Performance
Business Outcome
Project Due Diligence
- Clearly articulates how the technical solution addresses the stated objective/business need
- Provides details of the benefits achieved from the deployment
Model Performance
- Continuously evaluates model performance according to a clear set of metrics
- If monitoring has not commenced, is able to explain how it can be enabled
Business Outcome
- Understands who the end users are and is able to explain how the AI solution will benefit them
- Able to estimate the business impact of the AI solution
2. Exploratory Data Analysis Assessment
Derive Insights
Application of Insight
Derive Insights
- Performs EDA with good domain understanding and derives useful data insights
Application of Insight
- EDA results in relevant data cleaning or feature engineering
3. Data Preparation Assessment
Data Preprocessing
Feature Engineering
Train-Test Splitting
Data Preprocessing
- Makes appropriate choices of basic preprocessing techniques for outliers handling, missing data handling, erroneous data correction and other data transformation
Feature Engineering
- Feature engineering/input preprocessing reflects an understanding of the algorithm and ability to link to insights from EDA
Train-Test Splitting
- Performs train-test-validate data split for model building, linking it with insights from EDA to ensure that the final model is not overfitted and testing is unbiased
- Applies sampling method to ensure all classes are well represented in each dataset
- Ensures there is no data leakage with the train-test split approach used
4. Model Design and Development Assessment
Model Training
Evaluation Metrics
Modeling Parameter Tuning
Model Training
- Displays good conceptual understanding of ML algorithms and models
- Adequately and correctly presents the underpinning statistical or deep learning framework for the chosen algorithm
- Able to adapt current AI research to address problem statement
Evaluation Metrics
- Evaluates model performance using suitable metrics
- Able to explain the core concepts used in model selection, such as the trade-off between variance and bias
- Able to link metrics assessment back to business goals
Modeling Parameter Tuning
- Takes into account different modelling parameters and architectures for comparison
- Fine-tunes the model to achieve metrics that are comparable to current state-of-the-art models
- Includes parameter tuning methodology and is able to explain the experiments conducted and draw conclusion from the tuning
5. Deployment Assessment
Business / Operating Environment
Pipeline Deployment Workflow
ML Operations
Business / Operating Environment
- Addresses business/operating environment considerations when developing the AI solution
- Able to describe the high-level architecture
- Project data is organised and versioned
Pipeline Deployment Workflow
- Organises pipelines in a modularised fashion and adheres to software engineering best practices
- Automates or collaborates on the automation of pipelines for key tasks such as data ingestion, data preparation, model training, inferencing, metric evaluation and performance monitoring
- Sets up solution workflow environment and file structure to facilitate pipeline
ML Operations
- Understands how to set up a consistent infrastructure/platform to support ML development and deployment
- Able to gauge the need to scale data processing, model training and inferencing for the ML system, and to clarify how to scale where required
- Able to track system performance of ML deployments (using at least one identified metric/KPI)
6. AI Governance Assessment
Data Bias and Model Fairness
Data Bias and Model Fairness
- Able to explain assessments performed on the fairness of the ML models to mitigate data bias
7. Communication [During Interview]
Project Objective
Technical Concepts
Project Objective
- Able to elaborate the business objectives or goals of the project
Technical Concepts
- Able to explain the technical design and implementation of the project to a technical person
Downloads
CAIE Handbook
Sample Technical Report Template