Chartered AI Engineer
Level 2
Chartered AI Engineers (Level 2) are Working professionals who are team leads for AI engineers. They are able to manage projects as well as business stakeholders, both internal and external. They can see how the projects fit into the department’s business process and needs and are able to set up the appropriate process to monitor the effectiveness of the AI models that are implemented. They can architect the necessary AI modules to ensure successful deployment of machine learning models.
Assessment Rubrics for
Chartered AI Engineer Level 2
1. Project Management Assessment
Project Due Diligence
Model Performance
Business Outcome
Business Collaboration Outcome
Project Due Diligence
- Displays good understanding of the project development process
- The benefits achieved from the deployment are detailed clearly, together with objective evidence of success as well as considerations for future needs and/or scalability
- Assumptions behind project performance (including models) are well articulated, with built-in plans to discover and recover
- Able to manage data-to-insight latency to meet application requirements
Model Performance
- Continuously evaluates model performance according to a clear set of metrics
- Manages model performance issues consistently and makes plans for improvement
Business Outcome
- Understands who the end users are
- Able to explain how the AI solution will benefit end users and manage end users’ expectations
- Able to explain the business value and risks of the AI solution
Business Collaboration Outcome
- Works effectively with business stakeholders to co-develop AI solutions and drive business adoption
- Able to educate business stakeholders and add value to the business proposition of the AI solution
2. Solution Design and Development Assessment
Business / Operating Environment
Model / Pipeline Deployment Workflow
ML Operations
AI Technical Competencies
Business / Operating Environment
- Understands and able to analyse business needs, and play back the requirements clearly to stakeholders to gain their confirmation
- Able to design a solution that addresses the problem statement and reflects business and practical realities
- Employs a user-centred approach to designing and developing the integrated AI solution, taking into account ease of use, human-in-the-loop processes, and change management
Model / Pipeline Deployment Workflow
- Ensures that all stages of the model development process – including data pipeline engineering and data model design, model integration and test planning – are sound in terms of quality and efficiency
- Implements controls to track and manage input data quality
- Able to synthesise and integrate technologies for AI solution implementation, testing and deployment with knowledge of the trade-offs
- Able to define and implement a management and security plan for project data
ML Operations
- Able to clarify the budget for the ML system such as the cost of data collection and data processing
- Able to clarify issues related to scaling ML system
- Plans for change in the technology stack of ML system
- Able to set up infrastructure-as-code and automation to support ML development and deployment workflows
- Provides inputs for the procurement of ML infrastructure
- Able to interpret the output and the anomalies of ML system to business leaders
- Able to propose relevant metrics and track the performance of the ML system
AI Technical Competencies
- Demonstrates technical competencies needed to engineer a high-performance AI solution, such as computational modelling, computer vision, text analytics, self-learning systems etc
3. AI Governance Assessment
Data Bias and Model Fairness
Solution Robustness
Data Bias and Model Fairness
- Able to explain the consequences of flawed data and model bias
- Able to explain the steps taken to reduce model bias during solution development
- Assesses the fairness of ML models to mitigate data bias
Solution Robustness
- Takes into account regulations that ensure the security and safety of users and systems throughout the lifecycle of the AI product
4. Leadership Assessment
Team Management
Resource Management
Team Management
- Provides a clear plan for, and evidence of, successful team coordination and management during project development
- Demonstrates conflict resolution capabilities
Resource Management
- Provides evidence of successful planning and management of resources and budgets for the team
5. Communication [During Interview]
Project Objective
Technical Concepts
AI Business Value
Project Objective
- Able to elaborate the business objectives or goals of the project
Technical Concepts
- Able to explain technical design and implementation of the project clearly to non-technical person
AI Business Value
- Able to summarize the benefits and limitations of the AI solution to solving business challenges
Downloads
CAIE Handbook
Sample Technical Report Template