iSQI Certified Tester AI Testing CT-AI Exam Practice Test

Answer : A

Exploratory Data Analysis (EDA) is an essential technique for examining datasets to uncover patterns, trends, and anomalies, including outliers. In this case, the attacker manipulates the spam filter by injecting emails with red flags and masking them as internal company emails. The primary goal of EDA here is to detect these adversarial modifications.

Detecting Outliers:

EDA techniques such as statistical analysis, clustering, and visualization can reveal patterns in email metadata (e.g., sender details, email content, frequency).

Outlier detection methods like Z-score, IQR (Interquartile Range), or machine learning-based anomaly detection can identify emails that significantly deviate from typical internal communications.

Identifying Distribution Shifts:

By analyzing the frequency and characteristics of emails flagged as spam, testers can detect if the attack has introduced unusual patterns.

If a surge of internal emails is suddenly classified as spam, EDA can help verify whether these classifications are consistent with historical data.

Feature Analysis for Adversarial Patterns:

EDA enables visualization techniques such as scatter plots or histograms to distinguish normal emails from manipulated ones.

Examining email metadata (e.g., changes in headers, unusual wording in email bodies) can reveal adversarial tactics.

Counteracting Adversarial Attacks:

Once anomalies are identified, the spam filter's detection rules can be improved by retraining the model on corrected datasets.

The adversarial examples can be added to the training data to enhance the robustness of the filter against future attacks.

Reference from ISTQB Certified Tester AI Testing Study Guide

Exploratory Data Analysis (EDA) is used to detect outliers and adversarial attacks. 'EDA is where data are examined for patterns, relationships, trends, and outliers. It involves the interactive, hypothesis-driven exploration of data.'

EDA can identify poisoned or manipulated data by detecting anomalies and distribution shifts. 'Testing to detect data poisoning is possible using EDA, as poisoned data may show up as outliers.'

EDA helps validate ML models and detect potential vulnerabilities. 'The use of exploratory techniques, primarily driven by data visualization, can help validate the ML algorithm being used, identify changes that result in efficient models, and leverage domain expertise.'

Thus, option A is the correct answer, as EDA is specifically useful for detecting outliers, which can help identify manipulated spam emails.

Answer : C

For AI-enabled medical devices, the most required aspects for certification are safety, transparency, and side effects. Here's why:

Safety (Aspect III): Critical for ensuring that the AI system does not cause harm to patients.

Transparency (Aspect IV): Important for understanding and verifying the decisions made by the AI system.

Side Effects (Aspect V): Necessary to identify and mitigate any unintended consequences of the AI system.

Why Not Other Options:

Autonomy and Maintainability (Aspects I and II): While important, they are secondary to the immediate concerns of safety, transparency, and managing side effects in life-critical processes.

Answer : A

Data mislabeling occurs for several reasons, which can significantly impact the performance of machine learning (ML) models, especially in supervised learning. According to the ISTQB Certified Tester AI Testing (CT-AI) syllabus, mislabeling of data can be caused by the following factors:

Random errors by annotators -- Mistakes made due to accidental misclassification.

Systemic errors -- Errors introduced by incorrect labeling instructions or poor training of annotators.

Deliberate errors -- Errors introduced intentionally by malicious data annotators.

Translation errors -- Occur when correctly labeled data in one language is incorrectly translated into another language.

Subjectivity in labeling -- Some labeling tasks require subjective judgment, leading to inconsistencies between different annotators.

Lack of domain knowledge -- If annotators do not have sufficient expertise in the domain, they may label data incorrectly due to misunderstanding the context.

Complex classification tasks -- The more complex the task, the higher the probability of labeling mistakes.

Among the answer choices provided, 'Lack of domain knowledge' (Option A) is the best answer because expertise is essential to accurately labeling data in complex domains such as medical, legal, or engineering fields.

Certified Tester AI Testing Study Guide Reference:

ISTQB CT-AI Syllabus v1.0, Section 4.5.2 (Mislabeled Data in Datasets)

ISTQB CT-AI Syllabus v1.0, Section 4.3 (Dataset Quality Issues)

Answer : C

AI-based systems often exhibit non-deterministic behavior, meaning they do not always produce the same output for the same input. This makes ensuring safety more difficult, as the system's behavior can change based on new data, environmental factors, or updates.

Why Non-determinism Affects Safety:

In traditional software, the same input always produces the same output.

In AI systems, outputs vary probabilistically depending on learned patterns and weights.

This unpredictability makes it harder to verify correctness, reliability, and safety, especially in critical domains like autonomous vehicles, medical AI, and industrial automation.

Why Other Options Are Incorrect:

A (Simplicity): AI-based systems are typically complex, not simple, which contributes to safety challenges.

B (Sustainability): While sustainability is an important AI consideration, it does not directly affect safety.

D (Robustness): Lack of robustness can make AI systems unsafe, but non-determinism is the primary issue that complicates safety verification.

Supporting Reference from ISTQB Certified Tester AI Testing Study Guide:

ISTQB CT-AI Syllabus (Section 2.8: Safety and AI)

'The characteristics of AI-based systems that make it more difficult to ensure they are safe include: complexity, non-determinism, probabilistic nature, self-learning, lack of transparency, interpretability and explainability, lack of robustness'.

Conclusion:

Since non-determinism makes AI behavior unpredictable, complicating safety assurance, the correct answer is C.

Answer : D

Pre-processing is an essential step in data preparation that ensures data is clean, formatted correctly, and structured for effective machine learning (ML) model training. One common issue that can be resolved during pre-processing is numbers stored as strings.

Explanation of Answer Choices:

Option A: Insufficient data

Incorrect. Pre-processing cannot resolve insufficient data. If data is lacking, techniques like data augmentation or external data collection are needed.

Option B: Invalid data

Incorrect. While pre-processing can identify and handle some forms of invalid data (e.g., missing values, duplicate entries), it does not resolve all invalid data issues. Some cases may require domain expertise to determine validity.

Option C: Wanted outliers

Incorrect. Pre-processing usually focuses on handling unwanted outliers. Wanted outliers may need to be preserved, which is more of a data selection decision rather than pre-processing.

Option D: Numbers stored as strings

Correct. One of the key functions of data pre-processing is data transformation, which includes converting incorrectly formatted data types, such as numbers stored as strings, into their correct numerical format.

ISTQB CT-AI Syllabus Reference:

Data Pre-Processing Steps: 'Transformation: The format of the given data is changed (e.g., breaking an address held as a string into its constituent parts, dropping a field holding a random identifier, converting categorical data into numerical data, changing image formats)'.

Answer : B

AI systems, particularly machine learning models, need to exhibit adaptability and flexibility to handle slight variations in input data without requiring retraining. The ISTQB CT-AI syllabus outlines adaptability as a crucial feature of AI systems, especially when the system is exposed to variations in its operational environment.

Analysis of the Answer Options:

Option A: ''It has been trained to recognize cats and is given an image of a dog.''

This scenario introduces an entirely new class (dogs), which is outside the AI system's expected scope. If the AI was only trained to recognize cats, it would not be expected to recognize dogs correctly without retraining. This does not demonstrate adaptability as expected from an AI system.

Option B: ''It has been trained to recognize human faces at a particular resolution and it is given a human face image captured with a higher resolution.''

This is an example of an AI system encountering a variation of its training data rather than entirely new data. Most AI-based image processing models can adapt to different resolutions by applying downsampling or other pre-processing techniques. Since the data remains within the domain of human faces, the model should be able to process the higher-resolution image without significant issues.

Option C: ''It has been trained to analyze mathematical models and is given a set of landscape pictures to classify.''

This represents a complete shift in the data type from structured numerical data to unstructured image data. The AI system is unlikely to adapt effectively, as it has not been trained on image classification tasks.

Option D: ''It has been trained to analyze customer buying trend data and is given information on supplier cost data.''

This introduces a significant domain shift. Customer buying trends focus on consumer behavior, while supplier cost data relates to pricing structures and logistics. The AI system would likely require retraining to process the new data meaningfully.

ISTQB CT-AI Syllabus Reference:

Adaptability Requirements: The syllabus discusses that AI-based systems must be able to adapt to changes in their operational environment and constraints, including minor variations in input quality (such as resolution changes).

Autonomous Learning & Evolution: AI systems are expected to improve and handle evolving inputs based on prior experience.

Challenges in Testing Self-Learning Systems: AI systems should be tested to ensure they function correctly when encountering new but related data, such as different resolutions of the same object.

Thus, option B is the best choice as it aligns with the adaptability characteristics expected from AI-based systems.

Answer : A

Threshold coverage is a specific type of coverage measure used in neural network testing. It ensures that each neuron in the network achieves an activation value greater than a specified threshold. This is particularly relevant to the scenario described, where testers verify that neurons activate only when the future value of the commodity exceeds the spot price by at least 0.75%.

Why is Threshold Coverage Correct?

Threshold-based activation: The test case in the question is explicitly verifying whether neurons activate only when a certain threshold (0.75%) is exceeded. This aligns perfectly with the definition of threshold coverage.

Common in Neural Network Testing: Threshold coverage is used to measure whether each neuron in a neural network reaches a specified activation value, ensuring that the neural network behaves as expected when exposed to different test inputs.

Precedent in Research: The DeepXplore framework used a threshold of 0.75% to identify incorrect behaviors in neural networks, making this coverage criterion well-documented in AI testing research.

Why Other Options are Incorrect?

(B) Neuron Coverage

Neuron coverage only checks whether a neuron activates (non-zero value) at some point during testing. It does not consider specific activation thresholds, making it less precise for this scenario.

(C) Sign-Change Coverage

This coverage measures whether each neuron exhibits both positive and negative activation values, which is not relevant to the given scenario (where activation only matters when exceeding a specific threshold).

(D) Value-Change Coverage

This coverage requires each neuron to produce two activation values that differ by a chosen threshold, but the question focuses on whether activation occurs beyond a fixed threshold, not changes in activation values.

Reference from ISTQB Certified Tester AI Testing Study Guide

Threshold coverage ensures that neurons exceed a given activation threshold 'Full threshold coverage requires that each neuron in the neural network achieves an activation value greater than a specified threshold. The researchers who created the DeepXplore framework suggested neuron coverage should be measured based on an activation value exceeding a threshold, changing based on the situation.'

Thus, option A is the correct answer, as threshold coverage ensures the neural network's activation is correctly evaluated based on the required condition (0.75%).