Quiz title: C3 Model Evaluation and Performance Metrics

Title: Q1 Multiple Choice
Points: 1
1. What is the main limitation of using accuracy as the only performance metric?
a) It's too difficult to calculate
*b) It can be misleading when classes are imbalanced
c) It only works for regression problems
d) It requires too much computational power
 
Title: Q2 Multiple Choice
Points: 1
2. In fraud detection, what does precision measure?
a) How many fraudulent transactions occurred in total
*b) Of all transactions flagged as fraud, how many were actually fraudulent
c) Of all actual fraud cases, how many were detected
d) The overall accuracy of the fraud detection system
 
Title: Q3 Multiple Choice
Points: 1
3. What does recall measure in a classification model?
a) The percentage of predictions that are correct
*b) Of all actual positive cases, how many were correctly identified
c) Of all positive predictions, how many were correct
d) The speed at which the model makes predictions
 
Title: Q4 Multiple Choice
Points: 1
4. What is the F1 score?
a) The average of precision and recall
*b) The harmonic mean of precision and recall
c) The difference between precision and recall
d) The product of precision and recall
 
Title: Q5 Multiple Choice
Points: 1
5. What does overfitting mean in machine learning?
a) The model is too simple to capture patterns
*b) The model memorizes training data but fails to generalize to new data
c) The model takes too long to train
d) The model uses too many features
 
Title: Q6 Multiple Choice
Points: 1
6. What is a key sign of overfitting?
a) Low accuracy on both training and test data
b) High accuracy on test data but low on training data
*c) High accuracy on training data but low on test data
d) Equal accuracy on training and test data
 
Title: Q7 Multiple Choice
Points: 1
7. What is the purpose of cross-validation?
a) To increase model accuracy
*b) To get a more reliable estimate of model performance on unseen data
c) To reduce training time
d) To eliminate the need for test data
 
Title: Q8 Multiple Choice
Points: 1
8. In k-fold cross-validation, what does 'k' represent?
a) The number of features in the dataset
*b) The number of parts the dataset is divided into
c) The number of models to train
d) The number of iterations to run
 
Title: Q9 Multiple Choice
Points: 1
9. What does AUC stand for in ROC-AUC?
a) Accuracy Under Curve
*b) Area Under Curve
c) Average Uniform Classification
d) Automated Uncertainty Calculation
 
Title: Q10 Multiple Choice
Points: 1
10. What does an AUC score of 0.5 indicate?
a) Perfect model performance
*b) Random performance (no better than chance)
c) Very poor model performance
d) Excellent model performance
 
Title: Q11 Multiple Choice
Points: 1
11. In a confusion matrix for binary classification, what are True Positives?
a) Cases incorrectly predicted as positive
*b) Cases correctly predicted as positive
c) Cases correctly predicted as negative
d) Cases incorrectly predicted as negative
 
Title: Q12 Multiple Choice
Points: 1
12. What is algorithmic bias in machine learning?
a) Errors in the algorithm's code
*b) Systematic unfairness in model predictions that disadvantages certain groups
c) The model's preference for certain types of data
d) Random errors in model predictions
 
Title: Q13 Multiple Choice
Points: 1
13. What is historical bias in machine learning data?
a) Using old data that is no longer relevant
*b) Training data that reflects patterns of past discrimination
c) Bias introduced by the algorithm itself
d) Errors in data collection methods
 
Title: Q14 Multiple Choice
Points: 1
14. What does demographic parity measure?
a) The accuracy of predictions across different groups
*b) Whether positive predictions occur at similar rates across different groups
c) The representation of different groups in the training data
d) The computational efficiency across different demographics
 
Title: Q15 Multiple Choice
Points: 1
15. What is the precision-recall trade-off?
a) You must choose between calculating precision or recall
*b) Improving one metric often leads to worse performance on the other
c) Precision and recall always move in the same direction
d) There is no relationship between precision and recall
 
Title: Q16 Multiple Choice
Points: 1
16. When would you prefer high recall over high precision?
a) When false positives are very costly
*b) When missing positive cases is very costly
c) When you have unlimited resources for investigation
d) When the dataset is perfectly balanced
 
Title: Q17 Multiple Choice
Points: 1
17. What is stratified cross-validation?
a) Validation that uses different algorithms
*b) Cross-validation that maintains class distribution in each fold
c) Validation performed in multiple stages
d) Cross-validation that uses stratified sampling
 
Title: Q18 Multiple Choice
Points: 1
18. What is regularization in machine learning?
a) Making the model run faster
*b) Adding constraints to prevent overfitting
c) Cleaning the training data
d) Standardizing input features
 
Title: Q19 Multiple Choice
Points: 1
19. What does a ROC curve plot?
a) Precision vs. Recall
*b) True Positive Rate vs. False Positive Rate
c) Accuracy vs. Training Time
d) Bias vs. Variance
 
Title: Q20 Multiple Choice
Points: 1
20. What is representation bias?
a) Bias in how results are presented
*b) When training data doesn't adequately represent all groups the model will serve
c) Bias introduced by the model architecture
d) Bias in the evaluation metrics
 
Title: Q21 Multiple Choice
Points: 1
21. What is the main advantage of ensemble methods?
a) They train faster than single models
*b) They combine multiple models to create more robust predictions
c) They require less data than single models
d) They are easier to interpret than single models
 
Title: Q22 Multiple Choice
Points: 1
22. What is measurement bias?
a) Errors in measuring model performance
*b) Inconsistencies in how data is collected across different groups
c) Bias introduced by the measurement tools
d) Random measurement errors
 
Title: Q23 Multiple Choice
Points: 1
23. Why is ongoing monitoring important after model deployment?
a) To improve model accuracy over time
*b) Model performance can degrade as conditions change
c) To reduce computational costs
d) To comply with legal requirements
 
Title: Q24 Multiple Choice
Points: 1
24. What is individual fairness in machine learning?
a) Treating each person as an individual case
*b) Similar individuals should receive similar predictions regardless of group membership
c) Each person should have their own model
d) Fairness should be evaluated on individual metrics
 
Title: Q25 Multiple Choice
Points: 1
25. What is the most important principle when evaluating machine learning models?
a) Always use the highest accuracy model
*b) Match evaluation metrics to business objectives and consider fairness
c) Use as many metrics as possible
d) Focus only on technical performance measures
 
