--- title: "Introduction to TrustworthyMLR" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{Introduction to TrustworthyMLR} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) ``` # Why TrustworthyMLR? In modern Machine Learning, we often optimize for performance metrics like Accuracy or AUC. However, in sensitive domains like healthcare or finance, **reliability** is just as important. A model that changes its predictions significantly based on a small change in the training data or a tiny amount of input noise is not "trustworthy." `TrustworthyMLR` provides tools to quantify these dimensions of reliability. ## Installation ```r devtools::install_github("ahamza-msse25mcs/TrustworthyMLR") ``` ## Core Metrics ### 1. Stability Index The Stability Index measures how consistent a model is across different training runs or resamples. An index of 1.0 means the model is perfectly stable. ```{r setup} library(TrustworthyMLR) # Simulate 5 runs of predictions set.seed(42) base <- rnorm(100) preds <- matrix(rep(base, 5) + rnorm(500, sd = 0.1), ncol = 5) # Calculate Stability stability_index(preds) # Visualize Stability plot_stability(preds, main = "Prediction Stability Across Runs") ``` ### 2. Robustness Score The Robustness Score measures how sensitive a model is to input perturbations (noise). ```{r} # Define a simple linear model predict_fn <- function(X) X %*% c(1, -2, 3) # Data X <- matrix(rnorm(300), ncol = 3) # Robustness Score robustness_score(predict_fn, X, noise_level = 0.05) # Visualize Robustness Decay plot_robustness(predict_fn, X, main = "Robustness Decay Curve") ``` ## Conclusion By incorporating `TrustworthyMLR` into your validation pipeline, you ensure that your models are not only accurate but also reliable and robust enough for real-world deployment.