Enhancing Machine Learning Interpretability: Tackling Semantic Uncertainties and Hallucinations

Introduction:

In the evolving field of machine learning, particularly in natural language generation (NLG), semantic uncertainties and hallucinations present significant challenges. These issues can distort the reliability of model outputs, leading to misinformation and eroding user trust. This blog explores the origins and impacts of these phenomena, introduces innovative metrics for their detection, and discusses strategies for improving model accuracy.




Understanding Semantic Uncertainties:

Semantic uncertainties occur when a model generates multiple plausible outputs for the same input, reflecting the inherent ambiguity in human language. The Team Llama, comprising Yash Shivhare, Arush Sachdeva, and Vrinda Agarwal, has explored this phenomenon extensively, suggesting innovative approaches for uncertainty estimation:

1.Metrics for Uncertainty Estimation:

ROUGE Scores: These assess the overlap between generated text and reference texts, helping gauge the quality of generated content.

p(True): This metric prompts the model to evaluate the truthfulness of its outputs, reflecting confidence in its responses.

AUROC Scores: Used to measure the model's ability to differentiate between correct and incorrect outputs effectively.

2. Semantic vs. Syntactic Similarity:

Understanding the distinction between syntax (structure) and semantics (meaning) is crucial. While traditional metrics like ROUGE focus on syntactic elements, real understanding necessitates evaluating semantic content.

3.Sequence Embeddings and Wasserstein Distance:

These tools help quantify semantic similarities and the effort needed to transform one text distribution into another, providing a deeper understanding of model-generated uncertainties.

Addressing Hallucinations:

Hallucinations in machine learning refer to instances where models generate irrelevant or incorrect information. This section details the causes, impacts, and strategies for mitigation: 

About Hallucinations:

Hallucinations can undermine the credibility of NLG applications, from everyday AI interactions to high-stakes fields like medicine or finance. They typically arise from the model’s reliance on flawed data or inadequate understanding.

Mitigation Strategies:

Improving model architecture, enhancing data quality, and refining training strategies are essential to reduce hallucinations. These steps help the model better understand context and produce more accurate outputs.

Semantic Entropy and Sequence Log Probability:

To better handle the intricacies of semantic uncertainty and hallucinations, we propose two advanced methodologies:

Semantic Uncertainties Explored by-

Generation of Multiple Answers:

By examining how models produce varied responses to the same query, we uncover insights into the diverse interpretations a model can generate, highlighting the rich semantic context embedded within.

Assessing Correctness and Semantic Similarity:

Using advanced metrics like Bidirectional Entailment, ROUGE-L scores, and clustering, we evaluate the accuracy and similarity of responses. This analysis helps in identifying patterns and ensuring semantic coherence, which is crucial for reliable model performance.

Semantic Entropy Score:

We introduce the Semantic Entropy Score—a comprehensive metric that integrates standard evaluations with lexical similarities and other nuanced measures. This metric offers a holistic view of semantic richness and model reliability. This new method clusters possible model outputs based on semantic equivalence, using a bidirectional entailment algorithm. It then estimates the entropy across these clusters to quantify uncertainty not just in sequence, but in underlying meaning.


This plot indicates the impact of different features on a model's decisions. Lexical Similarity and P True are shown to be more influential than Semantic Entropy and Average Confidence, highlighting their importance in the model’s predictive accuracy.

Efficacy of AUROC Scores:

A comparative analysis using AUROC scores across different models sheds light on their ability to handle semantic intricacies. This helps determine which models are best at managing semantic uncertainties.


This graph compares the effectiveness of different uncertainty metrics across various machine learning models based on their size. Semantic Entropy consistently performs better, especially in larger models, demonstrating its effectiveness in estimating uncertainties.

Understanding Hallucinations-

Contextualizing Hallucinations through Multiple Answers:

Similarly to exploring semantic uncertainties, generating multiple answers for a single question can help identify hallucinations—misleading or false information generated by a model.

Sequence Log Probability for Detection:

We employ sequence log probability to detect anomalies in model responses. This metric highlights when a model's output deviates significantly from expected patterns, indicating hallucinations. This approach involves disabling gradient calculations during model inference to conserve resources, generating outputs, and calculating log probabilities to determine the likelihood of each predicted token.

Quantifying Hallucinations with Sequence Log Probability Scores:

By calculating the sequence log probability score across different responses, we assess the extent of hallucinatory content in model outputs. This detailed analysis helps in identifying the specific characteristics of hallucinations.

Overall Hallucination Measure through Average Log Probability:

By aggregating individual log probabilities, we derive an average log probability measure that encapsulates the model's overall propensity for hallucinations. This metric is vital for refining models to reduce false information.

Project Workflow and Analysis:

The final part of our exploration relates the level of semantic uncertainty to hallucinations. Our hypothesis posits that models with lower semantic uncertainty (as measured by metrics like semantic entropy) tend to exhibit fewer hallucinations, a relationship underscored by low Wasserstein distances between uncertainty and hallucination distributions.


This plot portrays Wasserstein distance, also known as Earth Mover's distance (EMD), is a metric used to measure the dissimilarity between two probability distributions. It quantifies the minimum amount of work required to transform one distribution into another, considering the cost of moving each unit of mass from one point to another.

This 3D plot illustrates how changes in Semantic Entropy and Lexical Similarity affect model performance. It provides a visual representation of the relationships between these variables, helping understand their interdependencies.

Conclusion:

By adopting these sophisticated metrics and strategies, we can significantly enhance the interpretability and reliability of machine learning models. The journey towards minimizing semantic uncertainties and hallucinations not only advances the field of NLG but also ensures that AI systems remain robust, credible, and beneficial across various applications.


Snapshot Stories








Team: Team Llama

Team Members: 

Yash Shivhare(2021101105)

Arush Sachdeva (2023121008)

Vrinda Agarwal(2021101110)

Link to Video:


Link to report:



Comments

Post a Comment

Popular posts from this blog

Data when seen through the Inference Web of LLM

Image
Background Today I saw a post from this guy saying “I was at the Marines today” . A few mins later I saw a post from the same guy saying: “I love vada pavs”. And i thought to myself “Hey, This guy is probably from Mumbai, India” . The next thought that followed this was if I, a mere human being can deduce this from 2 posts, what can the LLMs do? LLMS are trained on massive data sources from the internet. Isn't it possible that they infer this information too? Introduction Large language models (LLMs) have revolutionized natural language processing tasks. They demonstrate amazing capabilities in understanding and generating human-like text. In this project, we decided to delve into the accuracies of inferential capabilities of various LLM models.  Ob jectives Inference in LLMs is a vast and interesting topic to dive into. For the sake of this project, we chose 3 main objectives that we would be focusing upon: Identify the accuracies of Inferential Capabilities of the LLM​ Exploring...
Read more

EAR-VM: Exploring Methods for Improving Adversarial Robustness of Vision Models

Image
  Abstract CNNs have many uses, particularly in the field of Computer Vision, however, their vulnerability to Adversarial attacks leaves a lot to be desired, particularly to their robustness to these kinds of attacks in order to make them more safe.  The misuse of adversarial attacks are a major threat to CNN vision models; for example, self-driving cars can be made to misinterpret road signs or signals, putting the passengers at significant risk of harm.  To address this vulnerability, we have attempted to modify the architecture of a CNN to add an auxiliary classification SVM which will determine the maximum margin in which these adversarial attacks will impact loss.   Also, interpretability is a key concept in understanding the decisions and outputs of modern networks. By interpreting the working of the model, we can curate better adversarial attacks, or make the model more robust.   Objectives Our goal is to first implement the At-SVM, and then to test it to...
Read more

Exploring and Quantifying Bias in VLMs

Image
Uncovering and Quantifying Bias in VLMs Abstract Our goal is to investigate biases present in vision language models and their impact on image text pronoun resolution. We aim to assess the performance of the CLIP model on the VisoGender benchmark and compare it with existing research findings. We seek to explore innovative approaches to mitigate biases, including novel ablation methods, to enhance model fairness and performance. Introduction Vision-language models (VLMs) play a crucial role in various applications, but their reliance on uncurated data can introduce biases. Evaluating and understanding these biases are essential for ensuring fairness and addressing ethical concerns. We present VisoGender, a dataset for benchmarking gender bias in VLMs. Our experiments assess bias in the CLIP model, focusing on resolution and retrieval biases. Additionally, we introduce an ablation experiment using DINO to evaluate spatial...
Read more