Defining and Detecting Hallucinations in LLM's

`

 



DEFINING and DETECTING HALLUCINATIONS IN LARGE LANGUAGE MODELS

HALLUCINATIONS

Hallucinations occur when a model generates text that deviates from the input context or presents inaccurate information.

Example:

Input Context:
Prompt: "Describe the life cycle of a butterfly."

Hallucinated Text:
"The life cycle of a butterfly begins with a caterpillar hatching from an egg. Then, it spins a cocoon and transforms into a unicorn."

In this example, the hallucination occurs when the Language Model generates text that deviates from the factual information provided in the input context. While the first part of the generated text aligns with the prompt by describing the initial stages of a butterfly's life cycle, the mention of transforming into a unicorn is a hallucination—a fabricated detail not grounded in reality.

Hallucinations as we here define are: 

  •  A property of the training data. 
  •  Also a function of the input context. 
  •  Also a function of the model architecture, which we are exploring here.

Entailment : Response is in line with the reference.
Contradiction: Response contradicts the reference
Neutral: Cannot reach a conclusion, response is neutral.

Why Hallucinations Occur?

Hallucinations in Language Models can happen due to biases in training data, ambiguity in input context, limitations in understanding complex scenarios, model architecture, and the complexity of language itself.

Challenges & Motivation:

In the dynamic realm of Language Models (LLMs), the phenomenon of hallucinations poses a significant challenge. Addressing this issue is vital for enhancing the credibility and dependability of LLM's across diverse applications. Ensuring the accuracy and reliability of LLMs is critical in the ever-evolving landscape of natural language processing. Hallucinations can propagate misinformation, leading to serious repercussions. Detecting and mitigating these hallucinations is imperative to fortify the trustworthiness and utility of LLMs.

HALLUCINATION DETECTION PIPELINE

Exploring Knowledge Graphs (KG's) :

  • Knowledge Graphs (KGs) serve as powerful frameworks for representing complex information in a structured and easily navigable format. At their core, KGs aim to organize knowledge in a manner that facilitates efficient retrieval and comprehension.
The Essence of Triplets:
  • Within the domain of KGs, the concept of triplets emerges as a cornerstone. These triplets, composed of subject-predicate-object structures, offer a concise and intuitive representation of relationships between entities. Think of them as the building blocks of knowledge, encapsulating factual units in a digestible format.


Inspired by Knowledge Graph Studies: 

  • The genesis of the triplet approach finds its roots in the realm of knowledge graph studies. Here, researchers recognize the importance of granularity in representing knowledge. Triplets provide a granular view, allowing us to capture nuanced relationships and intricate details within the knowledge domain.

Purpose of Triplets Approach:

  • Triplets aren't just about simplifying complex information; they serve a deeper purpose. By breaking down knowledge into its fundamental components, we enhance our ability to analyze, interpret, and leverage it effectively. In essence, triplets empower us to unlock the full potential of knowledge graphs, facilitating deeper insights and informed decision-making.

    Dataset Creation:

                                                                         NQ Dataset

     

    • We commenced by sampling 100 challenging questions from NQ(Natural Questions) Dataset, a goldmine of diverse queries. Each question underwent scrutiny by our suite of LLMs, including Alpaca-7B, ChatGPT, Claude-2, Falcon-40B, GPT-4. Leveraging the Claude API, we queried the Claude-2 model for each question-answer pair generating a list of triplets. These triplets were meticulously annotated, classifying them as either entailing, contradicting, or remaining neutral concerning the ground truths. 

    Utilizing NLI Models: 

    • Initially, pre-trained Natural Language Inference (NLI) models were enlisted to ascertain the veracity of the triplets. By examining the entailment of triplets within the provided contexts, we gauged their truthfulness. Employing RoBERTa-Large, a powerful NLI model, we conducted a comprehensive evaluation of each triplet's alignment with its context, thereby establishing a baseline for further analysis.

    Transitioning to LLMs: 

    • To shift the paradigm from NLI-based evaluation to a more direct reasoning approach, we entrusted the LLMs themselves with discerning the correctness of the knowledge graph. Claude-3 Sonnet emerged as our prime choice for this task, owing to its exemplary performance and efficiency. Through iterative refinement of prompting strategies, we crafted a tailored prompt to solicit direct feedback from the LLM regarding the validity of the provided knowledge. This pivotal transition not only streamlined the evaluation process but also empowered us to delve deeper into the phenomenon of hallucination within LLMs. The prompt used for it was:


    Results:

    • For the human annotated data on the triplets with respect to context: 
    • A graph of different colored bars Description automatically generated
    • Results on the triplets by using an NLI on different models:
    • A graph of a bar chart Description automatically generated with medium confidence
    • Results on the major label on whether it is a hallucination or not by using an NLI on different models: 
    • A graph of blue bars Description automatically generated with medium confidence
    • Results on multiple new models using NLI:
    • Comparision of results on models using NLI against using an LLM instead of NLI:
    • A graph of different colored bars Description automatically generated


    Analysis:

    The analysis of our results sheds light on the effectiveness of predicting hallucinations within language models (LLMs) given contextual information. Initially, Mistral demonstrates superior performance compared to other models, attributed to its higher parameter count. However, Falcon and GPT4 emerge as frontrunners, surpassing all other models, largely due to their substantial parameter sizes. This correlation between model performance and parameter count underscores the pivotal role of model size in achieving superior results.
    Moreover, our comparison between LLMs and Natural Language Inference (NLI) models reveals a notable advantage of LLMs in hallucination detection. While NLI models struggle with triplets due to their training on full sentences, LLMs excel in this domain, leveraging their extensive training on next-word prediction tasks and augmented parameter capacity.

    Conclusion:

    In conclusion, our study focused on the critical task of detecting hallucinations in LLMs, leveraging a dataset of triplets sourced from challenging questions in the NQ dataset. Through comprehensive evaluation using both pre-trained NLI models and LLMs, we have elucidated the strengths of LLMs in this domain. Our findings underscore the potential of LLMs to enhance the accuracy and reliability of text generation by mitigating hallucination occurrences. Moving forward, further research endeavors could delve into advanced methodologies for detecting and addressing hallucinations, thereby bolstering the trustworthiness and applicability of LLMs across diverse domains.
    YOUTUBE LINK: Video
    GITHUB LINK : RSAI_Project

    Comments

    Post a Comment

    Popular posts from this blog

    Data when seen through the Inference Web of LLM

    Image
    Background Today I saw a post from this guy saying “I was at the Marines today” . A few mins later I saw a post from the same guy saying: “I love vada pavs”. And i thought to myself “Hey, This guy is probably from Mumbai, India” . The next thought that followed this was if I, a mere human being can deduce this from 2 posts, what can the LLMs do? LLMS are trained on massive data sources from the internet. Isn't it possible that they infer this information too? Introduction Large language models (LLMs) have revolutionized natural language processing tasks. They demonstrate amazing capabilities in understanding and generating human-like text. In this project, we decided to delve into the accuracies of inferential capabilities of various LLM models.  Ob jectives Inference in LLMs is a vast and interesting topic to dive into. For the sake of this project, we chose 3 main objectives that we would be focusing upon: Identify the accuracies of Inferential Capabilities of the LLM​ Exploring...
    Read more

    EAR-VM: Exploring Methods for Improving Adversarial Robustness of Vision Models

    Image
      Abstract CNNs have many uses, particularly in the field of Computer Vision, however, their vulnerability to Adversarial attacks leaves a lot to be desired, particularly to their robustness to these kinds of attacks in order to make them more safe.  The misuse of adversarial attacks are a major threat to CNN vision models; for example, self-driving cars can be made to misinterpret road signs or signals, putting the passengers at significant risk of harm.  To address this vulnerability, we have attempted to modify the architecture of a CNN to add an auxiliary classification SVM which will determine the maximum margin in which these adversarial attacks will impact loss.   Also, interpretability is a key concept in understanding the decisions and outputs of modern networks. By interpreting the working of the model, we can curate better adversarial attacks, or make the model more robust.   Objectives Our goal is to first implement the At-SVM, and then to test it to...
    Read more

    Exploring and Quantifying Bias in VLMs

    Image
    Uncovering and Quantifying Bias in VLMs Abstract Our goal is to investigate biases present in vision language models and their impact on image text pronoun resolution. We aim to assess the performance of the CLIP model on the VisoGender benchmark and compare it with existing research findings. We seek to explore innovative approaches to mitigate biases, including novel ablation methods, to enhance model fairness and performance. Introduction Vision-language models (VLMs) play a crucial role in various applications, but their reliance on uncurated data can introduce biases. Evaluating and understanding these biases are essential for ensuring fairness and addressing ethical concerns. We present VisoGender, a dataset for benchmarking gender bias in VLMs. Our experiments assess bias in the CLIP model, focusing on resolution and retrieval biases. Additionally, we introduce an ablation experiment using DINO to evaluate spatial...
    Read more