This AI paper from Tel Aviv University presents GASLITE: a gradient-based method for exposing vulnerabilities in dense embedding-based text retrieval systems

Dense embedding-based text retrieval has become the cornerstone for classifying text passages in response to queries. The systems use deep learning models to embed text in vector spaces that enable semantic similarity measurements. This method has been widely adopted in applications such as search engines and retrieval augmented generation (RAG), where it is essential to retrieve accurate and contextually relevant information. These systems efficiently match queries with relevant content based on learned representations, driving enormous advances in knowledge-intensive domains.

However, the main challenge for embedding-based recovery systems is their susceptibility to manipulation by adversaries. The reason is that these systems often rely on public corpora, which are not immune to contradictory content. Malicious actors can inject crafted passages into the corpus in a way that affects the retrieval system's ranking to prioritize adverse entries over queries that contain them. This can threaten the integrity of search results with the spread of misinformation or the introduction of biased content, jeopardizing the reliability of knowledge systems.

Previous approaches to countering adversarial attacks have used simple poisoning techniques, such as stuffing specific queries with repetitive text or embedding misleading information. Although these methods can break single-query systems, they are often ineffective against more complex models that handle diverse query distributions. Existing defenses also do not address the core vulnerabilities of integration-based recovery systems, leaving systems open to more advanced and subtle attacks.

Researchers at Tel Aviv University introduced a mathematically based gradient-based optimization method called GASLITE to create adversarial passages. GASLITE works better than previous techniques because it focuses precisely on the embedding space of the retrieval model instead of modifying the text content. It aligns with certain query distributions, resulting in conflicting passages achieving high visibility within the retrieval results. Therefore, this makes it a powerful tool for assessing vulnerabilities in dense integration-based systems.

The GASLITE methodology is based on rigorous mathematical principles and innovative optimization techniques. It constructs conflicting passages from attacker-chosen prefixes combined with optimized triggers designed to maximize similarity to specific query distributions. The optimization takes the form of gradient calculations in the embedding space to find optimal token substitutions. Unlike previous approaches, GASLITE does not edit the corpus or model, but rather focuses on generating text that the retrieval system's classification algorithm can manipulate. This design makes it stealthy and effective; Adversarial passages can blend directly into the corpus without being detectable by standard defenses.

The authors test GASLITE with nine state-of-the-art recovery models in various threat scenarios. The method consistently outperformed baseline approaches, achieving a remarkable 61-100% success rate in ranking contradictory passages within the top 10 results for specific concept queries. These results were achieved with minimal poisoning of the corpus, with contradictory passages comprising only 0.0001% of the data set. For example, GASLITE demonstrated top-10 visibility in most retrieval models when addressing concept-specific queries, demonstrating its accuracy and efficiency. In single-query attacks, the method consistently ranked conflicting content as the top result, which is effective even under the most stringent conditions.

Further analysis of the factors contributing to the success of GASLITE showed that the geometry of the embedding space and similarity metrics significantly determined the susceptibility of the model. Models using dot product similarity measures were particularly vulnerable because the GASLITE method exploited these features to achieve optimal alignment with specific query distributions. The researchers further emphasized that models with anisotropic embedding spaces, where random text pairs produced high similarities, were more susceptible to attacks. This again points to the importance of understanding the properties of the embedding space when designing recovery systems.

It underscores the need for robust defenses against adverse tampering in embedding-based recovery systems. Therefore, the authors recommend using hybrid recovery approaches, such as dense and sparse recovery techniques, which can minimize the risks offered by methods such as GASLITE. It alone serves to expose the vulnerability of current recovery systems to risks and pave the way for more secure and resilient technologies.

Researchers urgently call to focus on the risks these adversarial attacks present to dense integration-based systems. The minimal effort GASLITE could use to manipulate search results shows the potential severity of such attacks. However, by characterizing critical vulnerabilities and developing actionable defenses, this work provides valuable information to improve this robustness and reliability in recovery models.

Verify he Paper and GitHub page. All credit for this research goes to the researchers of this project. Also, don't forget to follow us on <a target="_blank" href="https://twitter.com/Marktechpost”>twitter and join our Telegram channel and LinkedIn Grabove. Don't forget to join our SubReddit over 60,000 ml.

UPCOMING FREE ai WEBINAR (JANUARY 15, 2025): <a target="_blank" href="https://info.gretel.ai/boost-llm-accuracy-with-sd-and-evaluation-intelligence?utm_source=marktechpost&utm_medium=newsletter&utm_campaign=202501_gretel_galileo_webinar”>Increase LLM Accuracy with Synthetic Data and Assessment Intelligence–<a target="_blank" href="https://info.gretel.ai/boost-llm-accuracy-with-sd-and-evaluation-intelligence?utm_source=marktechpost&utm_medium=newsletter&utm_campaign=202501_gretel_galileo_webinar”>Join this webinar to learn practical information to improve LLM model performance and accuracy while protecting data privacy..