Department of Computer Science: MSc Thesis Presentation

Preetha Datta’s final thesis presentation will be held online via Zoom as part of the Trust-M research seminars. 

10:00 – 11:00 
Preetha Datta: Construction of Hyper-Relational Knowledge Graphs Using Pre-Trained Large Language Models
Supervisor: Nitin Sawhney
Advisor: Fedor Vitiugin

Abstract: Hyper-relational knowledge graphs serve as a technique to organize previously unstructured data. Question-answering systems built on these graphs excel at handling multi-hop questions and offer clear, transparent answers. However, developing a question-answering system centered around knowledge graphs can pose significant challenges and demands considerable effort. This thesis endeavors to streamline the process by leveraging large language models to generate hyper-relational knowledge graphs since it implies cheaper knowledge graph construction methodologies in the future. My thesis tests a range of prompting strategies across a subset of large language models to thoroughly evaluate their effectiveness in extracting entities and relations. These entities and relations are essential building blocks for constructing a knowledge graph. By applying different prompting techniques, the research aims to determine the most efficient and accurate methods for entity and relation extraction. This evaluation provides insights into the capabilities and limitations of large language models in the context of knowledge graph development. We also perform a comparison of the prompting techniques with some existing supervised methodologies. The evaluation metric utilized in this thesis is BERTScore. Additionally, the thesis provides a comprehensive discussion on the advantages and disadvantages of BERTScore, as well as other evaluation metrics. This analysis aims to highlight the strengths and limitations of each metric, offering a balanced perspective on their applicability and effectiveness in assessing the outcomes of entity and relation extraction. The highest results achieved in this thesis are attributed to large language model based prompting that incorporates the relation descriptions of the dataset.