🚧Issues Encountered
The workflow was not always straightforward: here are some problems that were encountered
Marco
During the course of this project, I encountered two main challenges:
Interaction with AI: While AI is undoubtedly a powerful and useful tool, it often proved misleading for this specific task. Cognitive biases, despite having precise definitions, can manifest in a wide variety of contexts. When asking the AI to provide examples or basic models for the ontologies, the results were often highly variable and sometimes discordant.
Reuse of Ontology Design Patterns (ODPs): To ensure interoperability, I attempted to reuse as many ODPs as possible. However, this was not an easy task. I frequently found useful elements for my ontology under unexpected components, which inevitably increased the duration of my work. Being able to search for keywords on the platform for sure would have made the task faster.
Corrado
Like most of my colleagues I encountered several problems during the process of creating and modelling the ontologies:
Chat GTP answers: One of the main problems was the answers provided by Chat GTP. On several occasion these answers turned out to be incomplete (e.g., the required definition of the bias under consideration was sometimes too brief) imprecise (e.g., user story not completely relevant to the selected scenario), or misleading ( e.g., in the case of potential classes and properties to build an ontology of the bias some of them were poorly adapted considering the user story or forced). To get around this problem it was necessary to slightly change the prompts and reshape the answers provided to guide the AI to the best solution.
Framester Entities: The key concepts identified in the user stories of the individual biases did not always lead to corresponding frames in the Framester Hub being aligned. This sometimes led to slightly change the key concepts and to use other resources, in particular DBpedia proved to be a viable alternative.
Chosen ODPs: Choosing the appropriate ODPs for each cognitive biases ontology was not an easy task at all, because none of the available ODPs fitted perfectly in the process of defining the structure of the ontology. At the suggestion of our tutor I took advantage of the possibility of combining together the different ODPs, but even then I sometimes encountered some limitations that forced me to specially create specific classes and properties.
Ontologies inconsistencies: During the phase of merging all the ontologies together in the closure module, I found some inconsistencies and erroneous inferences due to conflicts in defining domains and ranges of some properties. This led to the necessity of defining via the logical operator "OR" the domains and ranges of the properties involved in each module to prevent such inconsistencies and unintended inferences.
Salvatore
The main issue I encountered during the development of the cognitive bias ontology regarded the limit of Chat GPT's knowledge base since its last knowledge update was in January 2022, and since then the Neglect of probability bias was not a widely recognized bias in the field of cognitive biases. To overcome the lack of data on this subject I used Gemini, an AI language model developed by Google, which is trained on publicly available data, and is continuously trained by processing information through Google search.
Alice
Working with large language models like ChatGPT to model cognitive biases within ontologies has been a fascinating task, blending technological innovation with the complexities of human cognition. One notable challenge that arises when working with large language models like ChatGPT is the task of addressing the inherent biases ingrained within these models. These biases often stem from the vast amounts of data they are trained on, which inherently contain societal, cultural, and linguistic biases. Consequently, a critical aspect of the work revolves around recognizing and rectifying these biases within the model's outputs.
Additionally, interpreting the responses generated by these models demands a nuanced understanding of context and subtle nuances. These models often grapple with ambiguity, necessitating a discerning approach to extract meaningful insights from their outputs. Despite these challenges, integrating such models into ontology frameworks holds significant promise for advancing our comprehension of cognitive biases. It's a delicate balancing act, requiring effective management of biases throughout the process to leverage the full potential of these models in uncovering valuable insights with practical applications across various disciplines.
Last updated