Computational technology has revolutionized the way much of the world operates, from agriculture to space science. Artificial intelligence (AI) is a branch that has captured immense popular and scientific interest over the last few decades. A new research paper discusses the role it could play in the practice of obstetrics and gynecology (OBG) in the future.
Clinical Opinion: The Exciting Potential for ChatGPT in Obstetrics and Gynecology. Image Credit: cono0430 / Shutterstock
Introduction
Natural language processing (NLP) is a branch of AI that deals with how computers recognize and deal with human language. Recent advances have led to excellent computational recognition of human text and speech via deep learning models.
The newly introduced ChatGPT is the poster child for these technological advances, as it seems to show that computers can ‘talk’ to humans using complex and grammatical language to convey complex insights.
ChatGPT is a publicly available online chatbot released in November 2022. The GPT in its name comes from the fact that it was developed by a firm called OpenAI, based on an NLP model called Generative Pretrained Transformer (GPT). Its advantages include a simple interface mediated by text on both sides, a massive data set of 57 billion words or more, and 175 billion parameters obtained from a search of the Internet, books, and other sources.
This means ChatGPT can access materials related to a dizzying array of disciplines, making it capable of handling questions for information or prompts from multiple areas. For example, it can write an article or furnish an answer to a question based on a clinical situation in plain English, proving its ability in the realm of language tasks. However, it has been accused of making up terms to provide seemingly correct but mistaken responses.
The current study, published online in the American Journal of Obstetrics and Gynecology, is a real-life demonstration of how ChatGPT could help users interested in OBG topics by making available introductory information about any area in this discipline.
What did the study show?
The authors asked the chatbot a set of questions aimed at evaluating how well ChatGPT could handle the need to provide authentic information about OBG topics. The topics addressed ranged from preterm birth, through progesterone supplementation, to gender-specific terminology.
Overall, the responses were “nuanced, eloquent, informed, with virtually no grammatical errors.” It could help educate both patients and their healthcare providers using easily understood language to convey information crisply and clearly.
However, the answers may change over time as data sets are upgraded and expanded and as the model continues to learn from the prompts supplied by users. Answers may even contradict each other if the prompts are worded inaccurately or in different forms.
This is attributable to the fact that its training set contains information related to both sides of any question, making it difficult to produce a single ‘right’ answer.
On the other hand, the data sets used for training the model are older and cannot be quickly updated as and when required because of the cost and time required. Thus, all answers are based on data from or older than 2021.
“ChatGPT is only as good as its derived training data. This data is potentially biased, unreliable and is not necessarily up to date.”
This is a feature of ChatGPT that all its users should know, as it precludes the presentation of views that take into account later research than this date.
Secondly, the fact that ChatGPT is, after all, not human may lead to misunderstandings of the queries. Moreover, it does not provide citations of its sources, which could mislead the user into believing the answers are sound when they could be derived from potentially flawed sources. “A seemingly satisfactory ChatGPT risks misleading without proper warning.”
Finally, it does not reason, it only compiles data.
“ChatGPT sometimes writes plausible sounding but incorrect or nonsensical answers and that it sometimes responds to harmful instructions or exhibits biased behavior.”
OpenAI has already declared the limitations of ChatGPT, and users would do well to take heed to these warnings while exploiting its advantages.
What are the implications?
The study authors conclude that despite the obvious advantage of having a source of early information for users who are anxious to know more about specific conditions than can be provided in a typical consultation with their healthcare providers, ChatGPT also has significant limitations.
They found that it has the potential to educate users in a readable, understandable, and generally correct manner, avoiding mistakes and misinformation. However, it can provide wrong, conflicting or outdated answers, potentially misleading users. It must be conceded that other scientific publications are also not guaranteed to purvey the most recent evidence.
Journals such as JAMA have announced guidelines on the use of such technologies in their publications. This is based on recognizing the limitations of chatbots as sources of scientific information and the possibility of plagiarism. Verification of accuracy is a mandatory precaution if these models are used to facilitate the writing of scientific text.
“The authors believe that ChatGPT and the prospect of new models to come have the possibility of adding a new dimension to our specialty. Responsible use of [such models] will be important in ensuring that they work to help but not harm users seeking information in Obstetrics and Gynecology.”
Study shows AI tool's high accuracy in answering genetic counseling questions