Natural Language Processing (NLP) has taken over the sphere of Artificial Intelligence (AI) with the introduction of Large Language Models (LLMs) comparable to OpenAI’s GPT-4. These models use massive training on large datasets to predict the following word in a sequence, and so they improve with human feedback. These models have demonstrated potential to be used in biomedical research and healthcare applications by performing well on a wide range of tasks, including summarization and question-answering.

Specialized models, comparable to Med-PaLM 2, have greatly influenced fields comparable to healthcare and biomedical research by enabling activities like radiological report interpretation, clinical information evaluation from electronic health records, and knowledge retrieval from biomedical literature. Improving domain-specific language models can result in lower healthcare costs, faster biological discovery, and higher patient outcomes.

However, LLMs still face several obstacles despite their impressive performance. Over time, the expenses related to the training and application of those models have increased significantly, raising each financial and environmental issues. Also, the closed nature of those models, that are run by large digital corporations, raises concerns about accessibility and data privacy.

In the biomedical field, the closed structure of those models prevents additional fine-tuning for particular needs. Though they supply domain-specific answers, models comparable to PubMedBERT, SciBERT, and BioBERT are modest in comparison with broader models comparable to GPT-4. 

In order to handle these issues, a team of researchers from Stanford University and DataBricks has developed and released BioMedLM, a GPT-style autoregressive model with 2.7 billion parameters. BioMedLM outperforms generic English models in multiple benchmarks and achieves competitive performance in biomedical question-answering tasks.

To provide a targeted and thoroughly chosen corpus for Biomedical NLP tasks, BioMedLM only uses training data from PubMed abstracts and full articles. When optimized for certain biomedical applications, BioMedLM performs robustly even whether it is smaller in scale than larger models. 

Evaluations have shown that BioMedLM can do well on multiple-choice biomedical question-answering tasks. It can achieve competitive results which might be on par with larger models. Its performance in extracting pertinent information from biological texts has been demonstrated by its scores of 69.0% on the MMLU Medical Genetics test and 57.3% on the MedMCQA (dev) dataset.

The team has shared that BioMedLM could be improved even further to supply insightful answers to patient inquiries about medical subjects. This adaptability highlights how smaller models, comparable to BioMedLM, can function as effective, transparent, and privacy-preserving solutions for specialised NLP applications, especially within the biomedical field. 

As a more compact option that requires less computational overhead for training and deployment, BioMedLM offers advantages when it comes to resource efficiency and environmental impact. Its dependence on a hand-picked dataset also improves openness and reliability, resolving issues with training data sources’ opacity.

Check out the Paper and ModelAll credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter. Join our Telegram Channel, Discord Channel, and LinkedIn Group.

Don’t Forget to affix our 39k+ ML SubReddit

This article was originally published at