Google AI Blog announced KELM, a way that could be used to reduce bias and toxic content in search (open domain question answering). It uses a method called TEKGEN to convert Knowledge Graph facts into natural language text that can then be used to improve natural language processing models.
KELM is an acronym for Knowledge-Enhanced Language Model Pre-training. Natural language processing models like BERT are typically trained on web and other documents. KELM proposes adding trustworthy factual content (knowledge-enhanced) to the language model pre-training in order to improve the factual accuracy and reduce bias.
TEKGEN converts knowledge graph structured data to natural language text known as the KELM Corpus
The Google researchers proposed using knowledge graphs for improving factual accuracy because they’re a trusted source of facts.
Continue Reading Below
“Alternate sources of information are knowledge graphs (KGs), which consist of structured data. KGs are factual in nature because the information is usually extracted from more trusted sources, and post-processing filters and human editors ensure inappropriate and incorrect content are removed.”
Google has not indicated whether or not KELM is in use. KELM is an approach to language model pre-training that shows strong promise and was summarized on the Google AI blog.
This research is important…