Finally, a replacement for BERT: Introducing ModernBERT

from blog Simon Willison's Weblog, | ↗ original
Finally, a replacement for BERT: Introducing ModernBERT BERT was an early language model released by Google in October 2018. Unlike modern LLMs it wasn't designed for generating text. BERT was trained for masked token prediction and was generally applied to problems like Named Entity Recognition or Sentiment Analysis. BERT also wasn't very useful...