BERT
BERT (Bidirectional Encoder Representations from Transformers) is a language model that Google began using in search in October 2019. It helps Google understand the meaning of words in context, particularly in longer, conversational queries where small words like “for” or “to” change the meaning.
What BERT is
BERT is a type of neural network based on the transformer architecture, which Google researchers introduced in 2017. Google released BERT as an open-source research model in 2018 and then applied it to search a year later. At launch, Google said BERT would affect about one in ten English-language searches in the US, calling it one of the biggest leaps forward in the history of search.
In December 2019 Google expanded BERT to more than 70 languages. It now plays a part in understanding almost all English queries.
How BERT works, in plain English
Older language systems tended to read a sentence one direction at a time. BERT is “bidirectional”: it looks at the words before and after each word together, so it understands each word based on its full context.
This matters because the same word can mean very different things. “Bank” in “river bank” and “bank account” is a simple example. BERT is good at picking up these differences and at noticing how prepositions and word order affect meaning.
Google’s examples
When announcing BERT, Google gave the example query “2019 brazil traveler to usa need a visa”. Before BERT, Google returned results about US citizens travelling to Brazil. With BERT, it understood the importance of the word “to” and showed information for Brazilians travelling to the US.
Another example was “can you get medicine for someone pharmacy”. BERT helped Google understand that the searcher wanted to know about collecting a prescription on someone else’s behalf.
Where BERT is used
- Ranking: understanding queries and matching them to relevant pages.
- Featured snippets: Google said it used BERT to improve snippets in many languages.
- Language understanding generally: it complements systems such as RankBrain and neural matching rather than replacing them.
Can you optimise for BERT?
Google’s Danny Sullivan said at the time that there is nothing to optimise for with BERT. It is not a penalty or a list of rules. Instead, it makes Google better at understanding natural writing, which rewards content that is clear and genuinely useful.
- Write naturally for people, not for exact-match keywords.
- Answer the specific question your audience is asking, matching their search intent.
- Be precise: say who something is for and in what situation it applies.
- Avoid awkward phrasing written to squeeze keywords in.
- Cover the topic thoroughly so detailed, long-tail questions are answered.
BERT and modern AI search
BERT was an important step towards today’s large language models. Both use transformer architecture, though BERT was built to understand text rather than generate it. Google has since added MUM and Gemini-based systems, but the lesson from BERT holds: clear, well-written content that answers real questions is easiest for language models to understand and use.
Key Takeaways
- BERT is a transformer-based language model Google added to search in October 2019.
- It reads words in context, using the words before and after each one.
- It improved understanding of conversational queries and small words like "to".
- There is nothing specific to optimise for: write clearly for people.
- BERT was an early step towards today’s large language models.
Related Terms
Wondering what to fix first on your site? Get a free check of your site’s SEO, or keep learning in the SEO Encyclopedia.