Search Engine

A search engine is a software system that is designed to carry out web searches. They search the World Wide Web in a systematic way for particular information specified in a textual web search query.

— Wikipedia’s definition of Search Engine

Search engines might be the most used software that not many people know how it works. Weather it’s googling the closest coffee shop or funny cat videos. Search engines are everywhere and powerful, yet we don’t think twice on how it came to the results.

In this blog, we are going to briefly look at how search engines work under the hood and the steps they take to reach the end results. Even though each search engine is unique they all work in a similar way, performing 3 simple tasks.

Search Engines Perform 3 Simple Tasks:

1st: Crawling (Bot/Spider)

Bots crawl though the web examining the content they learn about from a page. Their sole purpose is to visit and revisit pages looking for new links and new content to include in the index. However they only have access to content that they have permission to see.

2nd: Indexing

We can think of the Index as a massive list of all the web pages and content found by crawling. Categorizing each piece of content by many different tactics and data structures.

The search engine uses this index as a source of information displayed on the search results pages, however not everything the bot finds makes it onto a search engine’s index. For instance, if the bots found multiple copies of the exact same piece of content located on a different website they are not going to include in the index.

3rd: Ranking

Ranking, as you can guess, decide which content is most useful to the search.

After typing something in the search bar, the search engine compares the words/phrases to its index and looks for matching results. They thin out all the resources by many different tactics like words on a web page, number of other websites linking to the words/phrases, and the freshness of the content.

No matter what type of tactics the search engine uses, the goal never changes. Which is to try to connect the searcher with what they are looking for.

Conclusion

Although this was a quick look at how a search engine works, we can see they do a lot behind the scenes. They are constantly working to scour the web for content, organize it, and then display the most relevant results to the searcher.