To prevent unwanted content material from the research indexes, website owners can instruct spiders to not crawl certain information or directories in the common robots.txt file in the root Listing of your domain. On top of that, a webpage is usually explicitly excluded from a online search engine's database through https://news.rafeeg.ae