#General User-agent: * Disallow: /2016/* Disallow: /2017/* Disallow: /2018/* Disallow: /2019/* Disallow: /2020/* Disallow: /2021/* Disallow: /2022/* Disallow: /introduction-to-bairesdev/ Disallow: /author/alistair-gillespie/ Disallow: /terms-conditions/ Disallow: /privacy-policy/ Disallow: /candidates-privacy-policy/ Disallow: /press/page/* Disallow: /?s= Disallow: /search/ Disallow: /blog/instagram/* Disallow: /blog/main_success_stories/* Disallow: /insights-sharing/* Disallow: /blog/nacho_posts/* Disallow: /blog/tag/* Disallow: /author/roberto-rangugni/ Disallow: /author/yousof-kamal/ Disallow: /author/fernando-gonzalez/ Disallow: /*404* Noindex: /author/ Noindex: /category/ Crawl-delay: 10 #Known crawlers User-agent: LinkedInBot User-agent: bingbot User-agent: RSiteAuditor User-agent: SiteAuditBot User-agent: SemrushBot User-agent: SemrushBot-SA User-agent: SemrushBot-SI User-agent: SemrushBot-BA User-agent: SemrushBot-SWA User-agent: SemrushBot-CT User-agent: SplitSignalBot User-agent: SemrushBot-COUB User-agent: AhrefsBot User-agent: AhrefsSiteAudit Crawl-delay: 60 User-agent: Yandex User-agent: YandexBot Crawl-delay: 300 #Feeds crawlers User-agent: Feedly User-agent: feeder.co User-agent: FlipboardRSS User-agent: omgili User-agent: FeedBurner User-agent: UniversalFeedParser User-agent: Competitors.app Allow: /feed/ Crawl-delay: 60 #Blocked crawlers User-agent: RyteBot User-agent: OnPageBot User-agent: dotbot User-agent: rogerbot user-agent: MJ12bot User-agent: SeznamBot User-agent: DataForSeoBot User-agent: PetalBot User-agent: coccocbot-web User-agent: SEOkicks User-agent: AwarioRssBot User-agent: AwarioSmartBot User-agent: SerendeputyBot User-agent: MojeekBot User-agent: Scrapy User-agent: Orbbot User-agent: MauiBot User-Agent: Neevabot User-agent: x09Mozilla User-agent: x22Mozilla Disallow: / Sitemap: https://www.bairesdev.com/sitemap_index.xml Sitemap: https://www.bairesdev.com/blog/sitemap_index.xml