User-agent: *
Allow: /

#Known crawlers
User-agent: LinkedInBot
User-agent: bingbot
User-agent: RSiteAuditor
User-agent: SiteAuditBot
User-agent: SemrushBot
User-agent: SemrushBot-SA
User-agent: SemrushBot-SI
User-agent: SemrushBot-BA
User-agent: SemrushBot-SWA
User-agent: SemrushBot-CT
User-agent: SplitSignalBot
User-agent: SemrushBot-COUB
User-agent: AhrefsBot
User-agent: AhrefsSiteAudit
Crawl-delay: 60

User-agent: Yandex
User-agent: YandexBot
Crawl-delay: 300


Sitemap: https://divelement.io/sitemap-index.xml