User-agent:SemrushBot Disallow:/ User-agent: * Disallow: /admin Disallow: /login Disallow: /sample Disallow: /vendor Disallow: /sessions