SEO-Optimierung: Meta-Tags, Open Graph, Schema.org, robots.txt, sitemap.xml, semantisches HTML und Performance-Verbesserungen (.htaccess mit GZIP & Caching)
This commit is contained in:
22
robots.txt
Normal file
22
robots.txt
Normal file
@@ -0,0 +1,22 @@
|
||||
# robots.txt für baumer.traidendorf.de
|
||||
|
||||
User-agent: *
|
||||
Allow: /
|
||||
|
||||
# Sitemap
|
||||
Sitemap: https://baumer.traidendorf.de/sitemap.xml
|
||||
|
||||
# Verzeichnisse die nicht gecrawlt werden sollen
|
||||
Disallow: /phpmailer/
|
||||
Disallow: /config.php
|
||||
Disallow: /send-mail.php
|
||||
|
||||
# Verhindere Crawler von Spam-Bots
|
||||
User-agent: AhrefsBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: SemrushBot
|
||||
Disallow: /
|
||||
|
||||
User-agent: DotBot
|
||||
Disallow: /
|
||||
Reference in New Issue
Block a user