mahdium/CatsOfMastodonGo-Admin
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update robots.txt to disallow all crawlers

Browse Source
This commit is contained in:
Mohammad Mahdi
2025-06-11 09:15:23 +03:30
parent 21475ff08e
commit ebfe34faf0
1 changed files with 1 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

14
public/robots.txt
View File

@@ -1,14 +1,2 @@
User-agent: Googlebot
Allow: /
User-agent: Bingbot
Allow: /
User-agent: Twitterbot
Allow: /
User-agent: facebookexternalhit
Allow: /
User-agent: * User-agent: *
Allow: / Disallow: /