Robots.txt ntawv cia hauv lub hauv paus ntawm koj lub vev xaib yuav qhia cov web robots xws li kev tshawb nrhiav cav kab laug sab dab tsi thiab cov ntaub ntawv lawv raug tso cai nkag. Nws yooj yim mus siv cov ntaub ntawv robots.txt, tiam sis nws muaj qee yam uas koj yuav tsum nco:
- Dub kaus web robots yuav las mees koj cov ntaub ntawv robots.txt. Qhov ntau hom yog malware bots thiab robots tab tom nrhiav email chaw sau.
- Qee cov programmers tshiab yuav sau cov robots uas tsis pom cov ntawv robots.txt. Qhov no feem ntau yog ua yuam kev.
- Leej twg thiaj pom koj cov ntaub ntawv robots.txt. Lawv yeej ib txwm hu ua robots.txt thiab yeej ib txwm muab khaws cia rau ntawm lub hauv paus ntawm lub website.
- Thaum kawg, yog hais tias ib tug neeg txuas rau ib cov ntaub ntawv los yog directory uas tsis xam los ntawm koj cov ntaub ntawv robots.txt los ntawm ib sab uas tsis cais los ntawm lawv cov ntaub ntawv robots.txt, qhov kev tshawb fawb xyaw yuav pom nws lawm.
Tsis txhob siv cov ntaub ntawv robots.txt los zais txhua yam tseem ceeb. Xwb, koj yuav tsum muab cov lus qhia tseem ceeb tom qab kev ruaj ntseg passwords los yog tawm ntawm lub vev xaib tag nrho.
Yuav Siv Cov Qauv Cov Ntaub Ntawv no li cas
Luam cov ntawv nyeem los ntawm cov qauv uas ze rau yam koj xav ua, thiab muab tso rau hauv koj cov ntaub ntawv robots.txt. Hloov cov neeg hlau, phau ntawv teev npe, thiab cov npe hauv npe kom phim koj tus qauv siv.
Ob Ntaub Ntawv Cov Ntaub Ntawv Robots.txt
Neeg-tus neeg sawv cev: *
Tseg: /
Cov ntaub ntawv no hais tias txhua tus neeg hlau (Tus neeg siv-tus neeg saib xyuas: *) uas nkag mus rau nws yuav tsum tsis pom txhua nplooj ntawv ntawm lub tsev kawm ntawv (Tso Tseg: / Disallow: /).
Neeg-tus neeg sawv cev: *
Tsis pom zoo:
Cov ntaub ntawv no hais tias txhua tus neeg hlau (Tus neeg siv-tus neeg muag khoom: *) uas nws nkag mus pom nws tau saib txhua nplooj ntawv hauv lub tsev kawm ntawv (Disallow:).
Koj tuaj yeem ua tau qhov no los ntawm koj cov ntawv xa tawm robots.txt los tsis muaj ib qho ntawm koj qhov chaw txhua.
Tiv Thaiv Tej Kev Qhia Los ntawm Cov Neeg Hlau
Neeg-tus neeg sawv cev: *
Tsis txwv: / cgi-rau /
Tsis txwv: / temp /
Cov ntaub ntawv no hais tias txhua tus neeg hlau (Tus neeg siv-tus neeg saib xyuas: *) uas nkag mus rau nws yuav tsum tsis quav ntsej cov kev qhia / cgi-rau / thiab / temp / (Disallow: / cgi-bin / Disallow: / temp /).
Tiv thaiv cov nplooj ntawv tshwj xeeb ntawm cov Robots
Neeg-tus neeg sawv cev: *
Tsis pub siv: /jenns-stuff.htm
Tsis pub siv: /private.php
Cov ntaub ntawv no hais tias txhua tus neeg hlau (Tus neeg siv-tus neeg saib xyuas: *) uas nkag mus rau nws yuav tsum tsis quav ntsej cov ntaub ntawv /jenns-stuff.htm thiab /private.php (Disallow: / jenns-stuff.htm Disallow: /private.php).
Tiv thaiv kom tsis muaj neeg tshwj xeeb los ntawm kev nkag mus rau koj qhov chaw
Neeg-tus neeg sawv cev: Lycos / xx
Tseg: /
Cov ntaub ntawv no hais tias lub Lycos bot (Tus neeg siv-tus neeg saib xyuas: Lycos / xx) tsis pub nkag tau rau txhua qhov chaw hauv lub tsev kawm ntawv (Disallow: /).
Cia Tsuas Muaj Ib Tug Neeg Hlau Siv
Neeg-tus neeg sawv cev: *
Tseg: /
Neeg-tus neeg saib xyuas: Googlebot
Tsis pom zoo:
Cov ntaub ntawv no ua ntej tso tawm txhua tus neeg nyiam ua peb saum toj no, thiab mam li qhia meej rau Googlebot (Tus neeg siv-tus neeg saib xyuas: Googlebot) muaj txhua yam (Disallow:).
Muab cov kab ntau ntau los ua kom raws li cov Exclusions Koj Xav Tau
Thaum nws zoo dua los siv tus neeg siv kab-cev heev heev, xws li Tus neeg siv-tus neeg sawv cev: *, koj tuaj yeem ua tau raws li koj nyiam. Nco ntsoov tias robots nyeem cov ntaub ntawv hauv kev txiav txim. Yog li ntawd, yog thawj kab lus hais tias txhua tus neeg hlau raug blocked los ntawm txhua yam, thiab tom qab ntawd ces nyob rau hauv cov ntaub ntawv nws hais tias tag nrho cov neeg tsav tsheb raug tso cai rau txhua yam, cov neeg hlau yuav tau nkag mus rau txhua yam.
Yog tias koj tsis paub meej tias koj twb sau koj cov ntaub ntawv robots.txt kom raug, koj tuaj yeem siv Google Cov Ntawv Webmaster los xyuas koj cov ntaub ntawv robots.txt lossis sau ib daim tshiab.