#This is the robots.txt file for www.marukadod.com #in robots.txt file only two conventions are used #first is "user agent" and second is # Disallow # to specify a directory not to index or follow you should do # user-agent:* ->here the wildcard specifies all the web crawlers # Disallow:/kadodhtml/edir/ -> this will prevent indexing of edir directory # Disallow:/copyright.html -> this will prevent perticular file copy.html # to be indexed and followed # thats all the info # now the file user-agent:* Disallow:/css Disallow:/scripts Disallow:/media