https://voudeals.com/creating-and-customizing-a-robots-txt-file-for-a-wordpress-site/
Creating and Customizing a Robots.txt File for a WordPress Site