Add the X-Robots-Tag HTTP header to instruct search index bots on permissible activity.
Based on options listed in the Google Webmasters documentation on Robots meta tag and X-Robots-Tag HTTP header specifications.
Meta tags insert flags for robots in the website HTML output, however this adds robot tags to the HTTP connection handshake. This is only useful for global rules since that's all I cared to implement.
Supported flags are:
- Originally written for Drupal by David Norman
- Ported to Backdrop by David Norman (https://github.com/deekayen)
This project is GPL v2 software. See the LICENSE.txt file in this directory for