PS PrestaShop PrestaShop Beginner

robots.txt Editor — Installation & User Guide

Install, edit, validate and secure your PrestaShop robots.txt from the back office.

Updated Module version 1.0.0

DataFirefly’s robots.txt Editor lets you edit, validate, regenerate and back up your store’s robots.txt file straight from the PrestaShop back office, with no FTP or SSH access. This guide covers installation and every feature.

Compatibility

  • PrestaShop 8.0 to 9.x
  • PHP 7.4 to 8.x
  • Multistore
  • French interface

Installation

  1. Download the dfrobotseditor.zip archive.
  2. In the back office, open Modules → Module Manager.
  3. Click Upload a module and drop the ZIP (or copy the dfrobotseditor folder into /modules/ and install it from the list).
  4. Once installed, click Configure: the editor opens directly.

On installation, no robots.txt file is modified. The module simply opens an editor; every save stays under your control.

Opening the editor

Two ways in:

  • Via the module’s Configure button (recommended).
  • Via the Modules → robots.txt Editor menu in the back office.

The main screen shows the file path, its write status, a “View online” link and, when available, the date of the last backup.

Editing and saving

The file content appears in a monospace editor. Edit it freely, then click Save. On every save:

  • the previous version is backed up automatically;
  • line endings are normalized;
  • content is capped at 64 KB.

Regenerating the default file

The Regenerate (default) button rebuilds a standard robots.txt. The module uses PrestaShop’s native rules when available, otherwise an equivalent DataFirefly template. The current version is backed up before replacement.

Regeneration writes the file immediately. A backup is created beforehand: you can roll back via Restore backup.

Validating the syntax

The Validate syntax button analyses the editor content and lists issues, sorted by severity:

  • Error: non-absolute Sitemap, rule placed before any User-agent, malformed line.
  • Warning: unknown directive, BOM present, deprecated Noindex directive, non-numeric Crawl-delay.
  • Info: empty file (all robots are allowed).

Validation does not modify the file: it simply helps you fix issues before saving.

Quick templates

The Insert a template menu adds ready-to-use content at the cursor position:

  • Sitemap line: inserts the absolute URL of your sitemap.
  • Block AI crawlers: inserts a block disallowing GPTBot, ClaudeBot, CCBot, Google-Extended, PerplexityBot, Bytespider and others.
  • Allow all / Block all.

Example AI-crawler blocking block:

# AI crawler blocking — DataFirefly
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: CCBot
Disallow: /

Google-Extended concerns Google AI training, not Google Search indexing. Adjust the list to fit your SEO and AEO strategy.

Restoring a backup

Every save or regeneration creates a backup of the previous version, stored in the database (so never publicly accessible). The Restore backup button instantly writes that version back to the file.

Permissions and troubleshooting

The robots.txt file must be writable by the web server. If it isn’t, the module shows a warning and disables saving.

chmod 644 /path/to/robots.txt
# if the file does not exist yet:
chmod 755 /path/to/root/
  • “Read-only”: apply the permissions above, or adjust the file owner (the web server user).
  • The file does not appear online: check that no server rule or CDN serves a different static robots.txt.

Multistore

robots.txt is a single file served at the root of each domain. In single-domain multistore, one file is enough. In multi-domain setups, group the per-domain sections within this same file.

Uninstalling

Uninstalling the module removes its tab and its database backup. Your robots.txt file is not modified: it stays exactly as you saved it.

Was this page helpful?

Still stuck? Contact support