# robots.txt Editor — Installation & User Guide

> Complete guide to the robots.txt Editor module: installation, editing, validation, regeneration, AI crawler blocking templates and backup restore.

- Page: <https://www.datafirefly.com/en/documentation/robots-txt-editor/>
- Language: en
- Last updated: 2026-06-27
- Other languages: [fr](https://www.datafirefly.com/documentation/editeur-robots-txt/index.md), [es](https://www.datafirefly.com/es/documentation/editor-robots-txt/index.md), [de](https://www.datafirefly.com/de/documentation/robots-txt-editor-anleitung/index.md), [it](https://www.datafirefly.com/it/documentation/guida-editor-robots-txt/index.md), [pl](https://www.datafirefly.com/pl/documentation/edytor-robots-txt-prestashop/index.md), [pt](https://www.datafirefly.com/pt/documentation/editeur-robots-txt/index.md), [nl](https://www.datafirefly.com/nl/documentation/editeur-robots-txt/index.md)
- Index: <https://www.datafirefly.com/en/documentation/llms.txt>

DataFirefly's **robots.txt Editor** lets you edit, validate, regenerate and back up your store's `robots.txt` file straight from the PrestaShop back office, with no FTP or SSH access. This guide covers installation and every feature.

## Compatibility

- PrestaShop 8.0 to 9.x
- PHP 7.4 to 8.x
- Multistore
- French interface

## Installation

1. Download the `dfrobotseditor.zip` archive.
2. In the back office, open **Modules → Module Manager**.
3. Click **Upload a module** and drop the ZIP (or copy the `dfrobotseditor` folder into `/modules/` and install it from the list).
4. Once installed, click **Configure**: the editor opens directly.

On installation, no `robots.txt` file is modified. The module simply opens an editor; every save stays under your control.

## Opening the editor

Two ways in:

- Via the module's **Configure** button (recommended).
- Via the **Modules → robots.txt Editor** menu in the back office.

The main screen shows the file path, its write status, a "View online" link and, when available, the date of the last backup.

## Editing and saving

The file content appears in a monospace editor. Edit it freely, then click **Save**. On every save:

- the previous version is backed up automatically;
- line endings are normalized;
- content is capped at 64 KB.

## Regenerating the default file

The **Regenerate (default)** button rebuilds a standard `robots.txt`. The module uses PrestaShop's native rules when available, otherwise an equivalent DataFirefly template. The current version is backed up before replacement.

Regeneration writes the file immediately. A backup is created beforehand: you can roll back via **Restore backup**.

## Validating the syntax

The **Validate syntax** button analyses the editor content and lists issues, sorted by severity:

- **Error**: non-absolute Sitemap, rule placed before any `User-agent`, malformed line.
- **Warning**: unknown directive, BOM present, deprecated `Noindex` directive, non-numeric `Crawl-delay`.
- **Info**: empty file (all robots are allowed).

Validation does not modify the file: it simply helps you fix issues before saving.

## Quick templates

The **Insert a template** menu adds ready-to-use content at the cursor position:

- **Sitemap line**: inserts the absolute URL of your sitemap.
- **Block AI crawlers**: inserts a block disallowing GPTBot, ClaudeBot, CCBot, Google-Extended, PerplexityBot, Bytespider and others.
- **Allow all** / **Block all**.

Example AI-crawler blocking block:

```
# AI crawler blocking — DataFirefly
User-agent: GPTBot
Disallow: /

User-agent: ClaudeBot
Disallow: /

User-agent: CCBot
Disallow: /
```

Google-Extended concerns Google AI training, not Google Search indexing. Adjust the list to fit your SEO and AEO strategy.

## Restoring a backup

Every save or regeneration creates a backup of the previous version, stored in the database (so **never publicly accessible**). The **Restore backup** button instantly writes that version back to the file.

## Permissions and troubleshooting

The `robots.txt` file must be writable by the web server. If it isn't, the module shows a warning and disables saving.

```
chmod 644 /path/to/robots.txt
# if the file does not exist yet:
chmod 755 /path/to/root/
```

- **"Read-only"**: apply the permissions above, or adjust the file owner (the web server user).
- **The file does not appear online**: check that no server rule or CDN serves a different static `robots.txt`.

## Multistore

`robots.txt` is a single file served at the root of each domain. In single-domain multistore, one file is enough. In multi-domain setups, group the per-domain sections within this same file.

## Uninstalling

Uninstalling the module removes its tab and its database backup. **Your robots.txt file is not modified**: it stays exactly as you saved it.