# Topic Cluster Detector — Documentation

> This documentation covers installation, configuration and usage of the Topic Cluster Detector module on PrestaShop 8 and 9. The module automatically detects the thematic groupings within your catalog through semantic…

- Page: <https://www.datafirefly.com/en/documentation/topic-cluster-detector/>
- Language: en
- Last updated: 2026-06-28
- Other languages: [fr](https://www.datafirefly.com/documentation/topic-cluster-detector/index.md), [es](https://www.datafirefly.com/es/documentation/topic-cluster-detector/index.md), [de](https://www.datafirefly.com/de/documentation/topic-cluster-detector/index.md), [it](https://www.datafirefly.com/it/documentation/topic-cluster-detector/index.md), [pl](https://www.datafirefly.com/pl/documentation/topic-cluster-detector/index.md), [nl](https://www.datafirefly.com/nl/documentation/topic-cluster-detector/index.md), [pt](https://www.datafirefly.com/pt/documentation/topic-cluster-detector/index.md)
- Index: <https://www.datafirefly.com/en/documentation/llms.txt>

This documentation covers installation, configuration and usage of the **Topic Cluster Detector** module on PrestaShop 8 and 9. The module automatically detects the thematic groupings within your catalog through semantic clustering, and suggests missing pillar pages with a complete SEO draft for each opportunity.

## Overview

Topic Cluster Detector analyzes your product catalog to surface the **actual topic clusters present** in your offering, then detects **missing pillar pages**: those transversal subjects extensively covered by your products but without a structuring parent page (CMS or category).

For each detected gap, the module generates a complete draft:

- SEO-optimized H1 title
- URL-safe slug
- Meta description
- Complete H2 markdown outline
- Target keyword list
- Priority score (size × cohesion)

**Important.** The module does not create pages directly in PrestaShop. It provides you with a draft to copy-paste into a new CMS page or category landing page, keeping the editorial decision in your hands.

## Installation

### Prerequisites

- PrestaShop **8.0+** or **9.x**
- PHP **8.0 minimum** (PHP 8.1 or 8.2 recommended)
- `memory_limit` of at least 512 MB (1024 MB recommended for large catalogs)
- Optional: OpenAI or Mistral API key for embeddings mode

### Installation procedure

1. Download the `dftopicclusters.zip` file from your DataFirefly customer area.
2. In the PrestaShop back-office, go to **Modules → Module Manager** and click **Upload a module**.
3. Select the ZIP and confirm. The module installs automatically.
4. Once installed, click **Configure**.

At install time, the module automatically creates:

- The 5 SQL tables prefixed `df_topicclusters_`
- The parent tab **DataFirefly** under the Improve menu (if not already present)
- The child tab **Topic Clusters** under that parent
- The 19 default configuration keys

### First access

Once installed, the module is accessible via **Improve → DataFirefly → Topic Clusters**. The dashboard presents the run launcher form and the history of previous runs (empty on first access).

## Configuration

Click the **Settings** button at the top right of the dashboard to access the configuration page. Settings are grouped into five sections.

### General

| Key | Default | Description |
| --- | --- | --- |
| `DFTC_MODE` | `tfidf` | Clustering mode: `tfidf` (local), `openai` or `mistral` |
| `DFTC_K_AUTO` | enabled | If enabled, automatically computes k = ceil(√(N/2)) bounded to [5, 30] |
| `DFTC_K_MANUAL` | 12 | k value used when `DFTC_K_AUTO` is disabled |
| `DFTC_MAX_ITER` | 60 | Maximum number of k-means iterations |
| `DFTC_MIN_CLUSTER_SIZE` | 3 | Minimum cluster size; smaller clusters are discarded |

### Text extraction

Lets you choose which product fields are fed into the analysis. The per-field weighting is fixed (name × 3, meta × 2, short description × 2, categories × 2, tags × 2, long description × 1, features × 1).

- `DFTC_INCLUDE_DESCRIPTION` — Include the long description (recommended: enabled)
- `DFTC_INCLUDE_CATEGORIES` — Include category names (recommended: enabled)
- `DFTC_INCLUDE_TAGS` — Include PrestaShop tags (recommended: enabled)
- `DFTC_INCLUDE_FEATURES` — Include product features (recommended: disabled unless your features are very descriptive)

### TF-IDF settings

| Key | Default | Description |
| --- | --- | --- |
| `DFTC_MIN_DOC_FREQ` | 2 | Term discarded if it appears in fewer than N products |
| `DFTC_MAX_DOC_FREQ_RATIO` | 0.50 | Term discarded if it appears in more than X% of the catalog |
| `DFTC_NGRAM_MAX` | 2 | 1 = unigrams, 2 = unigrams + bigrams |
| `DFTC_TOP_TERMS_COUNT` | 8 | Number of terms displayed per cluster |

### Embeddings APIs

| Key | Description |
| --- | --- |
| `DFTC_OPENAI_API_KEY` | OpenAI Bearer token (sk-...) |
| `DFTC_OPENAI_MODEL` | Model (default `text-embedding-3-small`) |
| `DFTC_MISTRAL_API_KEY` | Mistral API key |
| `DFTC_MISTRAL_MODEL` | Model (default `mistral-embed`) |
| `DFTC_BATCH_SIZE` | Number of products per API call (default 32) |

### Pillar page detection

- `DFTC_PILLAR_MATCH_THRESHOLD` — Match threshold (default 0.45). Below this, the cluster is flagged as a _pillar gap_. Raise the threshold to be stricter, lower it to be more lenient.

## The three modes in detail

### TF-IDF mode (recommended to start)

TF-IDF (Term Frequency × Inverse Document Frequency) is a classical NLP statistical method. The module builds a vocabulary from all product texts, filters out terms that are too rare or too frequent, then represents each product as a sparse vector in this space.

**Pros:** 100% local, instant, free, no external dependency. Excellent for lexically homogeneous catalogs (one domain, one consistent vocabulary).

**Limits:** does not understand synonyms (two products using different terms for the same concept will be poorly grouped).

### OpenAI embeddings mode

Uses the OpenAI `text-embedding-3-small` API by default. Each product is represented by a dense 1536-dimensional vector that captures its semantics.

**Pros:** understands synonyms, lexical variants and context. Excellent for diverse catalogs or those with narrative descriptions.

**Indicative cost:** approximately 0.02 USD per million tokens, i.e. less than 0.10 USD for a 1,000-product catalog.

**Tip.** The embeddings cache is automatic: if you re-run on the same catalog without modifying the texts, vectors are retrieved from the `df_topicclusters_embedding_cache` table without a new API call.

### Mistral embeddings mode

Uses the Mistral `mistral-embed` API by default. A performant multilingual model, particularly strong in French.

**Pros:** hosted in Europe (easier GDPR compliance), excellent on French content, competitive pricing.

## Running an analysis

From the dashboard, the **Launch a new analysis** form offers six parameters:

- **Language** — the language in which product texts will be extracted and analyzed. Launch a separate run for each active language of your store.
- **Mode** — TF-IDF, OpenAI or Mistral (overrides the default setting for this run only).
- **Number of clusters (k)** — Leave 0 for auto-k. Otherwise, force a value between 2 and 100.
- **Minimum size** — Smaller clusters discarded (default 3).
- **Pillar threshold** — Match threshold below which a cluster is flagged as a gap (default 0.45).
- **Product limit** — Limits the number of products analyzed (useful for debugging or quick testing). Leave 0 to analyze the entire catalog.

Click **Run analysis**. The run starts immediately. For a 1,000-product catalog:

- TF-IDF mode: 5 to 15 seconds
- Embeddings mode (first run): 30 seconds to 2 minutes depending on batch size
- Embeddings mode (subsequent runs with warm cache): equivalent to TF-IDF

**Important.** The module sets `set_time_limit(0)` and `memory_limit=1024M` for the run's duration. On very constrained hostings, these directives may be ignored. Prefer a night-time run or use the product limit to split.

## Reading the results

Once the run is complete, you access the detail page. Each cluster is displayed as a card with four sections.

### Cluster header

The header combines a status badge, a cluster number, and a generated label. The badge is:

- **PILLAR GAP** (orange) — No existing pillar page covers this subject. Strong opportunity.
- **OK** (green) — A CMS page or category already covers this subject (the module matched it).

The label is composed of the cluster's 3 top terms joined by `·`. Example: "sneakers · premium leather · shoes".

### Statistics

- **Products** — Number of products in the cluster
- **Cohesion** — Average similarity of members to the centroid (0 to 100%). The higher, the more homogeneous the cluster.
- **Match** — Match score with the best existing pillar page. If below the threshold → gap.

### Top terms

The most representative terms of the cluster. In TF-IDF mode, these are the terms with the strongest component in the centroid. In embeddings mode, the module computes an in-cluster TF weighted by the global IDF to bring out distinctive terms.

### Pillar page suggestion

Present only if the cluster is flagged as a gap. Contains:

- **Title** — SEO-optimized H1 title in natural language
- **Slug** — URL-safe, in kebab-case
- **Meta description** — 150-160 characters
- **Priority** — Combined score size (0.6) × cohesion (0.4)
- **Suggested outline** — H2 markdown outline with classic sections (intro, what is, how to choose, comparison, best products, use cases, mistakes to avoid, FAQ, CTA)

### Cluster products

List of grouped products with their similarity score to the centroid, sorted by decreasing similarity. Click on the product ID to open the product sheet directly in a new window.

## Recommended workflow

Here is a typical use of the module in 4 steps.

1. **First audit** — Run a TF-IDF run on your main language with default parameters. Examine the clusters flagged as gaps: are they editorially relevant?
2. **Triage** — For each gap, use the **Dismiss** button if the cluster does not deserve a pillar page (e.g. an accidental grouping of heterogeneous products). The remaining gaps are your priorities.
3. **Writing** — For each retained gap, create a new CMS page in PrestaShop with the title, slug and meta of the draft. Use the H2 outline as a writing skeleton. Click **Mark Done** once published.
4. **Re-run** — After publishing the new pages, re-run the analysis. Previous gaps should now be OK (the module will detect the new pillar pages).

**SEO best practices.** A quality pillar page is at least 1,500 words, includes internal links to the cluster's products, and uses the top terms naturally in the content. The generated draft is a starting point, not a final deliverable.

## Export

From a run's detail page, two buttons at the top right enable export:

- **CSV** — Table with one row per cluster, columns: id_cluster, label, n_members, cohesion, pillar_gap, match_score, suggested_title, suggested_slug, suggested_meta, priority_score, target_keywords. UTF-8 encoding with BOM (Excel compatible).
- **JSON** — Complete export including the product list per cluster and the full markdown outline. Ideal for automation or external integration.

## Technical architecture

### Database

The module creates 5 tables with the `df_topicclusters_` prefix:

- `run` — Metadata of each execution (mode, language, status, duration, counters)
- `cluster` — Individual clusters (label, top terms JSON, cohesion, pillar_gap flag, match_score)
- `cluster_product` — Product → cluster membership with similarity score
- `pillar` — Pillar page suggestions (title, slug, meta, outline, priority, status)
- `embedding_cache` — Embeddings vector cache indexed by text hash

### PSR-4 and autoload

The root namespace is `DataFirefly/TopicClusters/`. A manual autoload is registered via `spl_autoload_register` in the module's main file, so no Composer dependency is required.

### Controllers and PS 8 / PS 9 compatibility

The module uses a legacy `ModuleAdminController` (and not a Symfony controller) to guarantee compatibility with both major versions. SQL queries are written to respect both schemas, notably the removal of the `meta_keywords` column in PS 9.

## Performance and limits

- **Catalogs up to 1,000 products** — Runs in seconds. No particular concern.
- **1,000 to 10,000 products** — TF-IDF mode remains fast (10-60 s). Embeddings mode: plan for 1 to 5 minutes for the first run, instant afterwards thanks to the cache.
- **More than 10,000 products** — Prefer the **Product limit** parameter to split, or increase `memory_limit` to 2 GB.

K-means complexity is O(n × k × iter × d) where n is the number of products, k the number of clusters, iter the number of iterations (typically 10-30) and d the vector dimension (variable in TF-IDF, 1536 in OpenAI).

## Troubleshooting

### No clusters detected

Check that your products have textual content in the analyzed language (at least a name and ideally a description). If `DFTC_MIN_DOC_FREQ` is too high for your catalog, lower it to 1.

### All clusters are flagged as pillar gaps

The `DFTC_PILLAR_MATCH_THRESHOLD` threshold is probably too high. Try 0.30 instead of 0.45 if your store has few CMS pages. Also check that your CMS pages and categories are active.

### "Unknown column meta_keywords" error

This error occurs on PrestaShop 9 with an earlier module version. Update to version 1.0.0 or later, which removes all references to `meta_keywords` (column removed in PS 9).

### "Compile Error: Access level to processExport() must be public" error

This error occurred on a pre-1.0.0 version. The method name is now `doExport()` to avoid colliding with `AdminControllerCore`. Update the module.

### The run fails with API error

Check that the API key entered in the configuration is valid and has credits. Test with `curl` in CLI to confirm the server can reach `api.openai.com` or `api.mistral.ai`.

## Coming soon

- Direct CMS page creation from the draft (one click)
- Run comparison (before/after pillar page publication)
- Graphical visualization of the inter-cluster semantic network
- Support for Cohere and Voyage AI embeddings
- Automatic cron for periodic re-runs

**Support.** For any question or bug, contact [support@datafirefly.com](mailto:support@datafirefly.com). Feedback is valuable for steering the roadmap.