DataFirefly Indexing API — Documentation

Overview

DataFirefly Indexing API automatically submits products, categories and CMS pages of your PrestaShop store to the two official direct submission APIs: Google Indexing API (Service Account OAuth2 authentication, native JWT RS256 signing) and IndexNow via the api.indexnow.org relay that propagates to Bing, Yandex, Naver and Seznam in a single call. The module hooks into native PrestaShop hooks, queues each change in a deduplicated queue, and a CRON processes the batch every few minutes. You keep a complete submission log and an acceptance rate dashboard.

Requirements

• PrestaShop 8.0 to 8.99, or PrestaShop 9.x
• PHP 7.4 to 8.3
• PHP extensions openssl (for Google JWT RS256 signing) and curl (for HTTP requests)
• A system CRON or external CRON service to call queue processing every 5 to 15 minutes
• Optional — for Google: a Google Cloud account with a project to enable the Indexing API and create a Service Account, plus a verified Search Console property for your domain

Installation

Step 1 — Download

Download the ZIP dfindexingapi-1.0.0.zip from your DataFirefly account after purchase.

Step 2 — Install via the back-office

1. Log in to your PrestaShop back-office
2. Go to Modules › Module Manager › Upload a module
3. Click Select file and choose the downloaded ZIP
4. Confirm. PrestaShop extracts and installs the module
5. Once installed, click Configure

Step 3 — Post-install checks

On install, the module automatically creates:

• The two SQL tables ps_df_indexapi_queue (queue) and ps_df_indexapi_log (log)
• A 32-character alphanumeric IndexNow key
• A 32-character random CRON token
• 5 admin menu tabs: parent DataFirefly Indexing API, then Dashboard, Queue, Log, Configuration

Open the Configuration tab to proceed to the next step.

Configuring Google Indexing API

The Google Indexing API requires a Google Cloud Service Account. The procedure takes about 5 minutes.

Step 1 — Create a Google Cloud project

1. Go to console.cloud.google.com and sign in
2. At the top, click the project selector, then New Project
3. Give it a name (e.g. Indexing API Store) and create it
4. Select the newly created project

Step 2 — Enable the Indexing API

1. In the left menu, go to APIs & Services › Library
2. Search for Indexing API
3. Click Enable

Step 3 — Create a Service Account

1. Go to APIs & Services › Credentials
2. Click Create Credentials › Service Account
3. Give it a name (e.g. indexing-api-prestashop)
4. No IAM role is needed — go to the next step and finalize creation
5. In the service accounts list, click the account you created
6. Keys tab › Add key › Create new key
7. Format JSON. Download and keep the file safe — it cannot be retrieved again afterwards

Step 4 — Add the Service Account to Search Console

1. Copy the Service Account email (shaped name@project.iam.gserviceaccount.com) from Google Cloud
2. Go to Google Search Console
3. Select your property (your store domain)
4. Go to Settings › Users and permissions
5. Click Add user, paste the Service Account email, select the Owner role
6. Confirm

Step 5 — Paste the JSON in the module

1. Open the downloaded JSON file in a text editor
2. Copy its entire content
3. In the module configuration, section Google Indexing API, check Enable Google Indexing API
4. Paste the full JSON into the Service Account JSON field
5. Save

Step 6 — Test the connection

Click the Test Google button on the configuration page. The module signs a JWT RS256, exchanges it for an OAuth2 token, and shows the result. If all is well, you see a green Authentication OK message.

Configuring IndexNow

IndexNow is simpler to configure: no Service Account, no OAuth, no quota. Just a key to publish at the root of your domain.

Understanding IndexNow

IndexNow is an open protocol pushed by Microsoft Bing and Yandex in 2021, since joined by Naver and Seznam. You generate an alphanumeric key, you publish it at the root of your domain as a publicly accessible file, and you call api.indexnow.org with a list of URLs. The server verifies the key by reading the file on your domain, then propagates the URLs to participating engines.

Method 1 — .htaccess rewrite (recommended)

This is the simplest method: the module serves the key file content via a frontend controller, and a .htaccess rewrite rule at the root of your store routes the request to that controller.

1. In the module configuration, open the IndexNow tab
2. Check Enable IndexNow
3. Verify the Host field — it must match your store domain without the protocol (e.g. my-store.com)
4. Save
5. Copy the .htaccess snippet displayed on the configuration page — it is dynamically generated with your current key
6. Paste this snippet at the top of the .htaccess file at the PrestaShop root, just after the RewriteEngine on block
7. Click Test IndexNow in configuration — the module calls the key file URL on your domain and verifies it returns the expected content as text/plain

Method 2 — Physical file

If you cannot modify .htaccess, manually create a physical file at the domain root.

1. Retrieve your IndexNow key from the module configuration (IndexNow key field)
2. Create a file whose name is exactly the key + .txt (e.g. a1b2c3d4e5f6.txt) at the root of your domain
3. The file content must be only the key itself, with no line break
4. Verify that https://your-domain.com/a1b2c3d4e5f6.txt returns the key as text/plain
5. Click Test IndexNow

Configuring the CRON

The CRON is the element that runs queue processing. Without a regularly called CRON, submissions pile up but never leave.

process action — queue processing

To be called every 5 to 15 minutes. The module processes a configurable batch (default 50 jobs) following deduplication and indexing filters, then updates the log and the queue.

The exact URL is displayed in configuration. It looks like:

https://your-domain.com/index.php?fc=module&module=dfindexingapi&controller=cron&dfaction=process&token=YOUR_TOKEN

purge action — log cleanup

To be called once a day. The module deletes processed jobs and logs beyond the configured retention (default 30 days).

https://your-domain.com/index.php?fc=module&module=dfindexingapi&controller=cron&dfaction=purge&token=YOUR_TOKEN

key action — IndexNow key file

Only used by the .htaccess rewrite. You never call this URL manually.

Configure your system CRON

On Linux/cPanel, add two lines to crontab:

# Every 10 minutes — queue processing
*/10 * * * * curl -s "https://your-domain.com/index.php?fc=module&module=dfindexingapi&controller=cron&dfaction=process&token=YOUR_TOKEN" > /dev/null

# Once a day at 3 AM — log purge
0 3 * * * curl -s "https://your-domain.com/index.php?fc=module&module=dfindexingapi&controller=cron&dfaction=purge&token=YOUR_TOKEN" > /dev/null

CRON token security

The token is a 32-character secret generated on install. Without the correct token in the token= parameter, the controller returns HTTP 403. You can regenerate the token anytime from configuration (Regenerate CRON token button) — remember then to update your crontab lines with the new token.

The Dashboard

The Dashboard tab is your real-time overview.

Queue counters

Five cards at the top of the page:

• Pending — jobs created but not yet processed
• Processing — jobs locked for processing by an active CRON
• Submitted — jobs successfully processed (non-purged historical cumulation)
• Error — jobs that failed after N maximum retries
• Skipped — jobs created but ignored by a filter (e.g. URL_DELETED for IndexNow)

Provider diagnostics

Two cards show configuration status:

• Google Indexing API — active, misconfigured, or disabled. Shows whether the Service Account JSON is present and valid
• IndexNow — active, misconfigured, or disabled. Shows whether the key and host are configured

30-day acceptance rate

Cross-tabulation provider × status over the last 30 days, with semantic coloring: green above 90%, orange between 60 and 90%, red below. If Google falls below 90%, it usually signals you have exceeded quota or that URLs are no longer accessible.

Daily submissions chart

Chart.js chart overlaying two daily curves: total submitted and total accepted. Useful for quickly spotting drops or abnormal peaks.

The Queue

The Queue tab lists all jobs (pending, processing, submitted, error, skipped) with native PrestaShop filters by shop, object type, ID, provider, status, date.

Job statuses

• pending — created, waiting for processing by the next CRON
• processing — locked by an active CRON (logical transition to avoid parallel double processing)
• submitted — successful submission to the API. The retry counter is frozen
• error — all retries failed. Remains visible with the exact error message returned by the API
• skipped — created then ignored (e.g. URL_DELETED for IndexNow, or disabled filter)

Per-row actions

Each row offers:

• Retry — sets the job back to pending and resets the retry counter
• Delete — removes the job from the queue

Bulk actions

Buttons at the top of the list:

• Retry all errored jobs — sets all error jobs back to pending
• Purge processed jobs — deletes all submitted/skipped regardless of age

The Log

The Log tab lists each individual submission performed: provider, type, object ID, submitted URL, action (URL_UPDATED or URL_DELETED), HTTP code returned, accepted/refused indicator, full response message, and date. Filterable, sortable, exportable as CSV via the standard PrestaShop HelperList.

Indexing filters

In configuration, you can independently enable or disable three object types:

• Products — submission on creation, update, deletion, deactivation
• Categories — submission on creation, update, deletion. The category root (ID 1 and 2) is ignored for safety
• CMS pages — submission on creation, update, deletion

Disabling a filter immediately stops enqueuing for that type, but does not purge the existing queue.

Multi-store

The module is natively multi-store. Configuration (Google keys, IndexNow key, host, activations) is independent per sub-store. Jobs and logs are scoped by id_shop — the same product in two sub-stores generates two distinct jobs with their own canonical URLs.

To independently configure each sub-store, use the multistore selector at the top of the admin before opening configuration.

PrestaShop hooks listened to

The module registers the following hooks on install:

• actionProductSave — creation or update of a product. If active, enqueues URL_UPDATED; otherwise URL_DELETED
• actionProductDelete — final deletion of a product. Enqueues URL_DELETED
• actionObjectCmsAddAfter — CMS page creation
• actionObjectCmsUpdateAfter — CMS page update
• actionObjectCmsDeleteAfter — CMS page deletion
• actionCategoryAdd — category creation
• actionCategoryUpdate — category update
• actionCategoryDelete — category deletion
• displayBackOfficeHeader — CSS fragment injection for dashboard styling

Each hook builds the canonical URL via the official PrestaShop Link object, which respects your SEO friendly URL preferences and multilingual language prefixes.

Troubleshooting

Test Google fails with code 401

Authentication failed. Check that:

• The pasted Service Account JSON is complete and well-formed
• The Indexing API is properly enabled in Google Cloud (Library)
• The server system clock is correct — a drift of more than 5 minutes invalidates the JWT

Test Google fails with code 403

Authentication succeeds but Google refuses the request. Common cause: the Service Account has not been added as Owner of the Search Console property. Re-check step 4 of the Google configuration.

Test IndexNow fails

The api.indexnow.org server could not read the key file on your domain. Possible causes:

• The .htaccess snippet was not pasted, or pasted in the wrong place (it must be after RewriteEngine on)
• The physical file was not created, or has the wrong name (must be exactly the key + .txt)
• The file content does not match the key (typo, extra line break)
• The web server serves the file with the wrong Content-Type (must be text/plain)
• The firewall or CDN is blocking IndexNow robot requests

Open https://your-domain.com/YOUR_KEY.txt in a private browsing window — you must see only the key as plain text.

Jobs stuck in processing status

This means a CRON locked the jobs but never released the lock (e.g. the process was killed by a PHP timeout). You can unlock them manually via phpMyAdmin:

UPDATE ps_df_indexapi_queue SET status = 'pending', attempts = 0 WHERE status = 'processing';

If the issue recurs regularly, increase the PHP max_execution_time on your hosting, or reduce the batch size in module configuration.

Google quota exceeded

Google responds with code 429 or message Quota exceeded. Three options:

• Wait 24h — quota resets daily
• Request a quota increase from Google via Cloud Console (Quotas and system limits). Justify by explaining that your e-commerce store needs more daily submissions
• Create a second Service Account and alternate — each Service Account has its own 200 URLs/day quota

Full reset

To start from scratch (useful in case of migration or complex issue):

1. Uninstall the module from Modules › Manager
2. Reinstall — tables are recreated, IndexNow key and CRON token are regenerated
3. Reconfigure Google and IndexNow
4. Update the .htaccess snippet with the new key
5. Update the crontab lines with the new token

Known limits

• IndexNow does not handle URL_DELETED — the protocol considers a 404 or 410 on the URL the proper way to signal deletion. The module therefore ignores IndexNow jobs in URL_DELETED (Google submits them properly)
• Google quota capped at 200 URLs/day per Service Account by default
• Product variants not submitted individually — the main product canonical URL is enough, Google consolidates variants naturally
• Category root ignored (ID 1 and 2) to avoid submitting non-relevant URLs
• Google Indexing API officially limited to JobPosting or BroadcastEvent pages — the API accepts other types and responds 200, but Google may ignore the final indexing. For most stores, IndexNow remains the most impactful solution in practice

Support

For any technical question: support@datafirefly.com — reply within 24 business hours in English or French. Included for 12 months after purchase.