AI Product Descriptions for Large Catalogs: The Pipeline I Use After a 90% Clicks Drop

Why AI product descriptions collapse at catalog scale

I run an auto-parts store as a client project. Buyers there search by part number first, keyword second, so the product page is the landing page and the description is the only prose on it. Product content was hand-written from the start.

In September the owner pushed a batch of AI-generated content onto the site without telling me. Rankings and clicks fell by roughly 90% inside weeks. Nothing else changed: same platform, same speed, same links. The parts pages that had been ranking on part-number queries for a year went with them, because the drop was site-wide, not page-wide.

That is the part most guides miss. A bulk description run is not scored page by page. If a few thousand near-identical pages land at once, they drag the pages you wrote by hand down with them. Three failure modes cause nearly all of it:

• Template collapse. The model gets a title and a category and nothing else, so every description in a product group is a paraphrase of every other one.
• Ungrounded facts. The model invents a torque figure, a thread pitch, a fitment year. One wrong number on a parts page costs a return, not a ranking.
• Merchant copy, reworded. Pulling the manufacturer blurb and synonymizing it is the oldest version of this mistake and it is explicitly named in Google's policies.

Does Google penalize AI-generated product descriptions?

No, not for being AI-generated. Google's spam policies for Google web search define scaled content abuse as generating many pages primarily to manipulate rankings rather than to help users, and the policy says it applies no matter how the content was created. Generative AI is listed as one method among scraping, stitching and automated transformation — not as the thing being policed.

Two clauses matter more than the headline one, and neither gets quoted in the vendor posts:

• Thin affiliation covers pages where product descriptions are copied from the original merchant with no original content or added value. If your "AI pipeline" is really a scrape-and-reword of the manufacturer's blurb, that is the clause you are under, and no amount of temperature tuning changes it.
• Scraping covers reproducing a content feed from another site without a unique benefit to the user. Distributors hand out feeds. Everyone selling that distributor's catalog gets the same feed.

The practical test I apply before generating anything: what does this page contain that the other fourteen stores selling this exact part do not have? If the honest answer is "nothing", generating prose will not create the difference — it will just make the sameness longer.

The data you need before you generate anything

Description quality at scale is a data problem wearing a prompt problem's clothes. The prompt is maybe 10% of the outcome. What decides it is how many distinct, true, SKU-specific facts you can hand the model.

My rule of thumb: if you cannot state five facts about a SKU that are not already in its title, do not generate a description for it. Ship the attribute table and the image instead, and leave the prose empty until the data exists.

When the data does not exist in a clean form, that is the actual project. On the parts catalog the blocker was never generation — it was that the shop's catalog system had no API, so the attributes lived inside a desktop tool. Extraction came first, prose came second. Budget accordingly: on a large catalog I expect to spend most of the time on normalization and attribute extraction, and comparatively little on the generation step.

The pipeline I use for a 10,000-SKU catalog

Six stages, each one writing to disk so any stage can be re-run without redoing the ones before it. On a catalog this size you will re-run stages four or five times.

1. Normalize. One row per SKU, one column per attribute, units standardized, nulls explicit. This is the artifact everything else reads.
2. Extract. Pull attributes out of PDFs, supplier sheets and scraped pages into that table. Use the model here — extraction into a fixed schema is what LLMs are genuinely reliable at.
3. Cluster. Group SKUs by category and variant axis. Each cluster gets its own prompt, its own required-facts list and its own forbidden-phrase list.
4. Generate. One call per SKU, structured output, temperature low. Pass only that SKU's attributes — never the category average, or the model will fill gaps with the cluster's mean product.
5. Gate. Automated validation, below. Failures go back to stage 2 with a reason, not to a human.
6. Sample. A person reads 2% of the output, weighted toward the clusters with the thinnest data.

The gate is the stage people skip, and it is the one that saves the project. Minimum viable version:

import re

MERCHANT_CENTER_MAX = 5000

def gate(sku: dict, text: str) -> list[str]:
    problems = []

    # 1. The SKU's own identifier must appear in its own description.
    if sku['part_number'].lower() not in text.lower():
        problems.append('part number absent')

    # 2. Every number in the output must trace back to an input field.
    known = ' '.join(str(v).lower() for v in sku.values() if v)
    for num in set(re.findall(r'\d[\d.,]*', text)):
        if num not in known:
            problems.append(f'ungrounded number: {num}')

    # 3. Feed-breaking content (see the Merchant Center section).
    for banned in ('free shipping', 'best price', 'click here', 'www.', 'http'):
        if banned in text.lower():
            problems.append(f'disallowed in feed: {banned}')

    if len(text) > MERCHANT_CENTER_MAX:
        problems.append('over the 5000-character description limit')

    return problems

Rule 2 is the one that earns its keep. Hallucinated specifications in a parts catalog are not an SEO problem, they are a returns problem, and a string check against the input row catches the large majority of them for no model cost at all. If you are scoping this kind of build rather than running it yourself, it sits squarely in AI development work for content and data agents, not in copywriting.

Merchant Center rules bulk AI descriptions break

If the catalog feeds Google Shopping, the description has a second set of rules on top of search, and a generated description will break them by default because the model writes marketing copy and the feed does not accept marketing copy. From the Merchant Center product data specification:

The attribute almost nobody uses

Merchant Center has dedicated attributes for AI-written copy. structured_title and structured_description each take two sub-attributes: digital_source_type and content. Setting digital_source_type to trained_algorithmic_media declares that the text was produced with generative AI; leaving it out defaults to default, meaning not AI-generated. Google's own instruction is direct: use the structured attribute for copy created with generative AI, and the plain one otherwise.

Two reasons to actually do this. First, it is a declaration about your data, and declaring it accurately costs you nothing. Second, AI disclosure rules in the EU, India and New York already require labelling for certain AI-generated or AI-edited ad assets — this is the mechanism that exists for it, rather than something you will have to retrofit later.

The same thinking applies to images: AI-generated product images are required to carry IPTC DigitalSourceType metadata, and the spec says not to strip those tags during processing. If your image pipeline re-encodes files, check that it preserves them — most naive resize scripts do not. Worth noting at the same time: the 500×500 pixel minimum for product images is enforced from 31 January 2027, so a bulk image pass and a bulk description pass can share a run.

Two attributes do more for Shopping performance than prose ever will: product_highlight (2 to 100 entries, 150 characters each) and product_detail (up to 100 section/name/value triples). Both are structured, both are far easier to generate reliably than paragraphs, and neither can collapse into template sameness the way prose does. If you have thin data and a big catalog, generate these first and treat the description as optional.

How to QA 10,000 descriptions without reading 10,000 descriptions

Nobody reads ten thousand descriptions. The job is to make the machine surface the few hundred worth a human's attention.

The duplicate check in practice:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.metrics.pairwise import cosine_similarity

# char_wb n-grams catch reworded sameness that word tokens miss.
vec = TfidfVectorizer(analyzer='char_wb', ngram_range=(3, 5), min_df=2)

def flag_duplicates(cluster_texts, threshold=0.85):
    X = vec.fit_transform(cluster_texts)
    sim = cosine_similarity(X)
    return [(i, j, sim[i][j])
            for i in range(len(cluster_texts))
            for j in range(i + 1, len(cluster_texts))
            if sim[i][j] > threshold]

Two practitioner notes. Run this within clusters, not across the whole catalog: a 10,000 × 10,000 similarity matrix is 100 million floats, around 800 MB in float64, and you will hit memory limits on a modest box for no benefit — descriptions of a brake pad and a headlight are not supposed to be similar anyway. And on library choice, scikit-learn is BSD-3 licensed, so it is safe for client and commercial work; that matters more than the API when the deliverable ships to someone else.

What I do with the flags: clusters that fail duplicate or template-collapse checks do not get fixed by regenerating. They get sent back to data extraction, because the output is sameness-shaped for the honest reason that the inputs were.

Roll out in batches, measure by SKU cluster

The expensive lesson from the 90% drop was not about generation at all. It was that everything went live at once, which made the damage impossible to attribute and slow to unwind.

What I do instead:

• Batches of 300 to 500 SKUs, two to three weeks apart, never the whole catalog in one deploy.
• Label the batch in your own data so you can segment Search Console by it. Without the label you are guessing.
• Segment queries by type. Part-number and model-number queries behave nothing like descriptive keyword queries. On the parts catalog, identifier queries convert and keyword queries mostly do not — a sitewide clicks average would have hidden that entirely.
• Watch the control group. Track the pages you did not touch. If untouched pages drop while a batch is live, the problem is site-level quality, which is exactly the signal a single big-bang deploy destroys.
• Keep the previous version. Store the prior description per SKU so a rollback is an update statement, not a re-run.

One distinction worth holding onto: generating structured product data at scale and generating editorial content are different jobs with different risk profiles. A grounded attribute-to-description pipeline with hard gates is a reasonable engineering task. Pointing the same machinery at your blog is how sites end up in the scaled-content conversation in the first place.

If you have a catalog in the thousands and the data problem underneath it looks bigger than the writing problem, that is the normal shape of this work — tell us what the catalog looks like and we will scope it, whether the answer turns out to be extraction, feed work, or generation.