Manual keyword grouping works until the list grows, naming becomes inconsistent, and different marketers classify the same query differently. A Python workflow turns that fragile process into a repeatable operation: ingest queries, normalize them, calculate similarity, form clusters, flag uncertain results, and export a plan that people can actually use.

The objective is not to remove editorial or paid-search judgment. It is to spend that judgment on ambiguous groups rather than obvious duplicates. This matters as advertising platforms automate more creative work. Google’s testing of AI-generated descriptions in Shopping ads illustrates the direction of travel: platforms may generate presentation, but marketers still need clean product, intent, and query taxonomies to guide campaign structure and evaluate outcomes.

Define the output before selecting an algorithm

Begin with the operational decision the clusters must support. A content team may need one row per topic, a primary keyword, supporting queries, intent, and a suggested page type. A paid-search team may need campaign and ad-group labels, product categories, negative-keyword candidates, and landing-page mappings. These are related outputs, but they should not be treated as interchangeable.

Write an export specification before coding. At minimum, retain the original query, normalized query, source, volume or other available business metric, cluster ID, cluster label, representative keyword, confidence indicator, and review status. Add an immutable row ID so results can be reconciled with the source system.

Set rules for what should not be merged. Terms can look similar while implying different products, audiences, locations, funnel stages, or tasks. “Running shoes for women” and “women’s running shoe size guide,” for example, share vocabulary but may warrant separate commercial and informational destinations. Your taxonomy must reflect the intended action, not merely linguistic resemblance.

Collect and clean queries without erasing meaning

Load source files into a consistent table with pandas. Sources might include search-console exports, keyword research tools, internal site search, paid-search reports, product feeds, and planning documents. Record provenance because a query observed in first-party performance data should not automatically receive the same priority as an untested suggestion.

A deterministic cleaning function should lowercase text, normalize whitespace and Unicode, standardize punctuation, and remove exact duplicates. Keep both raw and cleaned values. Treat stemming, lemmatization, stop-word removal, and singularization as configurable options rather than universal defaults. Removing words such as “for,” “near,” or “without” can destroy commercial, local, or constraint-based intent.

Create separate structured features where useful: brand, product type, model, audience, location, modifier, and intent. For Shopping applications, align those fields with the product taxonomy and feed terminology. Maintain an exclusion table for navigation queries, irrelevant terms, regulated topics, or phrases that require specialist review.

Combine similarity signals and cluster at scale

No single similarity signal captures every useful relationship. Lexical methods such as token overlap or TF-IDF are transparent and effective for close variants. Semantic embeddings can recognize conceptually related phrases with different wording, but may over-merge terms whose business intent differs. Structured attributes provide constraints that language similarity alone cannot guarantee.

A practical pipeline can use staged matching. First, group exact normalized duplicates and highly similar lexical variants. Next, calculate semantic similarity for the unresolved set. Then apply business rules that block incompatible combinations, such as different brands, product categories, countries, or intent classes. This reduces unnecessary pairwise comparisons and makes decisions easier to explain.

Choose the clustering method according to the data and governance requirement. Hierarchical clustering is helpful when teams want to inspect relationships at different thresholds. Density-based approaches can identify irregular groups and leave outliers unassigned. Graph-based clustering works well when keywords are nodes and sufficiently similar pairs become edges. Avoid choosing a fixed number of clusters merely because an algorithm requires it unless the business genuinely has a fixed taxonomy.

For scale, cache embeddings and cleaned features, process data in batches, and avoid calculating every possible pair when candidate retrieval can narrow comparisons. Version the code, model, thresholds, configuration, and input snapshot. A cluster export without that lineage cannot be reproduced or audited.

What most people miss

The cluster label is an operational deliverable, not decoration. Automatically naming every group with its highest-volume phrase can hide mixed intent or produce awkward headings. Generate a provisional label from a representative query or common attributes, then allow reviewers to edit it without changing the underlying cluster ID. That separation preserves lineage while producing labels suitable for briefs, campaign names, and reporting.

Route uncertainty to human reviewers

Automation should triage decisions rather than conceal uncertainty. Assign each cluster a review score using signals such as average similarity to its representative keyword, distance from neighboring clusters, conflicting intent labels, incompatible entities, cluster size, and the proportion of outliers. Route borderline cases to a review queue.

Give reviewers specific actions: approve, split, merge, relabel, exclude, or escalate. Show the representative query, members, key metrics, source mix, and nearest alternative cluster. Sample some high-confidence groups too; otherwise systematic errors can pass unnoticed because only flagged cases receive attention.

Capture every correction as structured feedback. Recurring splits may indicate that a modifier needs more weight. Recurring merges may reveal an overly strict threshold or inconsistent product vocabulary. Do not silently tune settings after each run; test revisions against a stable, human-reviewed benchmark set.

Export plans and measure operational value

For content planning, export one cluster summary per proposed page plus a member-level sheet. Include primary and secondary queries, intent, recommended format, existing URL, cannibalization risk, priority, and owner. Do not automatically create a page for every cluster: some should enrich an existing page, become a section, or remain unserved.

For paid search and Shopping, map clusters to product categories, landing pages, campaign structures, negatives, and reporting labels. AI-generated ad descriptions do not eliminate this work. If query groups and product attributes are muddled, automated copy may still be evaluated against an incoherent structure. Clean taxonomy helps teams diagnose whether performance differences come from creative automation, product coverage, targeting, or landing-page fit.

Evaluate the system on both quality and efficiency. Track reviewer agreement on a benchmark sample, split and merge rates, outlier rate, low-confidence share, and cluster stability between runs. Operational metrics should include processing time, reviewer minutes per thousand queries, total turnaround time, and correction volume. Downstream measures can include briefs produced, duplicate-page proposals prevented, query-to-landing-page coverage, and campaign mapping completeness. Compare these with the former manual process; faster output is not a success if correction work rises or misleading clusters reach production.

Implementation checklist

• Define whether the output serves content, paid search, Shopping, or separate workflows.
• Specify required export columns, review states, and immutable identifiers.
• Preserve raw queries, source provenance, and business metrics.
• Build conservative, configurable cleaning rules and test meaning-changing edge cases.
• Combine lexical, semantic, and structured similarity signals.
• Add blocking rules for incompatible brands, products, locations, and intents.
• Select a clustering method that supports outliers and explainable review.
• Cache expensive features and version inputs, models, thresholds, and code.
• Calculate confidence signals and create an explicit human review queue.
• Record approve, split, merge, relabel, exclude, and escalation decisions.
• Validate changes against a fixed, human-reviewed benchmark set.
• Export both cluster summaries and query-level mappings.
• Measure time saved alongside correction rates, stability, and downstream usefulness.