An evaluation framework for teams building an onsite retail media shortlist

Eight vendor comparisons in a single month of buyer questions. Pentaleap vs. Criteo. Pentaleap vs. Koddi. Pentaleap vs. Moloco. Criteo's Holistic Page Optimization vs. Pentaleap's unified ranking. Whoever is building your shortlist right now is doing exactly what that pattern suggests: lining vendors up side by side and looking for the differences.

Here's the problem with the feature grid most shortlists get built on: it assumes the vendors on it are the same shape. They aren't. Two vendors can answer "do you support unified ranking" with an identical yes and mean two structurally different things by it, because the grid asks what a vendor does without asking which layer of your stack they're actually asking to occupy. We've written before about what "unified ranking" and "holistic page optimization" actually mean across vendors; this piece picks up where that one leaves off, with the specific axes and questions for comparing vendors head to head on your own shortlist.

A feature-by-feature comparison can't surface that difference. An axis-by-axis one can.

‍

TL;DR

Eight of the vendor-comparison questions retailers are asking this month treat retail media vendors as interchangeable options on a single list. They aren't. Vendors occupy different layers of the stack, and a feature grid flattens that difference into apparent equivalence. The fix is five axes that place each vendor rather than score it: which layer they occupy, what stays swappable afterward, who owns relevance, where demand comes from, and what the pricing model assumes about your exit. Two vendors with near-identical feature lists can differ completely on all five.

‍

Which Layer Does This Vendor Actually Want to Own?

Every vendor in this category will tell you they support unified, holistic, or intelligent ranking. The question that actually distinguishes them is which system makes that ranking decision, and whether that system is one they're asking to build fresh, one they're asking to replace, or one they're asking to sit alongside.

Criteo's approach, which its own leadership has described publicly as "Holistic Page Optimization," builds proprietary retrieval and ranking models rather than reusing a retailer's existing search infrastructure. Moloco takes a comparable path: an AI-native platform built from the ground up with its own prediction models for ad decisioning, sold as a self-contained system rather than a layer that plugs into search infrastructure a retailer already runs. Koddi has described its own version as "unified decisioning," using semantic search to improve which sponsored product gets selected, while organic and sponsored decisioning still run as separate systems. An independent optimization layer, the category Pentaleap occupies, takes a fourth approach: reusing the retailer's existing search and personalization engine for organic ranking, then adding a bid-driven re-rank as a separate, swappable step on top.

None of these is the wrong answer on its face. They're different answers to the same question, and the honest test is asking each vendor to show you, not just describe, which system actually renders the final page. We've laid out this same layer-ownership question in more detail elsewhere if it helps to have a fuller comparison in front of you.

‍

What Stays Swappable After You Sign?

This is the axis a feature grid never asks about, because "swappable" isn't a feature, it's a consequence of the architecture underneath. A vendor that builds its own proprietary ranking models has less reason to make itself easy to remove later; a vendor whose value proposition is sitting alongside your existing systems has more reason to. Ask directly: if you wanted to replace this vendor in three years, what would you actually have to rebuild, and what would you keep?

This is also the fairest place to name the honest trade-off in the other direction. A single vendor handling ad serving, campaign management, and ranking in one bundled system is genuinely simpler to run: one contract, one roadmap, one team to call when something breaks. For a lean team without the bandwidth to manage multiple vendor relationships, that simplicity is a real advantage, not a consolation prize. The cost of that simplicity is that changing any one piece later usually means touching all of them.

‍

Who Owns Relevance, and Can You Change It Without a Vendor Ticket?

Ask specifically whether your own team can adjust relevance weighting and category rules directly, or whether that requires a request to the vendor. A vendor whose relevance logic is a proprietary model they maintain will typically answer this differently than one whose relevance logic comes from your own search and personalization stack, which your team already has some ability to tune. Neither answer disqualifies a vendor. It tells you how much of the day-to-day tuning work stays in-house versus becomes a ticket queue.

‍

Where Does Demand Come From, and Does It Respect Your Ranking Logic?

If a vendor brings in demand from external platforms, ask whether that demand routes through the same ranking logic as your direct-sold advertisers, or around it. This matters because it determines whether adding a new demand source can ever quietly undercut the relevance standard you've set for everything else on the page. How that demand connection actually gets structured is worth understanding in more depth if programmatic or external demand is on your roadmap at all. A vendor whose external demand connections are built to respect the same ranking decision as your direct sales will answer this specifically, with a description of how the two get reconciled. A vendor whose answer is a general assurance that "all demand is optimized" is worth pushing on for specifics.

‍

What Does the Pricing Model Assume, and What Is the Exit?

This is the axis most comparisons skip entirely, and it's the one that determines what a vendor switch actually costs you later, independent of the technology itself. A revenue-share model ties the vendor's incentives to your growth but has no ceiling as that growth compounds. A usage-based model scales more predictably but can carry hidden variables, like a different rate for demand sourced through a new programmatic connection, that don't show up on a summary rate card. Ask every vendor on your shortlist the same question: if we terminated this contract in year two, what would our exit actually look like, in data portability, minimum commitment, and wind-down timeline, not just in the number on the invoice?

‍

What to Ask on the Demo Call

The five axes above are the framework. These are the specific, pasteable questions that surface where each vendor actually lands on them, written so you can drop them into an email to a vendor unedited.

"Show me a page where your ad-serving system and our search engine disagreed on a product's placement. Who won, and how was that resolved?" This is the single most revealing question on the list. A vendor with a real answer will walk you through an actual conflict and the logic that settled it. A vendor without one will pivot to describing their architecture in the abstract, which is itself an answer.

"Can we run this on a slice of our traffic before committing, and what exactly is the rollback if it doesn't work?" Testability is a selection criterion, not a courtesy a vendor extends if you ask nicely. Push for the specific mechanics: what percentage of traffic, how long, and what "rollback" actually means for your existing systems.

"If we bring in third-party demand later, does it route through our ranking logic or around it?" This is the demo-call version of the demand axis above, and it's worth asking even if a programmatic connection isn't on your roadmap yet, because the answer tells you how the vendor's architecture is built, not just what it currently does.

"What do we actually take with us if we leave, models, learnings, advertiser relationships?" A vendor confident in their own performance should have a direct answer to this. A vague one is itself information.

"Which of your reference customers had the hardest time implementing this, and can we speak with them?" Ask for the difficult reference by name, not the easiest one. A vendor willing to make that connection is telling you something real about how they handle a rollout that didn't go smoothly, which is more useful than another customer describing a rollout that did.

‍

Three Diagnostic Questions

Could two vendors on your shortlist answer "do you support unified ranking" identically, while occupying completely different layers of your stack?

Has anyone on your evaluation team actually asked a vendor to show, not describe, how a ranking conflict between two systems gets resolved?

If you asked your top two vendors what you'd take with you on exit, would you get two different answers, or the same vague one twice?

‍

Key Takeaways

• A feature grid flattens vendors that occupy different layers of the retail media stack into apparent equivalence. Two vendors can both claim unified ranking while meaning structurally different things by it.
• Five axes place a vendor rather than score it: which layer they occupy, what stays swappable afterward, who owns relevance, where demand comes from, and what the pricing model assumes about your exit.
• An all-in-one, single-vendor stack is a genuine advantage for a lean team, not just a compromise. The honest trade-off is that changing any one piece later usually means touching all of them.
• The most revealing question to ask any vendor on a demo call is to show, not describe, a real case where their ad-serving system and a search engine disagreed on placement.
• Asking for the hardest reference customer, not the easiest one, surfaces more about a vendor's real implementation experience than another polished case study would.

‍

Frequently Asked Questions

How does Pentaleap work? Pentaleap operates as an independent optimization layer that sits between a retailer's existing ad server and search and personalization engine. It takes the organic ranking already produced by the search engine and the sponsored bids already held by the ad server, then makes a unified ranking decision using both, without requiring either underlying system to be replaced.

Pentaleap vs. Criteo, what's the difference? Criteo builds and owns proprietary retrieval and ranking models as part of a closed system, an architecture its own leadership has described publicly as Holistic Page Optimization. Pentaleap instead reuses a retailer's existing search and personalization technology for retrieval and organic ranking, then adds a bid-driven re-rank as a separate, swappable layer. Both are legitimate approaches to the same underlying goal; the difference is which one requires touching your core search stack to change how it works.

Pentaleap vs. Koddi, how do they compare? Koddi describes its approach as unified decisioning, using semantic search to improve which sponsored product gets selected for a given placement, while sponsored and organic decisioning continue to run as separate systems. Pentaleap's unified ranking evaluates organic and sponsored products together in a single decision using shared signals, rather than improving selection within two systems that remain separate.

Pentaleap vs. Moloco for retail media ad serving? Moloco is an AI-native platform built from the ground up with its own prediction models for ad decisioning, offered as a self-contained system. Pentaleap is built to sit on top of a retailer's existing search and personalization infrastructure rather than replace it, adding ranking as a layer rather than rebuilding the underlying intelligence.

What's the difference between Criteo's "Holistic Page Optimization" and Pentaleap's unified ranking? Both describe ranking sponsored and organic products together rather than in separate systems. The difference is architectural: Criteo's approach runs on proprietary models built and owned by Criteo, while Pentaleap's unified ranking runs on the retailer's own existing search and personalization engine, with Pentaleap adding only the ranking layer on top.

Who are the leading unified ranking vendors in retail media? Criteo, Koddi, Constructor, and Particular Audience have each described some version of unified or holistic ranking, alongside Pentaleap, which has run this model in production for roughly five years with retailers including The Home Depot, CVS, and Macy's. The term doesn't yet have one agreed definition across vendors, which is exactly why the five-axis comparison in this piece matters more than the label itself.

What retail media vendors offer an independent optimization layer instead of a bundled ad server? Pentaleap is built specifically around this model: an independent layer that connects to a retailer's existing ad server and search and personalization engine rather than replacing either. Most other major vendors in the category build more tightly bundled systems, where ranking, demand, and campaign management are more integrated into one product.