The first time I had to replace a tokenization vendor, I discovered that our tokenization strategy had been wrong for three years and nobody had noticed. The tokens were scoped to the vendor. Our database was full of them. Migrating to a new vendor meant re-tokenizing every stored payment method, which required retrieving the card data from the old vendor and submitting it to the new one, and the old vendor's contract didn't include bulk export. We spent six months negotiating a data transfer, then another six months processing the re-tokenization in batches, carefully validating at each step that no cards were lost in translation. The migration itself was the easy part; the re-tokenization was a second full-time project, and it existed only because we had treated tokenization as an implementation detail rather than a product architecture.
Tokenization is the most underrated architectural decision in payment systems. Teams pick a tokenization approach early, rarely revisit it, and end up with constraints that shape everything they can do with payment methods for years. I want to make the case for thinking about tokenization differently: not as a scope-reduction mechanism, but as the foundation for what your platform can do with customer payment methods.
What a token actually is
A token is a string that stands in for cardholder data in places where you don't want the real data to be. When your application needs to reference a payment method, it uses the token. When an operation actually has to touch the card — an authorization, for instance — the token is resolved to the real data in a restricted, audited context, the operation happens, and the real data is discarded from memory immediately.
That's the mechanism. The interesting part is what the token is scoped to, what operations it permits, and how portable it is across time and systems. These properties determine what you can do with a tokenized payment method, and they're where design decisions live.
Tokens come in at least three different flavors
The word "token" means at least three different things in a payment context, and teams confuse them constantly.
Processor tokens are tokens issued by a specific payment processor. Stripe gives you a Stripe customer's payment method ID. Adyen gives you a recurring detail reference. Braintree gives you a payment method token. These tokens are usable only with the processor that issued them. If you switch processors, these tokens are worthless.
Network tokens are tokens issued by the card networks (Visa, Mastercard, American Express) that represent the card itself. They're designed to be portable across processors — any processor that supports network tokenization can use them. Network tokens also have lifecycle benefits: when a card is reissued with a new number, the network token stays the same, and your stored reference doesn't need updating.
Merchant tokens (sometimes called "vault tokens") are tokens issued by an independent vault service that stores the actual card data. The vault resolves the token to the real data when you need to submit a payment request, and the data is sent to whichever processor you choose. These tokens are processor-independent.
Each of these has different properties:
| Property | Processor Token | Network Token | Merchant Token | |---|---|---|---| | Portable across processors | No | Yes | Yes | | Survives card reissuance | No | Yes | Depends | | Usable for recurring | Yes | Yes | Yes | | Revocable by user | Via processor | Via network | Via vault | | Required PCI scope | Minimal | Minimal | Vault in scope |
The right answer is usually a combination. Use a merchant vault for portability and control, and layer network tokens on top when the networks and processors support them. Processor tokens are convenient but should be treated as a lock-in tax — every processor token you store is a future migration cost.
Format-preserving vs. opaque tokens
When generating tokens, there's a choice between format-preserving (tokens that look like card numbers — same length, passing the Luhn check, with recognizable BIN) and opaque (tokens that are arbitrary strings with no structure).
Format-preserving tokens have one advantage: they can be substituted into systems that expect a card number format. If your downstream system expects a 16-digit PAN, a format-preserving token fits without changing schemas. This was historically valuable for legacy integration, but in practice, any modern system should just accept arbitrary strings.
Opaque tokens are safer. They can be longer (hard to brute-force), they carry no information about the underlying card, and they can't be confused with a real PAN if they leak. A token like pm_5d8f3a9b4c2e1f7a9d6b5c4e3f2a1b0c is clearly a token, not a card number.
Unless you have a specific reason to need format-preservation (regulatory requirement, legacy integration), use opaque tokens. Generate them with sufficient entropy (at least 128 bits, more for high-value use cases) and a deterministic format that allows quick validation (a known prefix, a known length, a known character set).
The vault architecture
A merchant tokenization vault is a narrow, high-security service whose only job is to store card data and resolve tokens. Its architecture is intentionally simple — complexity in security-critical systems is where vulnerabilities live.
The critical property: card data never leaves the vault perimeter except to go to a payment processor in a controlled way. The client that tokenizes sends card data in; it gets a token back. Anyone calling the vault with a token gets the result of an operation, not the card data itself. The vault, the encryption keys, and the processor connectors are all inside the PCI scope; everything else is outside.
Encryption at rest uses a key management system (AWS KMS, Google Cloud KMS, HashiCorp Vault, or similar) with hardware-backed key storage. Each card is encrypted with a data encryption key (DEK); the DEK itself is wrapped by a key encryption key (KEK) that never leaves the KMS. To decrypt a card, the vault asks the KMS to unwrap the DEK, then uses the DEK in-memory to decrypt the card, then discards everything. This is standard envelope encryption, and it's what makes the vault safe against database compromise — the database alone is not sufficient to decrypt anything.
Key rotation is not optional
Cryptographic keys have lifecycles. They get rotated — sometimes on a schedule, sometimes in response to an incident. A vault that doesn't support rotation is a vault that will eventually hit a compliance deadline it can't meet, or an incident response it can't execute.
Rotation strategies:
Envelope rotation: The KEK rotates, but the DEKs don't. Existing DEKs are re-wrapped with the new KEK. This is fast because you're not re-encrypting card data, just the small keys that encrypt it. It's the right choice for routine rotations.
Full rotation: Both the KEK and the DEKs rotate. Every card is re-encrypted with a new DEK, which is wrapped by a new KEK. This is expensive at scale but provides the strongest guarantee — a compromised old key is useless even against stored data.
The vault design has to support both. Envelope rotation is routine and automated. Full rotation is rare but must be executable — ideally as an online process that doesn't require downtime.
During a rotation, some data is under the old keys and some under the new. The decryption logic has to know which version to use, which usually means storing the key version alongside each encrypted blob. Keep this simple — complexity in the rotation path is where bugs hide.
Lifecycle of a token
A token is not a permanent identifier. Its lifecycle has more stages than most teams account for.
Creation. The token is generated when a customer's card is first added. At this point, the card data is encrypted and stored, and the token is returned to the client.
Active use. The token is used for authorization, capture, refund, and other operations. Each use can be logged for audit.
Update. The underlying card data might change. The card's expiration updates. The cardholder name changes. The card is replaced by a new one from the same issuer. Some of these updates come via account updater services (Visa Account Updater, Mastercard Automatic Billing Updater), which notify you when a card changes and can optionally provide the new card number. The token should survive these updates — the customer's reference to their payment method shouldn't change just because the card was reissued.
Deactivation. The customer removes the payment method, or the card is reported lost/stolen, or the merchant closes the account. The token is marked deactivated. Future use attempts fail.
Deletion. Depending on retention requirements, the token may eventually be deleted entirely. This is different from deactivation — deletion removes the encrypted card data, not just the token's ability to be used.
Each of these transitions needs explicit support in the vault's data model and API. A token that supports only creation and active use is a token that can't handle real-world card lifecycles.
Customer-portable payment methods
One of the most valuable capabilities a vault provides, if designed right, is customer portability of payment methods. A customer who adds their card on your platform should be able to use it across any merchant on your platform (with appropriate permissions), across any processor you integrate with, and through any access channel (web, mobile, in-store terminal).
This is hard if you use processor tokens, because each processor has its own scope. It's trivially easy if you use merchant vault tokens, because the vault controls the scope.
The vault's customer record:
CREATE TABLE vault_customers (
customer_id UUID PRIMARY KEY,
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE TABLE vault_payment_methods (
token TEXT PRIMARY KEY,
customer_id UUID NOT NULL REFERENCES vault_customers,
card_brand TEXT NOT NULL, -- 'visa', 'mastercard', etc.
bin TEXT NOT NULL, -- first 6 digits, safe to store
last_four TEXT NOT NULL, -- last 4 digits, safe to store
exp_month SMALLINT NOT NULL,
exp_year SMALLINT NOT NULL,
billing_zip TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
deactivated_at TIMESTAMPTZ,
network_token TEXT
);
The vault stores metadata about the card (brand, BIN, last four, expiration) in the clear — this data is not PCI-sensitive and is needed for UX (showing the customer which card they're using) and routing (some processors handle certain brands better than others). The sensitive data (full PAN, CVV) is encrypted and accessible only through authenticated API calls.
Customer portability means the same customer_id and token work across all your integrations. A customer who added a card on your web checkout can have it appear on a POS terminal's customer profile, and the in-store transaction uses the same token that the web transaction used. No re-entry, no re-tokenization, no per-channel configuration.
De-tokenization auditing
Every time a token is resolved to card data, that's an auditable event. It happens inside the vault, it's logged there, and the logs are retained. This is a PCI requirement and a security necessity.
The audit record includes:
- Which token was resolved
- Which caller requested it (authenticated identity)
- What the purpose was (authorization, refund, update, etc.)
- When it happened
- Where the result was sent (which processor, which endpoint)
- Whether it succeeded
Patterns in this log are meaningful. A sudden spike in de-tokenization for a specific customer might indicate fraud. A service that's de-tokenizing tokens it shouldn't need is a security concern. Unusual de-tokenization from unexpected geographies is an indicator of credential compromise. The audit log is not just a compliance artifact — it's a security signal.
Access to the audit log itself is restricted. Only specific roles can read it; nobody can modify it. It's stored in write-once storage or an append-only database where the history is tamper-evident.
Vault vendor risk
Most teams don't run their own vault — they use a vendor (Very Good Security, Basis Theory, Skyflow, and others). This is usually the right choice; running a PCI-compliant vault is expensive and not a differentiator for most POS platforms.
But vendor selection matters a lot, and the decision is hard to reverse. Things to verify:
Data portability. Can you get your card data out of the vault, in an encrypted form you can use with another vendor? If not, you're locked in. Some vendors won't offer this at all; others charge for it; others support it but the format is proprietary. Check before you sign.
PCI scope transfer. Using the vendor must meaningfully reduce your PCI scope. Ask the vendor for the specific scope reductions, and verify with your QSA that the reductions apply to your architecture. Some vendor setups only reduce scope if you're using their full platform; if you just use them for storage and do tokenization in your own code, scope may not reduce.
Availability SLA. Vault downtime means you can't process transactions against stored payment methods. For recurring billing platforms, this is an outage. For POS platforms, it affects any customer trying to pay with a saved card. The SLA should match your business requirements, and the penalties for missing it should be meaningful.
Geographic coverage. If you operate internationally, the vault has to support the regions you need. Some vaults are US-centric and don't handle EU data residency requirements.
Vendor health. Vaults are critical infrastructure. A vendor that goes out of business is a very bad scenario. Check financial health, customer growth, and how long they've been operating.
The decision is not just technical. It's commercial, operational, and strategic.
What a vault enables
If the vault is well-designed, it enables capabilities that a processor-token-based approach cannot:
- Multi-processor routing at the transaction level. The same token can be used against different processors based on routing logic, without re-tokenizing.
- Merchant-of-record flexibility. The token is owned by you; you can assign it to different merchant accounts for different use cases.
- Customer portability. A customer's payment methods follow them across merchants in your platform.
- Processor migration. Switching processors doesn't require re-tokenizing; you just update the connector.
- Compliance consolidation. Your PCI scope is narrow and well-defined, regardless of how many processors you integrate with.
Each of these is a capability your platform can offer that competitors using processor-specific tokens cannot. These aren't hypothetical; they're the difference between platforms that can adapt to customer needs and platforms that are locked into their original processor choice.
The architectural point
Tokenization is often treated as the consequence of a PCI compliance decision: "we need to reduce scope, so we'll tokenize." That's the minimum version. The better framing is that tokenization is the primary way you take control of your platform's relationship with payment methods — their storage, their portability, their lifecycle, their use across channels and processors.
A platform that treats tokenization as a product capability ends up with flexibility that a platform treating it as a compliance tax cannot match. The cost is higher upfront — a real vault is more work than a processor-token integration — but the cost curve is bounded, and the capabilities scale.
Choose the tokenization approach early and carefully. It will shape what your platform can do for years, and the migrations required to change it are some of the longest, most expensive engineering projects a payment platform will ever do.
Your tokens are not implementation details. They are the schema of every customer's relationship with your platform. Design them like it.
This is part of a series on payment systems architecture. See also PCI compliance shapes your entire architecture and the real cost of payment integration nobody talks about.