The Transients API is an incredibly useful API in WordPress, and unfortunately
one of the most misunderstood. I’ve seen almost everyone misunderstand what
transients are used for and how they work, so I’m here to dispel those myths.
For those not familiar with transients, here’s a quick rundown of how they work.
You use a very simple API in WordPress that acts basically as a key-value store,
with an expiration. After the expiration time, the entry will be invalidated and
the transient will be deleted from the transient store. Transients essentially
operate the same as options, but with an additional expiration field.
By default in WordPress, transients are actually powered by the same backend as
options. Internally, when you set a transient (say foo), it gets transparently
translated to two options: one for the transient data (_transient_foo) and an
additional one for the expiration (_transient_timeout_foo). Once requested,
this will then be stored in the internal object cache and subsequent accesses in
the same request will reuse the value, in much the same way options are cached.
One of the most powerful parts of the transient API is that it uses the object
cache, allowing a full key-value store to be used in the backend. However the
default implementation, and how the object cache can change this, is where two
major incorrect assumptions come from.
Object Caching and the Database
The first incorrect assumption that developers make is to assume the database
will always be the canonical store of transient data. One big issue here is
attempting to directly manipulate transient data via the option API; after all,
transients are just a special type of option, right?
In the real world however, anything past your basic site will use an object
cache backend. Popular choices here include APC (including the new APCu) and
Memcache, which both cache objects in memory, not the database. With these
backends, using the option API will return invalid or no data, as the data
is never stored in the database.1
I’ve seen this used in real world plugins to determine if a transient is about
to expire by directly reading _transient_timeout_foo. This will break and
cause the transient to always be counted as expired with a non-default cache.
Before you think about how to do this in a cross-backend compatible way: you
can’t. Some backends simply can’t do this, and until WordPress decides to
provide an API for this, you can’t predict internal behaviour of the backends.
Expiration
The second incorrect assumption that most developers make is that the expiration
date is when the transient will expire. In fact, the inline documentation even
states that the parameter specifies the “time until expiration in seconds”.
This assumption is correct for the built-in data store: WordPress only
invalidates transients when attempting to read them (which has lead to
garbage collection problems in the past). However, this is not guaranteed
for other backends.
As I noted previously, transients use the object cache for non-default
implementations. The really important part to note here is that the object cache
is a cache, and absolutely not a data store. What this means is that the
expiration is a maximum age, not a minimum or set point.
One place this can happen easily is with Memcache set in Least Recently Used
(LRU) mode. In this mode, Memcache will automatically discard entries that
haven’t been accessed recently when it needs room for new entries. This means
less frequently accessed data (such as that used by cron data) can be discarded
before it expires.
What the transient API does guarantee is that the data will not exist past the
expiration time. If I set a transient to expire in 24 hours, and then attempt to
access it in 25 hours time, I know that it will have expired. On the other hand,
I could access it in 5 seconds in a different request and find that it has
already expired.
Real world issues are common with the misunderstanding of expiration times. For
WordPress 3.7, it was proposed to wipe all transients on upgrade for
performance reasons. Although this eventually was changed to just expired
transients, it revealed that many developers expect that data will exist until
the expiration. As a concrete example of this,
WooCommerce Subscriptions originally used transients for
payment-related locking. Eventually, Brent (the lead developer) found that these
locks were being silently dropped and users could in fact be double-billed in
some cases. This is not a theoretical issue, but a real-world instance of the
expiration age issue. The solution to this particular issue was to swap it out
for options, which are guaranteed to not be dropped.
When Should I Use Transients?
“This all sounds pretty doom and gloom, Ryan, but surely transients have a valid
use?”, you say. Correct, astute reader, they’re a powerful tool in the right
circumstances and a much simpler API than others.
Transients are perfect for caching any sort of data that should be persisted
across requests. By default, WordPress’ built-in object cache uses global state
in the request to cache any data, making it useless for caching persistent data.
Transients fill the gap here, by using the object cache if available and falling
back to database storage if you have a non-persistent cache.
One application of this persistence caching that fits perfectly is fragment
caching. Fragment caching applies full page caching techniques (like object
caching) to individual components of your page, such as a sidebar or a specific
post’s content. Mark Jaquith’s popular implemention previously
eschewed transients due to the lack of garbage collection combined with
key-based caching, however this is not
a concern with the upcoming WordPress 3.7.
Another useful application of transient storage is for caching long-running
tasks. Tasks like update checking involve remote server calls, which can be
costly both in terms of time and bandwidth, so caching these makes sense.
WordPress internally caches the result from update checking, ensuring that
excess calls to the internal update check procedures don’t cause excessive load
on the WordPress.org server. While the object caching API would work here, the
default implementation would never cache the result persistently.
Summary
Transients are awesome, but there are some important things to watch out for:
- Transients are a type of cache, not data storage
- Transients aren’t always stored in the database, nor as options
- Transients have a maximum age, not a guaranteed expiration
- Transients can disappear at any time, and you cannot predict when this will
occur
Now, go out and start caching your transient data!
- The reason I say invalid or no data here is
because it’s possible for a transient to be stored in the database before
enabling an object cache, so that would be read directly. [↩]
Thanks for this post :-). Was a useful primer. Do you have any resources to good articles on implementing transients?
Ryan,
Great article! Very rare any developer would look into internal storage mechanisms of WP Transient API; that includes me. I ran into a similar issue where caches were lost and rebuild randomly. You article helped me understand what’s really wrong with WP Transient and Xcache…
Great article! Even two years after it was written. I’m new to wordpress development and this was very helpful. One question though. If you use the transient API to store an expensive html fragment, so it is not built for every request. Would this fragment be available for retrieval for request made by different users?
Example:User 1 takes the hit and builds the fragment. It is stored using the transient API. Would this fragment be available for retrieval when user 2 hits the site for the first time (assuming it was stored and not expired)?