Twitch wants streamers who spend hours just chatting and playing games on the platform to stay well-informed about the features it is testing.
To that end, the company is launching a new home page for the experimental features it tests, with the goal of making that experience more transparent and giving streamers a heads up about what’s on the way. The new experiments page will display a list of experiments running on Twitch on a monthly basis and these tests will disappear from the list once they complete their course or are integrated into the main experience.
“Sometimes I think we’ve been guilty of providing less clarity on what we’re doing because we’re doing a lot of things when we experiment and it can be a bit difficult to always keep the community in the loop,” said Twitch’s Head of Product. Officer Tom Verrilli told TechCrunch.
“But I think they’re rightfully expecting more from us, so what we’ve been trying to do with ‘here’s the full roadmap, and here’s the monthly page,’ is just put all of that out there and share with them what we are doing and why”.
Many social platforms test new features and other product tweaks with random swathes of their users. When platforms aren’t explicit about what’s being worked on, users often still spot new features and advertise them, but generally, these types of tests come and go in silence.
Occasionally, testing a social platform proves a major change or is particularly nasty and attention-grabbing, giving companies insight into what users are thinking in the process. The most infamous recent example of this is Instagram’s full screen feed test last year, which was reversed with a full apology from Adam Mosseri after Kim Kardashian rallied users against the new design.
Twitch has a long history of running tests like these with many becoming the main product, although some are not intended to. one month Twitch test for pinned chat, which allows channels to highlight ads and other key messages, became a feature-rich release in November, two months after the experiment took place. Another test last year explored a channel surfing function called the channel switcher that allows viewers to easily and quickly jump between streams without seeing a ton of pre-roll ads.
What might be less obvious to users, Verrilli explained, is that Twitch runs three different types of experiments, each with different goals. What Twitch calls “learn, not cast” experiments are designed to test a particular mechanic and see if users like how it works. Those experiments are never intended to make it into the final product, a fact that can be confusing to users who understandably assume that any testing feature could get a full version.
Another category of experiment is what Verrilli calls “genuine testing,” in which the company is confident that the problem needs to be solved, but doesn’t know exactly what the solution should look like. “So I would implement something for 20% of the people and see if this solves the problem, and if it doesn’t, develop a workaround for the same problem,” Verrilli said.
The third category is a more traditional phased rollout, where Twitch understands a problem and is confident of a fix, but still needs to move slowly so as not to disrupt streamer profits or their audiences with too many changes too quickly.
“So because all of those things look the same to streamers and end users, having a place where we can provide clarity when we’re just testing something versus when we’re implementing something tends to be really important,” Verrilli said. .
“Otherwise, people see an experiment somewhere, screenshot it, post it on Twitter, and then all of a sudden it becomes true, and as far as I know, nothing on Twitter ever is.”
Some companies (looking at you Meta) aren’t very transparent with this process and just run a bunch of tests behind the scenes, but Twitch is already quite forthcoming about what it’s playing, tweeting tests in progress. Now, that information will be collected in one place, a useful change for streamers who trust the platform and are interested in even small changes that could affect how their communities function.
In a new explanation about experiments, Twitch notes that it doesn’t always tell users that they’re part of the test group for a new feature, but when possible, they may receive an email or notification or see a small flask icon from laboratory on the platform. . Twitch also explains that occasionally active experiments won’t appear on the new page because letting the community know what’s being tested could confuse the results.
On the new experiments page, Twitch lists its February and March tests, which are very analytical and give streamers information on how to optimize to increase their views. Analytics-focused experiments include tag impression analytics (helpful insights into which tags help uncover), a new “research” tab that will help streamers determine when to schedule their streams, and a new analytics tool that shows streamers streamers how their channels are discovered from viewers who are already browsing Twitch.
Beyond analytics, Twitch is also running a five-week test for viewer milestone signals, which will recognize viewers who opt-in when they reach engagement milestones, such as watching a certain number of streams in a row. And on mobile, Twitch is testing a new chat mode that features transparent bubbles overlaid over the stream.
If a social media platform like Instagram can get away with upsetting its entire user base here and there, a creator-focused platform like Twitch needs to be more careful. Twitch is still the gold standard for long-form live streaming, but YouTube Gaming is waiting, even if it hasn’t broken through yet. And while almost everyone posts to a social app like Instagram, Twitch revolves much more around the big broadcasters that draw in viewers week after week. Keeping streamers happy and well informed is mission critical to the success of the platform, a fact that drives Twitch to stay in constant communication with the communities at its core.
