Photo via Unsplash

Five must-try content tests for UX writers

by James McGrath

As a writer, you’ve probably crafted cracking copy for your audience without really thinking about it. You know what works because you know what great writing is — and what it isn’t. It’s innate.

With UX writing, the value isn’t actually in your words, it’s in the value those words offer users and the business. Using content tests and data to prove that value should be your north star as a UX writer, as UX writing sits at the intersection of data and creativity.

Getting that data in a way that offers insight rather than noise can be a challenge. Here are five tests you can run to help you develop rich data and curate insights you can use to make your words better for everybody.

1. The A/B test

Simply put, A/B testing is the testing of a variable against a control to see which one comes out on top on a given metric. For a call to action (CTA) button, this would be clickthrough rate (or how many people are clicking the button).

You can show half your users one variable while you show the other half the control (your product team can help you set this up) to see what impact your changes may actually have.

Example of an A/B test log using Google Sheets

It’s super important that you keep a log of the tests you’re running, the tests you’ve already run, and the ones you want to run so you don’t end up doubling up. You don’t need to do anything too fancy for this — you can just put it in a shared spreadsheet.

Finally, make sure you assess the results for statistical significance to make sure that there is a measurable impact, or whether the numbers you’re seeing are a mirage. SurveyMonkey has a great tool for working this out.

2. Flesch-Kincaid test (and other readability tests)

Assigning your copy a Flesch-Kincaid score is one of the more tried-and-true methods of assessing readability by judging individual word syllables and sentence length. As a rule of thumb, you should aim for a score of 8, but this may dial up or down depending on your audience. You can take a deep-dive into Flesch-Kinkaid here.

There are other tools you can use, however. For example, while undertaking a Teaching English as a Second Language course, I bumped into VocabKitchen.

Tools like VocabKitchen can help identify potentially difficult terms for international audiences

It’s generally used to assess a text for suitability for various grade levels, but it will help you identify any difficult terms for non-native speakers of English. This is particularly useful if you have an international audience, but you’re not quite ready for full localization yet.

3. The Subject Matter Expert (SME) test

They’re on the front lines with your customers, talking to them about their worries, stresses, and tasks to do — all while trying to sell them products — which is great insight for any content design work you may need to do.

They’ll be up on industry-specific jargon, so they’ll be a great judge on whether the language you’re using is too complicated, or not specific enough. This is also where challenges to your copy will come up, and this is a great thing. If you have the data to back your copy choices, then you’ve done your job as a UX writer.

It’s also a great way to bring important stakeholders along for the journey as you develop content — but remember that you’re there to draw out insight that will help your users, rather than as a tick-box exercise.

4. The observed task completion test

Sit with your users (in-person or remotely) and ask them to complete key tasks such as ‘add X item to your shopping cart’. While they do so, ask them to talk out loud about what they are doing and how they are finding the experience so far while you record key quotes.

Example of a task completion recording log

If you see them looking between two buttons and not sure which one to press, ask them what they’re thinking about. If they reply that they’re a bit confused, ask them what’s causing the confusion. It’s in this rich qualitative data that you’ll start to be able to answer the why of design choices, rather than the what that quantitative analysis offers.

5. Tree testing and card sorting

TreeJack is a good platform for conducting tree tests, while you can do a deep dive on tree testing over at Norman Nielsen.

Card sorting is all about understanding the way users organize hierarchical information in a way that makes sense to them.

You give users a series of topics and labels and ask them to sort them into groups (these may be product groupings for an eCommerce site, for example). They can either label the groupings they’ve made (an open card sort) or sort them into predefined categories (a closed card sort).

Data is your duty

Working with data and being able to back up your work with numbers is a great way to increase job security, sure, but way more importantly, you’ll be doing a disservice to your audiences if you just wing it. Making task completion easier is at the heart of what good UX writers do, and if you’re not using data to inform the words you write, you’re only getting half the story.

James McGrath is a UX Writer, CX Specialist, and sometimes journalist who lives in Melbourne, Australia. Connect with him on LinkedIn.

Want to keep learning? Level up your content research and testing skills in our online, self-paced course.