How to turn session recordings into data

First, define your goals

Before deciding which user behaviors to detect, you need to answer one question: Why are you looking at session recordings? If you are doing conversion rate optimization (CRO) you have probably defined a funnel and are looking at a specific step of that funnel. Your goal might be to guide the user to the next step as smoothly as possible. And so signs the user gets confused or distracted will interest you the most.

Or you may be watching session recordings to optimize content on a landing page. In this case you might want to find out which blocks of content confuse your users and which ones draw in their attention.

Define behaviors with Ted

Watch a few session recordings to get an idea what real users do on your page. Pick the behaviors you want to quantify and describe them.

Here are three criteria that improve the descriptions - they should be T.E.D.:

• Timed
  For actions that take time, include a length in their description. For example after how many seconds does “has watched video X” really count?
• Evident
  Describe specific actions a user takes and not your interpretation of these actions. There should be little room for debate.
• Distinct
  The more the behaviors overlap, the harder it gets to identify them correctly. If you have long overarching behaviors, try to split them into smaller actions.

Great and bad examples

User watches at least 30 seconds of product video

🔥 great

That’s a specific description with a specified length. Excellent!

User clicks on “tell me more”

🔥 great

Assuming there is only one “tell me more” link, that’s a great description. No length needs to be specified because the action of clicking practically takes no time.

User appears confused

👎 bad

It isn’t obvious what you're looking for. Different people will interpret user behavior differently. It’s up to you to decide which sign of confusion you want to detect. You could look for users scrolling back and forth and reading the same section again and again. Or you could look for link-clicks that are followed by immediately hitting the back-button. Or something else--the key is to define the specific actions.

User scrolls around without reading anything

🤔 bad-ish

While this isn’t a terrible description, you can improve it by describing concrete actions and by specifying a minimum timeframe. For example “user spends less than 3 seconds per section” would be better. Or even better by testing for the opposite with “user spends at least 3 seconds on at least 1 section.”

Run the survey

Once you’ve decided on the 3-5 behaviors you want to track, schedule regular replay analysis sittings. With the standardized behavior descriptions you can easily delegate and distribute this task to your team.

Should you use pen and paper or go digital to collect the data? We’ve found that in general it’s easiest to use a spreadsheet with the different behaviors as columns and each session recording in a new row.

I’ve created a sample spreadsheet containing a summary sheet with stats that are handy in a report.

If several people analyze sessions at the same time in a shared online spreadsheet, make sure to either create one sheet or one file per person. Multiple people filling in the same table gets confusing and messy quickly.

Sampling sessions and bias

It’s likely you can’t analyze all sessions because too many people visit your website every week. The solution is to take a random sample of sessions and only analyze these. Random sampling is important here: If you do the analysis every Thursday at 2pm and only work through the 100 most recent sessions, you’ll only evaluate sessions from Thursday morning and that’s how you introduce bias into your data.

It’s OK to filter sessions by some criteria like session length or number of clicks. But be aware that this also biases your data: If you exclude bounced visitors, you will not find out why they bounced or whether you’re improving that metric.