Using segments is one of the most crucial tools in doing analysis. I’m sure many analysts started on the road to analysis with simple reports in Google Analytics and got themselves in trouble using segments in a faulty way. It even happens on occasion to the best Sr. Analysts. Let’s consider the experience of a very junior analyst named Hal Fwit.
Hal one day happily reported to his boss on how much traffic they got in the last 7 days. He very quickly realized that this information was not very helpful once he was asked by his boss, “So what?”
Hal walked out of the meeting asking himself “Isn’t it good enough just to know that you had a certain number of visits in a particular time frame? What else am I supposed to find in the data that is helpful?”
The reality is that any business question could use this type of information (x amount of visits), only once it is crossed by another data point. At the intersection of these data points Hal could have the beginnings of an insight. This is segmentation and our analyst friend is ready to experiment.
Hal segments the 7 day traffic by acquisition channel and quickly informs his boss that someone must have forgotten to send out that mass email marketing campaign as traffic coming from email didn’t spike on Wednesday like it should have. Now Hal can answer the ‘So what?’ question to a certain degree.
After realizing that crossing one data point (visits) by another data point (email traffic over 7 days) he derives insight into the customers and the proceedings of the business. Our young analyst Hal is intrigued and wants more. He thinks to himself, “What if the traffic we get from email, where we focus so much attention, doesn’t actually make much revenue and therefore has a low ROI?”
Wow Hal, big words and confusing acronyms in one sentence! So now Hal figures out which success event fires when a visit has completed a purchase on a particular page. So Hal stumbles his way into building a segment of email visits that completed a purchase on this specific page.
Our elated analyst friend Hal pulls this across his acquisition channel report and sees that (sure enough!) Email visits only convert at 0.25% and Display visits (of all places!) converts at 9.7%! Nearly tipsy with the endorphins being dumped into Hal’s brain because of his victorious discovery, he Slacks his boss without delay that he has a discovery paired with recommendations on how to divert all advertising funds into the Display campaigns.
Upon entering the meeting room his boss reserved for the emergency marketing budget reallocation meeting, Hal to his despair meets the Sr. Analyst of Marketing Awesomeness who pokes massive holes in his hasty analysis.
“Are you looking at conversions for just the non-signed in users on the guest checkout page?” “GGGGuest checkout page?” Hal stammers… “Yes, signed-in email users, which account for 95% of email traffic, convert on an entirely different page once signed in.
Did you account for that when you built your segment? How did you validate this segment?” Account for? Validate? “What did these words mean?” Hal wondered… If only Hal would have spent time to validate his segment before running wild with the flawed data it produced he wouldn’t have looked so foolish.
He could have taken and built a report that looked like this to make sure he didn’t miss out on conversion happening on a particular page.
A report like this which has 3 dimensions and one metric allows Hal to see everywhere the conversion event is happening to make sure he doesn’t exclude important pages by making such a narrow segment that only includes that one page.
A report like this can be created in a multitude of ways to make sure that what he either wanted to include or exclude would not accidentally make or break his segment. How many times have we tried to narrow down a segment by excluding certain variables when making mutually exclusive user segments but keep finding that the numbers don’t add up.
In these cases we should pull a report with those variables as the dimension and whatever metric we need (total events, instances, pageviews, sessions, users) to make sure what we exclude hasn’t crept into our report because our segment was built in a faulty manner.
Subscribe to our blog and never miss one of Hal Fwit’s adventures on his way to become a better analyst.