26 February 2011
Session 1: Data Driven Decisions #pcampldn
Talk by Lucy from LoveFilm (@lucyjspence) Slides at http://slidesha.re/hFq1OB
Measure change through Experiments.
About half the room has done some form of A/B testing before. Different changes, pages etc etc. Next thing to try is multivariate testing (MVT).
Radical A/B testing. Taken to an extreme you can have different journeys (separate website, even?).
Statistical significance. Important to take note of since you need a certain volume of traffic before you can declare something as valid or not.
Relation to product lifecycle. Lifecycle affects the kinds of tests you can do. Early on, can only do simple tests since you don’t have volume to do valid MVT. As volume grows, then You can do more stuff and segment people. Eg separate ppl who come in via email campaign from other sources.
Advice:
- Respond to results
- Design for experimentation
- Design to segment/identify people by preference behaviour.
- Get the right tools for your org.
- Get ppl involved and pulling same direction
- Be prepared for things to go in the wrong direction from time to time. Don’t get dispirited.
Q&A:
Q Getting the right tools. Any recommendations?
A Google is a good start. website optimizer. Can let marketing ppl see what’s going on. It’s a monthly thing
Q. A/B testing is great for testing ideas you’ve already got but doesn’t help come up with new ideas. Does MVT etc help org come up with new ideas?
A. In some situations saying you can test things helps reduce risk. Depends on risk appetite of ppl too. It’s just a tool so it just helps provide more info on what’s happening.
[Ok, I need to work on my note-taking skillz. Missed a lot while trying to type.]
Sent via mobile
Comment on original post: Feb 26, 2011 - Cathy said... Great thank you for the notes - I was gutted that I missed her presentation... Sounds like it was a great one!
