I recorded a podcast with Andy Edmonds of Scrutinizer last week, and he even got transcript which I posted below.
With this transcript this will definitely be my longest blog post ever.
[Update: Andy's last name was misspelled in the transcripts, it should be Edmonds not Edmunds. It's correct now.]
Andre Charland: My name is Andre Charland. Welcome to our podcast here. I’m with Andy Edmonds from StomperNet. I work over at Nitobi. We’ve been working with Andy on a new AIR application called “Scrutinizer,†which is a tool that will web developers and we designers improve the usability to their website to raise conversion rates, and just basically come up with a design that’s more user friendly and create a better user experience.
Andy, do you want to give us a little background on yourself and then a little bit of an overview of Scrutinizer?
Andy Edmonds: Sure. I’m a cognitive psychologist by training, but back in ’95 when I left Georgia Tech, a lot of people didn’t want to hire a cognitive psychologist to do internet work, and that’s what I wanted to do.
So I earned my chops up through 2000 when I was an advanced certified instructor for Macromedia in ColdFusion, and taught some Flash and so on.
To make a long story short, I now work StomperNet, which is a leading eBusiness, eCommerce training program. And we developed this tool to really help people through who are not designers get a feel for how design impacts the user experience on it. So it comes from some really deep cognitive psychology background that I think is pretty darn cool. I blogged about this over on surfmind.com.
Back in 2003 at CHI, which is the leading ACM computer-human interaction conference, somebody actually proved that given good white space, headings and bulleted lists underneath the white space, people could jump around the page just really efficiently. They could jump from one heading to the next with their eyes, with no loss of efficiency. Where without the headings, they were meandering around the screen.
And so I had this idea then. Hacked it up into a Flash movie. Actually, my first version was with NodeBox, a Mac OS X graphics hacking library. And it was great. The NodeBox community actually gave me a little help figuring out the code.
So then I built it in Flash where I used static bitmaps. And then we came along and got Nitobi in the game to help us convert it to AIR. But what it does is it simulates vision, so it simulates the fact that you have high-resolution vision for a very small part of your visual field, two degrees. And that’s the only place where you see really high-res and color and in full detail.
You have another visual system, your peripheral system, which has a different type of receptor aimed at black and white primarily, and contrast in motion. And so in our click-through video, we talk about the WHAT system, the foveal vision, being attune to help you figure out whether you can eat the berry that’s on the ground. And the other visual system being aimed at keeping you from getting eaten by a lion while you’re reaching for the berry.
So when you apply this to a website, what you see is a small area of full resolution, and then the rest of the screen is desaturated with color, which is something Flash does really well, and blurred. And so this simulates the way people are really seeing – now admittedly, this is five times a second. So every five times a second, they’re bouncing their eye around and grabbing another portion of your screen.
Andre Charland: But when you’re looking at a website in Scrutinizer, basically, it’s kind of like you’re looking through a browser, but you see kind of most of the web pages are blurred except for this small region of focus, right?
Andy Edmonds: Right. And that’s the foveal view. So there are a couple of things that are neat about this. One is that you can understand how the features in your design are available to people in their peripheral vision. So how well is your site laid out? How strong is the visual grid? What kind of contrast is there? Are the navigation elements, say, the individual items in your menu well delineated enough for people to be able to detect them in their peripheral vision?
And remember, we already talked about if you can see it in you peripheral vision, you can move your eye directly to it. So these are called “major saccades†and they’re fast and large versus minor saccades, which are even faster, but very small. It’s sort of the difference between cassette tape and random access, a CD. You could jump to any place on the screen if it’s detectable in the peripheral vision. If it’s not, then you essentially have to do the cassette tape walk, or you have to pick a random destination on tape and jump there.
So this is neat because it helps you understand what features jump at people and what people are gonna be able to learn, right. So every site has a template these days. And as you move through the site, you want to be able to – you want your users to learn that template and to be able to benefit from that learning. And if your visual design doesn’t support that, IE people can’t recognize things they’ve interacted with in their peripheral vision, then no luck.
Andre Charland: Yeah. One of the things I notice when I use it is like a lot of sites are really low contrast and it’s hard to kind of make out other, I guess, blocks or chunks of content or UI elements. So whether header bars are all blurred together, they look like one line, they don’t look like individual menu items, and those sorts of things, I found, is really handy for changing design in that way.
Andy Edmonds: And there are some tools out there for colorblindness that I think we might be able to do in Flash with some of the bitmap transforms. So that would be a nice addition to the tool. But really, right now, it’s meant to be a design insight generator. A good designer really understands this stuff, right, they understand the layout principles and white space.
The best non-designer story here comes from the non-designer’s handbook. It’s called CRAP. So CRAP stands for consistency, repetition, alignment, and proximity. And these are some basic things that you can use to think through a design, and they’re kinds of violations that Scrutinizer reveals.
Andre Charland: Right. So if I was gonna download Scrutinizer and open my website in it, what would you recommend some of the first things be that I try to do with it or look for on my site?
Andy Edmonds: Sure. So certainly, you start off with you doing it. But one of the best things about Scrutinizer is that it helps you observe someone else doing it.
Andre Charland: Right.
Andy Edmonds: So in particular, everybody’s eyes are moving around five times a second, so it’s really hard to sit somebody down and understand what they’re thinking in Scrutinizer because they really have to be pretty explicit to get detail from the page. They have to move their mouse to the part of the screen that they want to read. You can really make that a lot easier to observe and get some mileage out of.
But for you, when you turn on Scrutinizer, auto-zoom will be turned on. So as you move your mouse around, the faster the move your mouse, the bigger your visual field will get, but the more blurry it will get. As you slow down and stop your mouse, you’ll zoom in on the current space. So one thing you might want to do is turn off auto-zoom and use the mouse wheel to adjust your level of focus, whether you’re looking at maximum detail or you’re trying to read larger headline text.
So the first thing to do is to draw your visual grid. And we’re looking at putting some support into the tool for this, but right now you can grab a screenshot and go into your favorite graphic editing tool, or just draw it mentally. And the question here is what kind of layout is apparent from the blur. Do you see content blocks? Do you see clearly delineated sections of the site? That grid is kind of the infrastructure of the road that your users perceive as they _____ around the page.
Andre Charland: Got you.
Andy Edmonds: The next thing to do is to say, “Okay.†Give yourself a particular challenge, right. So I want to find†– I’ve got a video that records me trying to find the Assassin’s Creed Xbox 360 game on Amazon. But that’s a nice exercise. Take some common task that your site is designed to support and try and accomplish it.
Andre Charland: Got you. But definitely, I think, one of the more effective ways to use Scrutinizer is to use it as a usability testing tool. So get some people to sit down in front of Scrutinizer, go to your website and get them to run through some standard usability tasks like completing a form on your website or finding a product or finding information they’re looking for.
Andy Edmonds: Right. And you should be able to understand what’s going through their head a lot more effectively then if you just did that with a normal web browser.
Andre Charland: Right. And that’s – as they move their mouse, we’re able to see kind of where they’re focused because the rest of the page is blurred out, so we’re not confused about where they’re reading ‘cause typically on a website they can have their mouse on the top right corner, but actually be looking in the bottom left or something like that. With Scrutinizer, we’re able to see what elements they’re looking for and where they’re eyes kind of drift to.
Andy Edmonds: Exactly. And we have an eye tracker in our usability lab and that’s fairly expensive. But one of the – the way it’s set up is there’s one screen where the user is actually doing their stuff, and there’s another screen which we tilt to the side that the observer can watch and it’s reproduced in the observation room where we see – where their eye is at any moment in time. So we’re actually able to interact with them, talk to them. They can see the entire screen with no blur. And we’ve got this really expensive bit of hardware of shooting near infrared light into their eyeball bouncing it back and inferring where they’re looking.
So that’s the high-end version of this solution, but Scrutinizer gives you some of those benefits without $35,000.00 investment.
Andre Charland: Got you. And you can download Scrutinizer for free right now, right?
Andy Edmonds: Right. So we’re looking at – we’re committed to a beta through the end of Q1, so you got all next month. Where we’re going with it is trying to add some more support for this usability testing process, so give you the ability to configure tasks and monitor user success and do some visualization, perhaps, like RobotReplay or some of the other analytic solutions –
Andre Charland: Got you.
Andy Edmonds: – what people did.
Andre Charland: Yeah. That makes a lot of sense. So would you use this alongside kind of eye tracking technology? Is it a replacement? In the overall kind of usability tools and testing standpoint, where do you – Scrutinizer fitting in?
Andy Edmonds: So we’re still figuring that out. We’re looking at – we’re bringing people into the lab. In fact, the lab is busy right now. And we’re having them use Scrutinizer along with the eye tracker, and we’re gonna try and validate how well that works. Certainly, it’s not gonna be as good as an eye tracker, but it’ll help. And particularly, get a organization that doesn’t have a lab or doesn’t have somebody who’s really well practiced at what they call “verbal protocols.†So verbal protocols is this thing where you get people to sit down and talk aloud as they’re doing the usability test.
Andre Charland: Yeah.
Andy Edmonds: So it’s really challenging to get people – to give them the right level of encouragement and handhold them into actually giving you the information you need. And so hopefully, this tool will help bootstrap that process for somebody who hasn’t done a lot of that.
Andre Charland: Got you.
Andy Edmonds: And I’m not gonna claim it’s gonna replace an eye tracker. Hopefully, it will give you some of the insights that you wouldn’t have gotten without it, and that you probably would get from an eye tracker.
Andre Charland: Yeah. That makes a lot of sense. I think also possibly kind of more accessible and easier and cheaper usability testing tools would probably drive more people toward something like eye tracking to really – deeper understanding. But today, there’s not a lot of tools out there that are cheap and easy to do for usability testing.
Even writing a usability test is quite time consuming and expensive when you think of you have to recruit probably at least five people. Get them in a room. Get them there. And then know how to run the test is not that easy to do really, especially on smaller software involvement projects or in smaller teams or internal projects and what not.
Andy Edmonds: Sure. And one of the things that Scrutinizer is useful for are sort of very basic usability and attentional issues. And one of things you really want to get at if you do a full-fledged usability task in a lot of the user research you might do is, “Is my product compelling? Am I giving them the – am I convincing my customer that they want to do business with me or whatever the goal is?â€
One of the nice things about Scrutinizer, or at least the things that makes it a little more lightweight, is that you’re aiming at some things that are more common to every human being, right. Every human being has eyeballs of the same general engineering spec with a variety of different deviations.
So a lot of the stuff that you test at this level is really common across all of your human users, so you can potentially use people in the office as long as they haven’t really overdosed on learning your site. You can get some value out of that from a interpretation, visual design, readability perspective.
Andre Charland: Right. Got you. So kind of a long hanging _____ of usability basically.
Andy Edmonds: And maybe not the thing that you really want if you’re trying to compare two different sales messages or other complex content. Maybe it’d be good if you’re trying to compare two different logos. And certainly, it’s ideal for something you got internally. So because it’s a browser, you can deploy it on your internet and you can access resources, which you might want to install a traditional analytics package on, right. You can access _____ yet.
Andre Charland: Yeah, cool. And so the application is just kind of a high-level here, is built an AIR, brings up the HTML page in the background basically, creates an image of that, and then puts a Flash movie over top, which does all the kind of fancy blurring and changes the color, like gets rid of the colors basically. Why did you guys think AIR was a good platform for building this?
Andy Edmonds: Well, I guess that I built this first in NodeBox, and I did that purely with static images of web pages, so saved screenshots. And then I imported that to Flash, which performed a little bit better for integration with mouse movement. But that required that I have a bitmap of every page I wanted to evaluate. And so over the last six, nine months, I’ve been producing bitmaps of web pages and uploading them to my server and then typing in this special URL.
So this AIR, because of the built-in WebKit browser, seemed like a great way to cut that whole process down. I’ve also – I have played a lot with Firefox and I would have loved to have been able to ship this as a Firefox extension, but I think the graphics layer is just not quite there.
Andre Charland: Right.
Andy Edmonds: And getting there with Firefox 3 – and SVG is shipping by default now. And they’re gonna need things with foreign objects. But I really couldn’t do the level of blur and masking and desaturation that I’m doing with Flash, or not as easily.
Andre Charland: Yeah. Got you. That’s all of the kind of catches like, “It might be possible, but isn’t worth the extra effort,†right?
Andy Edmonds: Well, I think in the long run, I certainly look forward to the – well, we’ll see where WebKit goes. But there is an issue for us, which is that because the Safari browser share is fairly low, some sites really – Safari doesn’t make the cut in terms of fidelity. And so that is a weakness of the tool that if you’re not aiming it at Safari as a core market, then some of layout may vary a little bit. And really, this is about evaluating your layout at a pixel-by-pixel level.
Andre Charland: Yeah. No. That is definitely an issue with WebKit as far as not having the market share. A lot of developers just don’t aim for it and people are really quick to criticize developers for that. But even a lot of Google applications that come out don’t work in Safari 3 right away, or there’s quirks. That’s something – hopefully stuff like AIR and Safari on Windows, and those sorts of things, will continue to drive adoption of that platform so it kind of gets more of the attention like a Firefox would compared to IE obviously. Everyone’s gonna support IE because that’s what they want.
Andy Edmonds: I think the iPhone is driving some significant adoption there.
Andre Charland: Yeah. I would see that, but kind of almost in a different way though too, right. Those are apps designed for a small screen and a low – not a very powerful computer, right. The iPhone kind of limits what you can do with JavaScript in certain ways, caps memory usage.
Andy Edmonds: At least folks will start to learn the caveats of Safari CSS.
Andre Charland: Yeah, that’s true.
Andy Edmonds: _____ hacking in AIR. So one of the things I’m working on right now is extracting your styles guide, essentially, from the site. So I can give you some insights into the range of typography you’re using and how that’s visible at different levels. I see way too many sites now that use just not enough range of typography.
The tech name is really great, right, ‘cause you can scan the headlines. And if you like a headline, you can focus in and scan the individual details. In Scrutinizer, particularly if you turn off auto-zoom and manipulate the focal depth with your mouse wheel, gives you a really powerful simulation of what that experience is like under the hood.
Andre Charland: Yeah, that’s pretty cool.
Andy Edmonds: We learned some lessons in building this. Lots of good stuff in there. We do screen captures, and we’re investing a bit more in that. You got to watch the event listeners on that HTML control though.
Andre Charland: Yeah, definitely.
Andy Edmonds: Five megabyte per page load memory leak.
Andre Charland: (Laughter) Yeah, that’s –
Andy Edmonds: I wish it was easy to spot.
Andre Charland: Yeah, exactly. It’s not something that’s gonna slip under the radar for too long, right.
Andy Edmonds: That was a fun one along the way. I think everything else was fairly smooth. I was really impressed with the AIR 1.0 runtime update. I just changed the runtime _____ and the batch code, and people are seamlessly updated. So it both updates the Scrutinizer AIR application file to the 1.0 version and it updates the AIR runtime.
Andre Charland: Yeah. I just went through that process this morning, actually, since AIR went 1.0 today. And it was really slick. I upgraded Scrutinizer also. Got all the new features. My runtime’s updated also. So definitely a very seamless process for the user. I think I updated another application _____. There’s only like a couple clicks and put in my password, and the way I went. So it’s pretty nice to have that feature. Like that functionality just kind of abstracted for the developer – really don’t have to worry about how the upgrade’s gonna work ‘cause it works pretty well.
We’ve have had a number of other customers working on AIR projects talk to us about they’re really concerned about how smooth the upgrade process is. I think it passes on most accounts.
Andy Edmonds: Adobe’s been shipping software for a while.
Andre Charland: (Laughter) Yeah. Yeah, especially runtime type apps that you need to use _____ the content or application. They really care about the details and how slick it is for the user.
Andy Edmonds: It’s pretty amazing. I realize I’ve been – I’ve actually been using Photoshop now for 20 years. This is gonna date me a little bit.
Andre Charland: Wow.
Andy Edmonds: My first year at Carnegie Mellon was 1988 and they had 20 _____ screens on Mac 2’s all over campus with Photoshop and Illustrator installed.
Andre Charland: Wow. I thought I was up there with 12 years, but you kind of – you beat me there by a couple.
Andy Edmonds: Well, while we’re on history, let me tell you one fun story about where this technology came from. I didn’t make up this foveal distortion piece. It actually comes from a really interesting time in computing history. So there was at time – imagine you’re building a giant flight simulator and you’ve got real expensive hardware. You’ve got twelve foot of screen so that your pilots in training can have a really realistic experience. And you’re trying to cost out all this stuff. Probably the No. 1 cost in that entire system is the hardware, the graphic hardware, would take to render that twelve-foot screen.
So what they did was they bought – they incorporated an eye tracker into these flight sims and usually known as “Gaze Contingent Display.†So it was cheaper to have an eye tracker detect where the flight trainee was looking and only rendered that part of the screen in high resolution than it was – so the eye tracker was cheaper than the video hardware.
These days, the eye tracker is probably a lot more than the video hardware to drive a twelve-foot screen.
Andre Charland: Yeah, definitely. Huh.
Andy Edmonds: But it’s an interesting bit of computing history.
Andre Charland: It’s neat how seemingly different of an application something like Scrutinizer is, but basically is doing the exact same thing. Also, interesting thing about how much cheaper it was to build something like Scrutinizer versus what that would have been, what the current – using Flash and AIR and these sorts of technologies. But it’s interesting. We’re using kind of old tricks to solve new problems, I suppose.
Andy Edmonds: Yeah. As the technology gets more available, new uses are found.
Andre Charland: Yeah, exactly. But it’s always good to look back and see what other things we’ve done in the past for some of these new things.
Andy Edmonds: Heck, a lot of usability in human factors came out of the need to get young kids safely to the battleground in World War II in airplanes.
Andre Charland: Yeah.
Andy Edmonds: Too many of our flight trainees weren’t making it to the battlefield.
Andre Charland: Yeah. I guess that’s early studies of usability and ergonomics came into place. Also, a lot of QA related things or for some other military type reasons originally.
Well, that’s great, Andy. I think that’s a good podcast. Do you have any other questions or comments or?
Andy Edmonds: Oh, I think that’s a wrap. Like I said, we’re aimed at adding some additional features for usability testing. Stay tuned. Grab the free demo now and have fun. Thirty days, we’ll see where we end up with the business model. The demo also does neat screen captures. So one of the best ways to use this is to grab a screen capture of the blur and send it out with some comments to your designer for a redesign idea. And then go play with those pages in Scrutinizer.
Andre Charland: Cool. All right.
Andy Edmonds: Congrats to Adobe on a great 1.0.
Andre Charland: Yeah, definitely. All right.
Andy Edmonds: Hey, thanks, Andre.
Andre Charland: Hey, no problem. I think that was good.
Andy Edmonds: And shout out to Nitobi for some good work helping us get this done.
Andre Charland: Yeah. Thank you very much. It was – it’s fun to work _____ we can add some more cool features in there and get it integrated with RobotReplay down the road too.
Andy Edmonds: That would be killer.
