Josie: The fact is I like it when people can just naturally use the phone the way they normally would. I don't have a sled on, it's an open question, but right now, I don't have a sled, but I do have this lovely IPEVO camera that allows you to get a very, very clear picture of what's going on. The wonderful thing about this site, I have on my phone itself, the reason this office may very well have been available was because it had an extra special feature that I'm going to play for you right now. [bird noises] You hear that noise? That noise is the sound of predator birds. It's a recording playing outside, just outside that door right there. Because they want to discourage the unwanted creatures that might populate the area, pigeons, rodents, whatever. Understand, we're all biologically based, right? Three days of listening to that, I was pretty ready to take cover, too. Yes, our participants definitely did hear it. They were able to accomplish the tasks that they need pretty quickly. I don't think it proved to be a huge barrier. Three days' worth was a little bit of a challenge. Other challenges that we had. We had several. One of them, sort of the thing to keep in mind is that what can go wrong is going to go wrong. Let's be honest. You need to be ready for it. We had a family emergency that kept the interaction designer from being there in the war room. We did have a nice conference room set up, we had observer on the other machine over there. They were watching and taking notes and so forth, but the designer wasn't able to be present. We had a little last minute equipment juggling to do. As I was doing my run throughs, always, always, always, do, not one run through, but two. Make sure one of them is a technical run through so you have all of your equipment working. As I did the run through, the machines died. Ah, what? I had to quickly back down one version, sorry guys, on my Morae, and then, switch machines. That worked, thank goodness. Got through the test for that. We had all sorts of unusual things. Every night, the network seemed to slow down a little bit. Other little problems came up along the way. One of them was sort of interesting, and I don't know if it was the HTC that we were using to test with or if AxShare had -- AxShare in particular, which is an online sharing tool, right? It's just a Web-based tool to put your prototype up. Participants would put their finger into an input box, and the Android would go blank, quietly, for a minute. Then it would give you a phone, because it was confused, didn't know what to do, so it would give you the phone interface by default. That was an unusual one. We had to think, frequently, how to handle some of the situations that were coming up along the way. Some of the surprises that we ran into had more to do with the findings, and this is why testing and testing and testing is such a great thing. How many people recognize this button right here? Right? The hamburger button. Where'd you see it? Where'd you see it? Where was the last place you saw it? Go ahead. Audience Member: I read a blog post that described it as the hamburger that makes the basement slide out. [laughter] Josie: I love that description. You see it on Facebook. It is almost a de-facto standard. Just to make doubly, doubly sure that we were using the right elements, Maria and I went ahead and put one on paper and shopped it around the office as a hallway test -- "What thing would you press to find all your options in this ad?" Won, hands-down, no problem. This is where context of use really matters, because in that picture above that, you see something else -- this thing right here. That is the Android browser menu button in some versions of Android, and it sits above that button just like that. It says "menu." I think it says "menu" because their testing go so well either. [laughter] Josie: That word is like an eye magnet, and people, even when they successfully use the hamburger button, went back and tried to use the menu button, which got them in trouble every single time. That was a surprise. Because the team was very passionate, and I wasn't in there with them and Maria wasn't in there with them, they wanted to make some changes. They got on the phone with Maria and said, "Quick, make me some changes." One of the changes that we tried was a little layover help that explained the button. It was absolutely beautiful, as nice as any layover help I've seen yet. It pointed to the menu button, and it said, "Do this," and you had to click to get rid of it... People read it out loud and then failed to use the menu button. We have a different button now. Lessons learned. There were several lessons learned. One of them that I will always say is have a backup for the backup. This particular thing, however, is something that we learned on the fly, and it's part of the environment with mobile. Every operating system has a different hardware interface, OK? These are keyboards for Android. You right there, what button has the forward slash on the left there? Can you see it? Anybody else, can you see it? Go ahead, call it out. [audience members speak] Josie: OK. How about the middle one? [audience members speak] Josie: What about this one? [laughter] Josie: If you don't know that keyboard, now you're stuck in a test and you're trying to figure out how to operate through something that you might otherwise use quite well. We learned to facilitate, to accommodate, and to be generous with our help. The same thing -- by the way, it's not just Android. Every OS upgrade with iPhone is exactly the same issue, that things move around and people might need to learn it. It might not be their thing. We had to use our equipment in this particular case, because we didn't want any artifacts left behind, number one, and we didn't want them to risk anything, even though it's relatively low-risk. Nonetheless, we wanted to make sure that they were risk-free, so we gave them our equipment. We did recruit according to, "Are you an Android user? Are you an iPhone user? You get this one." We even had a Lumia person, a Windows phone. Unfortunately, she sits down. She's about 70-something, and she says, "I just got my Windows phone." Then she says, "I don't know how to use this keyboard." She's never used the phone! She wasn't really our client. She also felt odd about our financial-management app. The fact is we did get her a stylus. We got her through the study. She was able to complete the study, which says something, right? Nonetheless, we did not include her in the study in the long run. Some of the things that we might be doing in the future. I am going to consider a sled, but I'm still looking for the perfect sled. I want the sled that doesn't interfere with how people actually normally use their phone, doesn't force them into one orientation, doesn't get in their way when they're trying to look at things. This image comes from UXIM, and it is Cyd Harrell, a renowned researcher. She recommends having people sit in a nice, big, comfy seat -- I don't know why this didn't occur to me before -- use a boom stand for a microphone, put your camera on that, and then just let people use their phone like they normally do. That is an important element, an important part of our app, is that people are going to be using it the way they ought to, not in this formal setting. The closer we can get to that, the better. Like I say, we want to consider a local prototype, if we can, putting it on the phone. I'm going to pursue that. I am still looking for the Morae for mobile. I know it's going to happen someday -- not necessary Morae, but something like it, where you get the underlying data. The reason I love Morae, as much as anything else, is it helps me reduce that 10 days to do analysis. I get my measurements so much quicker, and I know where they've clicked and what they've done, and it's all right there for me and it's basically charted. I would love to have that, looking forward to that, and, like I say, looking for that sled. The last thing I might consider is some sort of controller in different cans, because the problem might've been this baby right here. The USB hub might not be up to the job anymore. It gets overheated, and that causes a problem. I learned that from a friend that I met at UXIM, who explained to me how not running through the hub might improve my results, so I'll be considering that as well. That's about it. Do you have any questions for me? Jared: We'd like to give you a round of applause, for starters. [applause] Jared: Who's been trying to do stuff with mobile or is wondering about getting started with this? Pass it. I'm going to figure out how to get back to you there. [laughs] There we go. We'll go this way... [applause] Male Audience Member: I like the idea of using the comfy chair for the mobile testing. Josie: I can't take credit it. I encourage you to follow Cyd Harrell. She's got great ideas. Male Audience Member: Have you tried it? Josie: I haven't tried it yet, but the fact is we have exactly the right setup in our offices, so I can't wait to. I do have a boom-stand mike available to me. I'm going to give it a try. Male Audience Member: One of the problems we've run into with doing mobile usability testing is getting them to hold the phone so we can see what they're doing. What's your experience with that, without being unnatural and being weird? Josie: Right, exactly. You're constantly dealing with that problem, which is why, do you have a sled? What is the advantage of a sled, you might wonder? The sled keeps the camera at exactly the right distance. Therefore, when they move it, you don't get that wiggliness. Let me show you how it looks, as a matter of fact. In a moment, this will fire up and you will see how holding it naturally creates a problem. I'm in a task logger. First off, get your image sharp. One second here. Ah, there we are. Hold it still. You can see, the problem movement can actually -- yes, thank you, thank you very much. Movement can actually create quite a bit of an issue for you. In better light, you can control quite a lot of that with this setup, but it was definitely an issue. It reduced the number of frames that I could actually use as video, because it seemed to jump around. We could easily see what was going on, so that wasn't much of an issue at all. Some folks, I think, also, because of the circumstances, were encouraged to put it down on the table. How do we handle it? The blue tape was tremendously helpful. Tape off an area you want your participants to hold the phone within. Some of them still go off and wander away. That's OK. You can bring them back again, just the same way as you facilitate their thinking-aloud process, but it is an issue that I think we'll be dealing with. Some of the software that I'm seeing will eliminate all of that altogether. They'll use a FaceTime-like experience. They'll record the screen. They'll get the underlying data. That's really, really, really what I'm looking for. I think it's on its way. It'll be soon. Other questions? Jared: Right here. Male Audience Member: More so a comment. I don't know if anybody subscribes to Steve Krug's Sensibility emails that he sends out, but there was a software that was sent out last year that he talked about the type of thing you're looking for, and it's out there. It only works if it can be done through the Web browser. It doesn't do apps, I think, is what it was. I don't do mobility testing, so I was like, "Oh, that's cool," and just read about it. I don't know if that's the same stuff, maybe, you've been seeing or if you've noticed anything like that. Josie: Did you say it was from Foraker? I've talked to those folks and I think that they're moving in that direction, Foraker Labs. Silverback does a great screen recording, so that's something right there, and then you can always do the face separately. Ultimately, I still want to port all of it back into Morae. I'm sorry, I like my Morae, and I want to do my analysis there because it's faster. I can play my recordings back faster. I can figure out where things started and stop, let it do the measurements, let it do the charting. I'm not quite there yet either. We recently had something -- Jennifer Aldridge, I believe, wrote an article about that, and I tweeted about it because her sentiments were mine exactly. I'm looking for UX nirvana on mobile testing. Male Audience Member: I just wondered if anyone had considered using Google Glass as a way of recording it. Josie: I'm sure. Male Audience Member: Because if you were to attach the camera to someone's head, it might actually allow for... Josie: Interesting... Male Audience Member: It's one of the only things I can think of that's a really interesting use for Google Glass. Josie: I agree. [laughter] Josie: I might have a friend there that I might be able to poke about that. She's in user experience at Google. Maybe I will. That's a great idea... [computer notification tone] Josie: I should know how to shut this down. Right, folks? Male Audience Member: Hi. These were pretty controlled environments, in your office, where you were testing this. I'm curious. Did you test anything, say, in different lighting conditions, say, outside, where you've got a strong glare from the screen, or they're trying to use it under different conditions? Josie: Very good point. Glare from the screen. iPhones are particularly glare-y. That is a regular problem. Even when you're staying in the same office, you have the course of a day where light changes in the room. In that picture, noticed all the blinds were shut? There was a reason why. I added light to the work area, because, again, the glare problems and some of the other issues. Every environment, that's a really good point. You need to go in and test your environment as well. Do a run-through back at the ranch, but also, make sure that the image that you're getting while you're on-site can be captured, because the glare will just kill your recording. Jared: Got a question over there. Female Audience Member: It's not a question. It's just more of a comment to the previous question. Josie: I used to work with Casey. I want you to know she and I worked together. Jared: Oh, so this is a ringer. Josie: Yeah, I guess so. [laughs] Well, she better be nice. [laughs] [laughter] Casey: Just some light hazing. Just kidding. We have an application called Coach's Eye. Alan Dennis is the designer for Coach's Eye, behind me. That is an application for sporting and athletes. It relies heavily on context of use, so we're trying to learn about it. We've had to do some testing activities where we're out in a soccer field, outside, with the glare of the sun. I would encourage everybody to try and understand what sacrifices you're making when you're trying to capture your evaluations, because if you're sacrificing context of use to get a high-quality recording, then maybe think, "Well, let's just do that and just really try and observe how people are using the product outside on a soccer field," and if that is good enough. Josie: Instead. I agree. Jared: OK. Josie: Maybe we need to talk about how to use Coach's Eye for research, Casey. [laughs] Casey: Surely out of UX curiosity, going back to the three keyboards that you were showing for the Android device, I've used all three of those. I've used the Android, the Swype, the SwiftKey, and I believe the last one was the Google keyboard. Would that be correct? Josie: It probably was, yes. Casey: OK. I think I ended up with SwiftKey at the end, but out of curiosity -- because, like you were showing, nobody could figure out where the end would be on the last, on what I assume was the Google keyboard. Now, because of my experience, I was tapping the N when I was using the Google keyboard. Obviously, the key isn't there. You actually have to go to the period. Out of UX curiosity, why would the Google keyboard be so hailed as such a great keyboard when all of the other ones have all these other functionalities to them? Josie: I just downloaded the SwiftKey, too, to replace the keyboard on this -- this is my S3 -- because it was so problematic, and I love SwiftKey. I can't answer that question. It is sort of a good thing to ponder, because Chrome is theirs, right? A number of things that they're doing will be using Google to accomplish and yet... Casey: What do you think in your opinion might be the reason why they would go with something so simple out of all the other keyboards? Josie: It is sort of a hallmark of Google, isn't it? Casey: Mm-hmm. Josie: I have to admit. Jared: Google tries to keep us guessing on a regular basis, is their general philosophy. Josie: I'm going to unplug it. It's OK for me to unplug it? Jared: Well, Josie, thank you very much. [applause]