Reflecting on User Research Smoke and Mirrors

Objectivity is in the eye of the beholder

Having read the article by Christopher Farhey, a question arises in my mind - What exactly is objective methods? I believe that there is no such thing as true objectivity and everything, as objective as it seems, it's only in the eyes of the beholder.

Scientific methods are supposed to be used for a representation of our reality, but no matter how hard it tries to do so we need to realize that people functions in different realities and what seems objective to one may not seem to be so for another. Hence, the issue of misinterpretation of data.

Studies are all Subjective

Design methods need not depend on objective analysis. There is no such thing in the first place. I find it ironic that we talk about designs influencing people's emotions, attention and pleasure level, which are all subjective in nature, and link it to science to create a facade that such things can be measured objectively and that there are variables we can look at to make a design work.

Of course, certain amount of testing is still needed to make sure designers are not going the wrong direction. But whether there is a need to feint such methods as pretentiously as being objective should be examined.


Understanding Management View

Still, after all the critiques on the idea of objectivity, I guess we need to examine the mindset of the management - the ones who are taking the risk and sponsoring you with the money to complete your work. The fact is people do need confirmation and certainty especially when monetary issues are involved. This is pretty much linked to the idea of political tool mentioned by Farhey. The issue is management do need convincing and in a way, it eases the designing process as well.

I simply love the way the author puts it - It's like chicken soup. You don't need it But it's not bad either.

Phase 3 - Design and Looks

At this point of time... this may help:



The issue of first impression

It is said that first impression is formed in the first 50 milliseocond of laying your eyes on something. Hence, it seems relevant to place the first page of the three low-fidelity prototype to see its progression from totally not aesthetic to a (hopefully) more satisfying design:












Design 1 - Forgeting about design

The first design - or rather the lack of it - is due to our obsession with adding in the functionality to a project management website. As a result, the website has too many hierarchies, and even from the initial look of it, we know that it's not intuitive and pleasing.

However, we are just too concerned about the depth that we forgot about the main objective of having a pleasing design.

User testing and opinions from Mr. Reddy are being taken into account. It quickly makes us realize how far off we have become. In fact, we were wondering how come we were so far off when the latter two prototypes are what we have initially set out to do.

Fortunately for us, we still have time to make changes and simplify the overly complex site.


Design 2 - Less than five clicks initiative

For the second design, we decided to put an organizer calendar as the central theme for the whole platform with rotating icons at the bottom to serve as links to features. Hence, the depth is not as complicated as before, and users do not need many clicks to get to where they want.

This was again tested with users and this helped us in coming up with the third low-fi prototype.

Design 3 - Revamped for better functionality and aesthetics

In this design, the communications icon was separated into two - forum and announcements - for lower hierarchy of accessibility. The interface will be as graphical as possible to enhance user experience. An add task bar will also be included at the side. However, as commented by Mr. Reddy, we will need to seek a better link betweeen the task bar as well as the calendar. Also, how do we navigate users through the website such that they know that they are already in a project page.


Reflections - Back to a more comfortable position

Having got the design right again, we will do some more testing before proceeding onto the high-fi final product of this assignment. The process has been rather intense, and at times, stressful for us all. But hey.. we are going to get through this fine.

Phase 3 - Design

Previously...

With results from our previous study - survey and the kano analysis - we continue with other methods to get an idea of how users would like us to structure our website.


Card Sorting and Cluster Analysis

Card sorting is used by asking respondents to group features that they found similar together. Also, we also try to find out what short cut features would respondents want to see on the front page. However, apart from the categories obtained, test of trying to know what short cuts should be included on the first page proves less conclusive.

Information Architecture

With information generated with card sorting, we came to this tenuous process of getting the IA right. As a project management website, this is really where the main headache began...

At times, in fact, we got lost in trying to come up with a workable naviagtion system...



Reflection

Faces a major hurdle

I am sure Ed and Jay will agree with me that this stage proves to be the hardest in surging forward. We do not wish to rush things ahead with a messy hierarchy. However, at this point of time, we got a little wind up and obsessed with functionality rather than pleasure of usage.

This will be further elaborated when you see our first low-fi prototype.


Digressing a little

Sure, this is the point of time when most students got busy with their various assignments and deadlines. In a way, this also affects our ability to come together in coming up with better solutions. Nevertheless, we will be getting there :)

Phase 2 - Define

Defining our users




Coming into this project, we understand that people have different needs, and not everyone may belong to our target group. So just like a throng of people walking on the streets, how do we sieve out the users who have the need?

Fortunately for us, we have the users in mind right at the beginning of designing process, and has very earlier on know that meeting the needs of students - specifically NUS students - is what we want to do.


Personas...

We come up with personas of anti, novice and advanced users to understand them better. While we acknowledged that there are people who accept this technology fast, there are still people who will be as stubborn as mules in their adoption...





Users specifications and requirements

While some requirement of users can be obtained from the preliminary study, we want to futher narrow down the scope by conducting another survey based on the Kano Model.

We came up with an exhaustive list of items to be included in the system before posting the questions to 26 respondents.

Personally, I find this test very interesting as for every positive question posted, you have to post a counter-question in understanding how much people can live without it. Rating of the scale is also unconventional - instead of a symmetrical scale of very unsatisfied, unsatisified, neutral, satisfied and vey satisifed - the scale ranges from i like it this way, i expect it this way, i m neutral, i can tolerate it this way and i dislike it this way.

Hence, while a person can answer I like it if the system has photosharing, and states that I expect that the system usually do not have photosharing. This feature can be classified as an excitement attribute to the website.

Going along this line... we are able to obtain a lot of interesting information as to what will need to be inside the system and taking out what is not necessary.


REFLECTION

Interesting to conduct

This process proves enjoyable for the team as it seems to gear us a step closer to the design of the end product. Also, the final product becomes less abstract here with studies conducted in understanding users' specifications.

Results are subjective, but so what?

Guess the results cannot be objective in that sense that we have already generated a set of features. Also, testing on 26 respondents will not be what you call as "scientific" from a statistical sense. Still, we believe that the results are meaningful in steering us to a correct direction.

Of course, the article by Christopher Fahey helps too in addressing this issue of make-believe objectivity.

Phase 1 - Discover

Brainstorming for ideas...

The initial stage of brainstorming proves to be the hardest, especially when we are in a dire need of a tipping point.

Our team - Ed, Jay and I - took great pain in generating workable ideas. Most of which circulate around the idea that it should be practically implementable - like an online portal.

Here are the final two that we presented to Mr. Reddy in class.

Planning an itinerary for your free & easy travel?




This seems like a good idea initially. We come up with this idea because of the lack of travelling websites that deal specifically in generating out itineraries for young travellers. As I've friends who endear wikitravel as the travelling bible in planning their routes around a continent like Europe, we simply felt that the idea was implementable.

Of course, later on, Jay suggested limiting the scope to merely Singapore first for the ease of implementation.

However, when presenting this idea, the issue of need arises - is there really a need for such thing? Can target users identify with our concept? Anyway, we scrap this idea in the end because of all the hurdles we are facing.

In need of a virtual place to manage your projects?



With IVLE community revamped to something completely unusable for projects, and other online methods insufficient to meet our needs, we come to the conclusion that THIS is what we want to do for our final project.

Who are we?

We took the name of an online software developer ZOHO since they already have an existing range of softwares targeted at making online project management easier. We felt that in assuming this role, we are better positioned as someone who has the expertise to make such a software a success if it is in demand among the student population.


What we did?

We conducted a preliminary survey on 31 students and collected qualitative information from five respondents to get a general idea of whether there is a need for a new project management portal.

Questions are designed to test the current project management behavior, attitude and perception of NUS students. From then on, we make inferences on the needs of NUS students.


SO... really... is there a need?

From our results, we have proved that more than 90% of users do use online platforms for project management and feel the current methods inadequate in meeting their needs.

This is really the gist of our research as well as the data we need to further our efforts in understanding the users.


Reflection

Keeping our focus - Not the product first, need first, please.

Before the process of brainstorming, we have basically started off with a wrong approach - instead of thinking about the issue of need from a user's perspective, we are geared towards thinking as marketers who seek to market an already conceptualized product to the people without really understanding what is needed in a product by users. Hence, we discard our initial idea, and starts a new thought process of imaginingg ourselves as users, and as students about what we will want. This helps us alot.

Seriously, we have no idea what product will turn out... now

Going through this thought process makes me realize that unlike other courses that I have taken, I did not already have an envisioned end-product in mind. At times, it is like treading on unfamiliar ground, and could turn scary.

But as we got deeper into the issue of need, and realized what users really want, I got more interested in finding answers from users and coming up with ways to "outsmart" them by providing them with not only something that meet their needs, but exceed their needs.

Designing Probes - Ideas for Improving Lecture

Lecture observed
GEK 2020 Introduction to Film Art

Reasons for choosing this lecture
We are interested to see how the integration of visual and audio elements in the presentation of a lecture can affect the attention and interest of students.

Hence, we chose a lecture with the following technological elements:

• DVD player and projector
• powerpoint presentation
• mic and amplifier
• lights dimming system
• air-conditioner
• seating arrangement in an auditorium setting

Assumption

We came with the assumption that students do have higher expectation from modules with more visual and audio elements, and hence may be easily disappointed when the module did not meet their expectations. However, this does not mean that we believe the lecture will be a boring one prior our entry into the research ground.

Methodology

1. Ethnography

The three of us - Edmund, Jaysen and me - went to attend one of the lecture without notifying either the lecturer or the students about it. It is our wish to conduct an unobtrusive observation of the actual lesson scenario.

Seating Arrangement and Equipments

Edmund - Back row with a note pad and video camera

Jaysen - Front row with a note pad

Kate - Middle row with a note pad and a phone camera

The video camera and camera were used discretely to avoid alarming those who are attending the lessons. We took different seats to observe if seating arrangements affect attention, and also to have a wider coverage of the large class.

2. Interviews - Technology as Experience Method

Interviews are also conducted with the lecturer Dr. Gilbert Yeoh, and two of the students attending the class. We feel that it is important to interview both sides, to get a wider view of the perception of both sides in their interaction with the environment and the other party.

We use the technology as experience approach in which questions are designed to express the different dimensions of the methodology, namely anticipation, connecting, interpreting, reflecting and recounting.

Interviews can be downloaded below:

Interview with Dr. Gilbert Yeoh

Interview with Respondent 1

Interview with Respondent 2

Analysis

Ethnographic data are recorded under different categories that made up the lecture setting. This includes: time, place, environmental/situational factors, lecturer, recipients and technology.

Interviews data are used to create two user profile - the lectuer and the student - by using the 4-pleasure analysis and the 4 emotions.

Suggestions

Suggestions are provided using dimensions provided in the article Using Evaluate to Improve Students Learning which talks about difficulties in engaging students in lectures and large classes.

Limitations and concerns

1. Time factor

We were not able to attend another lecture to confirm our findings. Given more time, we would have attended more lectures, and interviewed more students to gain a wider perspective.

2. Ethical concerns

Through our ethnographical research, we did not inform anyone of the intrusion. We took photos and did recording discretely. However, we also know that had we attend the lecture after we informed the lecturer, the students and teacher may have behave less naturally.

Assignment 2: Four Pleasures Analysis

Imaginary Subject: James, a 24 years old Singaporean studying overseas


Being from a well-to-do family, James was send to London for his studies to become a barrister after his national service in Singapore. He is still a barrister-in-training and it is his hope to carry out his practice there after graduation. He is most probably going to get a job there with the connections that his father has with a major law firm. He has no intention of being back in Singapore in the short term.

He is the youngest child in the family, and has two older brothers who are doing rather well in their respective jobs as a doctor and lawyer. His father owns an insurance company. Because of his background, James always has the freedom to do whatever he want without restraints as everything at home is always being taken care of by his father and older siblings. On the other side of the coin, this make James feels rather dependent and passive at home as no one in the family seems to need him.

He loves his life as a student in London where he hang out with friends who are mostly other Singaporeans studying there like him. They take pleasure hitting on the English pubs and watching football matches there. They also frequent parties held by their schoolmates. James takes a lot of care dressing up for these occasions. He also has a humorous and charming personality; this makes him the centre of attention in the parties easily. He enjoys the attention and feels relied upon by his friends there.

James loves driving and takes pride in his ownership of the Lamborghini Gallardo, a present from his father on his twenty-first birthday. He takes pleasure discussing about the specifics and maneuverability of his car among his guy friends, and enjoys the admiration he gets from the ladies.
He also goes to the gym, albeit not that regularly. He is not a very disciplined person, but knows that he needs to exercise regularly to keep himself in shape. It also helps to keep his mind off things when he feels troubled. Besides, a little bit of muscles toning is always good at attracting attention of the ladies!

Four Pleasure Analysis of James

Physio-Pleasure

• Dress up for parties (appreciation pleasure)
• Goes to the gym to keep himself fit (needs)

Socio-Pleasure

• His relations with his friends are very important due to the lack of attention he is getting at home.
• He frequent social events to be with his friends and get attention from the ladies.
• His ownership of the Lamborghini also eases the process of making conversations with his friends.

Psycho-Pleasure

• James does not get his sense of accomplishment from excelling in his education, or to get a good job upon graduation as he knows that everything is more or less planned.
• He gains satisfaction by attracting people’s attention with his humor and charm. This makes him feel needed by his peers. (Need)
• Exercise to keep his mind off things – things that are usually related to his family.

Ideo-Pleasure

• James leads a sophisticated lifestyle, and loves to appear to be so as a reflection of his style and good taste.
• He is a non-conformist and likes to do things his way due to the ease of getting what he wants when he wants it.
• He enjoys the freedom he gets by being away from his family, and feels liberated by his independence overseas where he can make decisions for himself.
• In terms if his moral value system, he does not have a religion. He sees laughter and fun as important elements of a good life. That is why he enjoy reading works of humorists who make a joke out of all ordinary and banal stuff.
  
  

Mobile Phone – Products Benefits Specification

Appearance:

· Sleek, slim, lightweight and compact -- Need
Men generally like to place their mobile phones in their pants’ pockets.

· Responsive keypad – Need
Basic functionality is essential.

· Image enhancing – Appreciation
Can show an image of prestige:
For an instance, a wide screen with touch screen function plus key pad may give an image of higher class status and prestige due to the association of touch screens with PDA and wide screens with iPhones and many 3G phones.

Can shown the unconventional and fun side
This can be done by creating different innovative forms of the phone (i.e. slider phone plus touch screen?) or by experimenting with the colors.

Technical Aspects
· Ease-of-navigation and use interface – Need
This helps the user to attain a sense of accomplishment as he easily navigates through the content.

· Top-notch technology – Appreciation
Having top-notch technology in the mobile phone improves the image of the user.

- WIFI services will be a must due to its status as the new and coming technology.
- Organization skills of the mobile software will also be essential in keeping track of the user’s busy social life. - Good camera quality with xenon flash are also elements to show-off during social functions

Others
· Status item with high pricing and limited production
· Preferably, the mobile phone will be produced by a brand name associated with exclusivity