Usability
International Workshop on the Interplay between User Experience Evaluation and System Development
We understand the relationship between UX and usability as the latter is subsumed by the former. Usability evaluation methods (UEMs) and metrics are relatively more mature. In contrast, UX evaluation methods (UXEMs) which draw largely on UEMs are still taking shape. It is conceivable that feeding outcomes of UX evaluation back to the software development cycle to instigate the required changes can even be more challenging than doing so for usability evaluation (UE). It leads to several key issues.
- UX attributes are (much) more fuzzy and malleable, what kinds of diagnostic information and improvement suggestion can be drawn from evaluation data. For instance, a game can be perceived by the same person as a great fun on one day and a terrible boredom the following day, depending on the player's prevailing mood. The waning of novelty effect (cf. learnability differs over time in case of usability) can account for the difference as well. How does the evaluation feedback enable designers/developers to fix this experiential problem (cf. usability problem) and how can they know that their fix works (i.e. downstream utility)?
- Emphasis is put on conducting UE in the early phases of a development lifecycle with the use of low fidelity prototypes, thereby enabling feedback to be incorporated before it becomes too late or costly to make changes. However, is this principle applicable to UX evaluation? Is it feasible to capture authentic experiential responses with a low-fidelity prototype? If yes, how can we draw insights from these responses?
- The persuasiveness of empirical feedback determines its worth. Earlier research indicates that the development team needs to be convinced about the urgency and necessity of fixing usability problems. Is UX evaluation feedback less persuasive than usability feedback? If yes, will the impact of UX evaluation be weaker than UE?
- The Software Engineering (SE) community has recognized the importance of usability. Efforts are focused on explaining the implications of usability for requirements gathering, software architecture design, and the selection of software components. Can such recognition and implications be taken for granted for UX, as UX evaluation methodologies and measures could be very different (e.g. artistic performance)?
- How to translate observational or inspectional data into prioritised usability problems or redesign proposals is thinly documented in the literature. Analysis approaches developed by researchers are applied to a limited extent by practitioners. Such divorce between research and practice could be bitterer in UX analysis approaches, which are essentially lacking.
While the gap between HCI and SE with regard to usability has somewhat been narrowed, it may be widened again due to the emergence of UX.
The main goal of I-UxSED 2012 is to bring together people from HCI and SE to identify challenges and plausible resolutions to optimize the impact of UX evaluation feedback on software development.
The Digging into Data Challenge has returned for a second round
What is the "challenge" ? The idea behind the Digging into Data Challenge is to address how "big data" changes the research landscape for the humanities and social sciences
During the first round, in 2009, nearly 90 international research teams competed in the challenge. Ultimately, eight remarkable projects were awarded grants.
Communities of Practice and virtual learning communities: benefits, barriers and success factors
Virtual communities of practice (CoPs) and virtual learning communities are becoming widespread within higher education institutions (HEIs) thanks to technological developments which enable increased communication, interactivity among participants and incorporation of collaborative pedagogical models, specifically through information communications technologies (ICTs) They afford the potential for the combination of synchronous and asynchronous communication, access to -and from- geographically isolated communities and international information sharing.
Clearly there are benefits to be derived from sharing and learning within and outwith HEIs. There is a sense of connectedness, of shared passion and a deepening of knowledge to be derived from ongoing interaction. Knowledge development can be continuous, cyclical and fluid. However, barriers exist in virtual CoPs and these are defined by the authors and illustrated with quotes from academic staff who have been involved in CoPs.
Critical success factors (CSFs) for a virtual CoP are discussed. These include usability of technology; trust in, and acceptance of, ICTs in communication; a sense of belonging among members; paying attention to cross-national and cross-cultural dimensions of the CoP; shared understandings; a common sense of purpose; use of netiquette and user-friendly language and longevity.
The authors recognise the enormous potential for the development of CoPs through e-mail discussion lists and discussion boards but have themselves experienced the difficulties inherent in initiating such a community. These are corroborated and illustrated with text from interviews with academic staff. Much of the literature on CoPs emanates from outside Europe, despite the fact that e-learning articles have a large diffusion around Europe. The authors suggest further exploration of this topic by identifying and studying CoPs and virtual learning communities across EU countries.
The Archive for Sexology – The World’s Largest E-learning Project
E-learning courses can be used by colleges, universities, governmental and non-governmental agencies as well as by professional organizations to reach distance students all over the world, especially in developing countries that otherwise lack the necessary resources.
This article describes an “open access” curriculum in sexual health that is freely accessible in several languages and has already found millions of users in over 100 countries: http://www2.hu-berlin.de/sexology/
Open access to scientific information is the way of the future. Following repeated recommendations of the World Health Organization (WHO 1975, 2001), The Archive for Sexology at Humboldt University, Berlin is offering a basic curriculum in sexual health. It is a global project consisting of six independent, but complementary courses that cover the most important practical issues:
- Basic Human Sexual Anatomy and Physiology
- Human Reproduction,
- Physical Problems in Females and Males,
- Sexually Transmitted Diseases and Their Prevention,
- Sexual Dysfunctions and Their Treatment,
- Human Sexual Behavior.
Adult Learners’ Information Seeking Behaviours Using the Web
Document based hypermedia environments such as the World Wide Web contain unparalleled amounts of information on any subject you care to choose. Success in operating within this environment requires having effective information search strategies. And many users seeking information from the Web find it a problematic exercise.
A great amount of software is written and many databases and websites are created without any consideration of what methods users will really apply in using them, or on the assumption that users will employ the same strategies as the system designers use or think they do. There is a wide gap between the intended and designed uses of ICT and the actual uses which are discovered or evolved by users themselves.
The Socrates Minerva Programme project SEEKS: Adult learners’ information seeking strategies in the Information Society, has carried out a trans-national census of the Information Seeking Behaviours (ISBs) of some those groups of ICT users which have been less well-studied in previous ISB studies. As a result we can offer Guidelines to educational software developers and ICT-based learning instructors to produce curricula and training content, which will assist in accelerating the integration of previously marginal or excluded groups into full use of ICTs and thus into the Information and Learning Society.
Results: a model for Information Seeking using the Web
This section presents the key results obtained after the empirical study that helped to develop the model for Information Seeking using the Web and a taxonomy of low-IT skilled users:
· Type of Website used to solve the tasks; we distinguished among three types of websites used by the target population: Commercial portal; Search engine (mostly Google); and Thematic website (many types depending on the subject). It was observed that beginners only used commercial portals, experienced beginners used portals and search engines, and that experienced users used the three kinds of websites. Thus, the type of website or the types used appears to be related to the experience on using the Internet.
· Time spent for complete the scenarios-tasks; the time used by the participants was not relevant in performing the scenarios, but it did have an effect on the quality of the information. Everyone spent more or less the same amount of time, but the experienced ones achieved the best quality results (in terms of completeness) in several scenarios.
· Number of alternatives; only in three out of the fifty scenarios performed, the participants (one different in each case) visited a second Web page of alternatives produced by the search engine. In no case did the participants check more than eight websites, and in twenty cases out of the total fifty they only checked one website.
· Internet Access-experience; having internet access at home was of prime importance. This result correlates with the stated level of experience in web use that participants reported themselves, and in the observed level of experience. Taking into account the tacit knowledge and the capacity to learn practicing it can be useful to take advantage of this result in order to provide learners with strategies adapted to their needs and stage.
· Preference of search engine; 76% of the users preferred one popular search engine. The main reason was related to usability understood as easy interface (very simple to use), speed of providing different alternatives, cited information in each alternative, automatic correction of errors, and own language websites searching.
· Alternatives chosen when using a search engine; the participants reported the following reasons for deciding on why a website is relevant: Reliability, speed, quality of information, quality of the design, confidence, clear and simple format and language, own idiom, direct access to solve the need, no personal data required, previously known website, or known from friends or media.
· Gender; SEEKS did not produce any difference in terms arising from gender.
Information Seeking Strategies
SEEKS identified a clear relationship between the seeking behaviours and the Internet experience of the learners. Depending on the parameters employed in the searching process, we could distinguish among three kinds of searchers among low-IT users: passive searcher, selective searcher, and dynamic searcher. Below we state the main parameters that characterize the different types of users identified:
1. Passive searcher. They are low Internet users. When searching, these individuals go to a common website (generally a commercial portal).When browsing, they mostly query using natural language, as well as their own speaking language; when they select the data, they just look for direct information. They trust on simple and clear interfaces. If they don’t find the information they take for granted that the information is not available on the Web.
2. Selective searcher. They are average Internet users. Usually they either go to a website they already know, but also they could use a search engine; when starting browsing they take into account the number of alternatives, whether they use natural language or not, if navigation is friendly, etc. When selecting content they look at direct and clear format of the information, as well as subject knowledge.
3. Dynamic searcher: Ample Experience in Internet use that provides the user with different kinds of searching approaches. The Internet knowledge allows them to access to different kind of websites depending of the nature of the need. The own subject knowledge helps to solve the need faster more satisfactorily., While searching the possibilities to solve the need are bigger than in the other two behaviours: a. Commercial portal, b. Search engine, or c. Thematic website (specific for a subject). When browsing, he/she takes into account all the parameters defined in the model: number of alternatives, time, natural language, own speaking language, speed and clear interface. While selecting the information all the parameters are taken into account, too (reliability, confidence, relevance, direct information, simple and clear format, and own subject knowledge). It is the most experienced user behaviour – regarding to the low-IT users studies –, and those who get the best results, in terms of personal satisfaction. Even in this case, the knowledge of Web use comes from the personal use, and they do not know how to select good information. The information selection is the result of their experience and personal decisions.
Implications for education and software creators
SEEKS led us to identify some determining aspects of the Internet users, and moreover of the “new” users (low IT-skill users who are starting to use the Internet) who have to adapt their existing skills to the Information Society requirements. The three types (passive, selective and dynamic searcher) are not really far from other taxonomies of searching processes in non-Web based environments. What is new is how concrete aspects of virtual environments can fit in with previously existing strategies facilitating the informational literacy of those who have not received any training in this field. This approach to the most common user profile of low-IT adult user allows us to introduce some Guidelines for training and for developing products http://www.seeks-it.net/outputs.html taking into account the stage which the participant has reached at any particular moment and the natural evolution which arises from further practice. In that sense, it is interesting to observe the relationship pertaining between use of the Internet and the strategies of searching exemplified in each behaviour or in each type of user. Users learn from their own experience and develop personal strategies more complex, but not structured in a procedural way.
It is known that adult users often achieved superior information seeking results despite inferior technical skills because their assessment of the value of information was superior. One of the aims of the SEEKS project was top discover how to ensure that users transferred their prior knowledge of the value of different kinds of knowledge into Internet use. The fund of tacit knowledge which older users bring in to Internet use should not be made redundant by the structures of the Web itself. Therefore both developers and teachers should accommodate the use of methods which allow the maximum transfer of prior knowledge. In any case, there is a clear need for feeding any kind of training programmes with information seeking strategies that go beyond the technical use of the searching instruments.
How to Remove the Major Obstacle of Learning: poor usability
· relevance and form of the content;
· navigational aids;
· activation and motivation of the students;
· guidance for studying.
Several recent publications of e-learning report usability problems in the current VLEs. The problems include, for example, unnecessary features, which are either rarely used or not used at all (Beasley&Smyth 2003), lack of navigational freedom affecting the user’s feeling of control (Armitage et al. 2003) and difficulties in identifying content (Lindh and Soames 2003). The environment may also be so difficult to use, that it causes deep dissatisfaction in the students (Kent 2003) or even withdrawals from the course (Jones et al 2003).
The VLE being highly technical or artistic does not matter, if the usability of the environment is poor. The lack of good usability makes students avoid using the environment or concentrate on irrelevant issues instead the content. One solution for achieving a VLE with good usability is to design the VLE using User Centred Design (UCD) approach.
Usability and user centred design
Usability is defined as effectiveness, efficiency and satisfaction (ISO 9241-11) which, in case of a learning environment, means:
· Are the users able to use the VLE for studying purposes?
· Is the learning with the VLE efficient, for example, is the user able to access the information and adapt the content?
· Is the interaction between the user and VLE smooth and comfortable?
Problems mostly occur, because learning environments are not planned and tested their target user group. In the beginning of the design process of a learning environment, the targeted user group and the environment of use, the context, should be defined. If a defined learning environment development method is used, this phase is usually included in the design process. However, the following phase is often somewhat, or even totally, neglected. During the design and implementation, the learning environment should be iteratively tested with prototypes within the targeted user group.
The motivation for usability studies during the design and implementation phases, in addition to better learnability, is money. Like in every other software engineering project, also in case of learning environments:
The later the change is made the more it costs in terms of labour, time and customer satisfaction. (Boehm 1987)
Case example
Virtual learning environment concerning safety at work, Virtu, was designed and implemented in co-operation between two institutes, Software Systems and Occupational Safety Engineering, in Tampere University of Technology using the UCD approach (Ihamäki and Vilpola 2003). In the beginning of the case Virtu, two user groups were defined; students at the university and workers in industry. The first paper prototype was tested within both groups and the most significant finding was that the same content was not appropriate for both user groups. As a result, two separate contents were made for students and industrial workers. This decision also affected the structure of Virtu, but the change was easy to make in requirements specification (Figure 1).
Development of Virtu had six usability actions:
· ISO 13407 Human centred design processes for interactive systems (ISO 1999)
· Interviews
· Paper prototype and its testing
· Functional prototype and its testing
· Heuristic evaluation
· Iterative design
According to the standard ISO 13407, context of use and user and organisational requirements were specified using stakeholder interviews. From the basis of requirement specification, a paper prototype was made and tested within end users. Changes ware made to the requirements specification according to the test results. Next prototype was implemented in the design and implementation phase and it was tested again within the end users. Comments received from the tests were adapted to the implementation. Testing and implementation were iterated until no big changes occurred. In addition to the end user testing, a usability expert evaluated Virtu. In this evaluation the expert used Jakob Nielsen´s heuristics (Nielsen 1994).
How to ensure usability of the VLE
Usability is a key issue what comes to teaching and studying using VLEs. There is no way you can add usability in a ready-made VLE (of course the current VLEs can be tested and improvements can be made). The usability aspect has to be taken into account right from the beginning of the VLE design process.
To ensure usability in virtual learning environments (VLEs):
1. Define the user group(s) of the VLE and the context of use.
2. Make a paper prototype containing few key displays and play the computer while end users are testing the prototype. Make notes of all the actions and comments the users make.
3. Apply the ideas and iterate the tests and the design until no big comments occur.
4. Implement the environment and test it again.
5. When all the critical problems have disappeared, it is time to release the version 1 of the environment.
6. Encourage users to give feedback on the version 1
7. Collect the feedback and use it for improving the design of version 2.
8. Long-term acceptability study (optional).
You will be happy to see a new kind of commitment by the designers, implementers, and students to your VLE. And even more, you can now honestly say that the VLE is especially designed for users; effective, efficient and satisfying to use.
A Recommendation
We recommend that you should visit the usabilitynet site, to see other ISO standards related to usabilty.
Figure 1. Project activities and usability actions in Virtu´s development.ReferencesArmitage, U., Wilson, S., Sharp, H. The Effects of Navigation Aids on Ownership for Learning with Electronic Texts, Proceedings of the 2nd European Confrence on eLearning, Ed. Roy Williams, Academic Conferences International, Reading (2003) pp.47-58.
Beasley, N., Smyth, K. Students´ Selective use of a Virtual learning Environment: Reflections and Recommendations, Proceedings of the 2nd European Confrence on eLearning, Ed. Roy Williams, Academic Conferences International, Reading (2003) pp.71-79.
Boehm, B.W. Improving Software Productivity. IEEE Computer (September 1987) pp. 43-57
Ihamäki, H., Vilpola, I. Designing an Adjustable Learning Environment Concerning Safety at Work, Proceedings of the 2nd European Confrence on eLearning, Ed. Roy Williams, Academic Conferences International, Reading (2003) pp.217-226.
International Organization for Standardization ISO (1999) ISO 13407 Human-centred design processes for interactive systems, International Organization for Standardization, Geneve.
Jones, P., Packham, G., Miller, C., Davies, I., Jones, A. “e-Retention: An Initial Evaluation of Student Withdrawals within a Virtual Learning Environment”, Proceedings of the 2nd European Confrence on eLearning, Ed. Roy Williams, Academic Conferences International, Reading (2003) pp.239-248.
Kent, T. Supporting staff using WebCT at the University of Birmingham in the UK, Electronic Journal of e-Learning, Volume 1 Issue 1, (February 2003) pp. 1-10.
Lindh, J., Soames, C-A. Are Students´ and Teachers´ Views of Online Courses in Accordance? A Dual Perspective on a University Course, Proceedings of the 2nd European Confrence on eLearning, Ed. Roy Williams, Academic Conferences International, Reading (2003) pp.275-282.
Nielsen, J. Ten usability heuristics, 1994 [online, referenced January 23, 2004].