Welcome back to my blog!
I have performed usability evaluation on Tesla Model S Dashboard. The evaluation was done with the use of simulator created by Andrew Goodlad. You can try the interfaces in the below site. https://projects.invisionapp.com/share/DPCUX2ETA#/screens
But before jumping into the evaluation, lets get familiar usability, usability engineering and the concepts of usability engineering.
Usability is a narrow concern compared to the larger issue of system acceptability, which basically is the question of whether the system is good enough to satisfy all the needs and requirements of the users and other potential stakeholders, such as the users’ clients and managers
Usability Engineering is a field that is concerned generally with human-computer interaction and specifically with devising human-computer interfaces that have high usability or user friendliness.
It provides structured methods for achieving efficiency and elegance in interface design.
In usability engineering study, overall acceptability of a computer system can be defined as,
Overall acceptability for a computer system = Social acceptability + Practical acceptability
Here, Social Acceptability encapsulates the socially constructed factors that affect user experience and therefore the acceptance of latest interaction techniques. In practice, those are what make an interface more acceptable or unacceptable. The social acceptability of a technology concerns whether an actual encounter with the technology (or user thereof) affects the social comfort, status, reputation, moral convictions, then on of participants or close witnesses within the encounter.
For Example, touch screen makes it more easy than the old type of button based dashboard, Touch to lock/unlock & Parking Brake.
And we can further analyze the Practical Acceptability within various categories, including traditional categories like cost, support, reliability, compatibility with existing systems, etc., also the category of usefulness. Usefulness is that the issue of whether the system are can be utilized to achieve some desired goal. It can again be further divided into the 2 categories of utility and usability, where utility is that the question of whether the functionality of the system in principle can do what’s needed, and usability is that the question of how well users can use that functionality.
Foe Example, Cost-the luxurious cost of tesla is not suitable for everyone.
Support- the UI of the dashboard support many functionalities which are timely needed.
Reliability- Though there are many security features that increase the reliability of the system. There are cons too such as network failure can make it impossible for the user to access features like map.
Compatibility-The colors are balanced to be clearly visible in bright sun, but soft enough to be less distracting for the driver. And Software of the tesla dashboard is compatible with Bluetooth media.
…
There are five main goals of usability evaluation such as
- Learnability-The system should be easy to learn so that the user can rapidly start getting some work done with the system. Tesla UI achieves this goal by its great design that enables user to lean the system in a short period of time such as the large buttons, with plenty of surrounding space so it’s easy to tap and identify .
- Efficiency of use- The system should be efficient to use, so that once the user has learned the system, a high level of productivity is possible. It refers to the expert user’s steady-state level of performance at the time when the learning curve flattens out. A typical approach to measure efficiency of use is to determine on some definition of expertise , to obtain a stratified sample of users with expertise, and to measure the time it takes these users to perform some typical test tasks.
- Memorability- The system should be easy to remember, so that the casual user is able to return to the system after some period of not having used it, without having to learn everything all over again. But with Tesla UI, the usage of icons which are hard and numerous to remember can cause user to be confused when he/she returns to the system.
- Errors- The system should have a low error rate, so that users make few errors during the use of the system, and so that if they do make errors, they can easily recover from them. Further, catastrophic errors must not occur. In tesla feature such as Touch to lock/unlock avoids erroneous actions being performed by the user.
- Satisfaction- The system should be pleasant to use, so that users are subjectively satisfied when using it; they like it. The tesla UI has both satisfactory features and also dissatisfactory features in its UI as mentioned in the above situations.
…
Heuristic evaluation is a usability engineering method for finding the usability problems in a user interface design so that they can be attended to as part of an iterative design process. Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognized usability principles (the “heuristics”).
There are ten heuristics such as,
Simple and natural dialogue
User interfaces should be simplified as much as possible, the mapping between computer concepts and user concepts becomes as simple as possible and the users’ navigation through the interface is minimized.
Tesla UI contains good graphic design that achieve a simple and natural dialogue. It has also limited the design to a small number of consistently applied colors such as red, blue ,green, white and yellow.
Heuristic Violated: No
Speaks the user’s language
The system should speak the users’ language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
As shown on below screen, the users can come to know the state of their vehicle, if its locked/unlocked. They can also check the location, the connectivity, the time, interior temperature, etc. The labeling of the menus follows real-world conventions and the structure is set in a natural way.
Heuristic Violated: No
Minimize user memory load
Minimize the user’s memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
As already said with Tesla UI, the usage of icons which are hard and numerous to remember can cause user to be confused when he/she returns to the system.
Heuristic Violated: Yes
Consistency
Users shouldn’t need to get confused if different words, situations, or actions mean identical thing.
In tesla UI access to functionalities, it’s shown how the interaction structure always remain identical & it’s easy for the users to acknowledge where they’re and the way to return. The look maintains consistency standards for design and labeling of elements
Heuristic Violated: No
Feedback
The system should continuously inform the user about what it is doing and how it is interpreting the user’s input. Feedback should not wait until an error situation has occurred: The system should also provide positive feedback, and it should provide partial feedback as information becomes available.
The user is given real time feedback of the speed, battery power, distance, connection strength, whether the vehicle is parked, temperature, warning of high speed, etc.
Heuristic Violated: No
Clearly Marked Exits
Users often choose system functions by mistake and will need a clearly marked “emergency exit” to go away from the unwanted state without having to go through an extended dialogue. Support undo and redo.
The below screens are great examples to show that Tesla offers the users control and freedom in the system. In the first figure it indicates the CANCEL TRIP option where has control over the journey and he can change in the middle way. The second figure indicates the minimizing and maximizing of music screen that can be controlled as per user preference. And the third figure shows the cancel option with which user can go away from the unwanted state without having to go through an extended dialogue.
Heuristic Violated: No
Shortcuts
Even though it should be possible to work a interface with the knowledge of just some general rules, it should also possible for the experienced user to perform frequently used operations especially fast, using dialogue shortcuts.
Tesla UI also provides shortcut by just clicking the icons to visit the needed feature than following long steps to reach the desired feature.
Heuristic Violated: No
Good Error messages
Error situations are critical for usability for two reasons: First, by definition they represent situations where the user is in trouble and potentially will be unable to use the system to achieve the desired goal. Second, they present opportunities for helping the user understand the system better
Even though the error such as high speed are indicated in tesla UI, informative error message are not displayed hence making harder for the novice users to identify that there is an issue.
Heuristic Violated: Yes
Prevent errors
It is more important to prevent errors in the first place than notifying or correcting after an erroneous action took place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they decide to the action.
Actions like Set Speed Offset, Walk-Away Door Lock, Driver Door Unlock Mode prevents the occurrence of erroneous state.
Heuristic Violated: No
Help and documentation
Even though it’s better if the system is used without documentation, it’s going to be necessary to supply help and documentation. Any such information should be easy to look , focused on the user’s task, list concrete steps to be applied, and not be overlarge .
The Tesla app offers help and tutorials through simple and clear videos on the topics that would cause the foremost problems for users and also offers access to support. https://www.tesla.com/sites/default/files/model_s_owners_manual_north_america_en_us.pdf
Heuristic Violated: No
…
Before conducting any testing, one should clarify the need of the test as it can have significant impact on the kind of testing that has to be done. A major distinction is whether the test is intended as a formative or summative evaluation of the user interface.
Formative evaluation is done to help improving the interface as a part of iterative design process. The main goal of formative evaluation is to learn which detailed aspects of the interface are positive and negative, and how the design can be improved. A typical method to use for formative evaluation is a thinking-aloud test.
In contrast, summative evaluation targets assessing the overall quality of an interface, for example, for use in deciding between two alternatives or as a part of competitive analysis to learn how good the competition really is. A typical method to use for summative evaluation is a measurement test.
…
Often, usability testing is conducted so as to match the usability of two or more systems. If so, there are two basic ways of employing the test users: between-subject testing and within-subject testing.
Between-subject testing is in some sense the simplest and most valid since it involves using different test users for the various systems. Thus, each test user only participates during a single test session. the matter with between-subject designs is that the huge individual variation in user skills. Therefore, it may be necessary to run a really sizable amount of test users in each condition to gloss over random differences between test users in each of the groups.
Between-subject testing also risks a bias because of the assignment of test users to the varied groups. as an example, one might plan to test 20 users, require volunteers, and assign the primary 10 users to at least one system and therefore the next 10 to the opposite . although this approach could appear reasonable, it actually introduces a bias, since users who volunteer early are likely to vary from users who volunteer late. as an example, early volunteers could also be more conscientious in reading announcements, or they’ll be more curious about new technology, and thus more likely to be super-users.
In Within-subject testing every test user get to use all the systems that are being tested. This method automatically controls for individual variability since any user who is especially fast or talented will presumably be about equally superior in each test condition.
Within-subject testing does have the main disadvantage that the test users can’t be considered as novice users anymore once they approach the opposite systems after having learned the way to use the primary system. Often, some transfer of skill will occur between systems, and therefore the users are going to be better at using the second system than they were at using the primary . so as to manage for this effect, users are normally divided into groups, with one group using one system first and therefore the other group using the other system first. the problems discussed above regarding the assignment of users to groups also apply to the present aspect of within-subject testing.
…
Stages of a Test
A usability test typically has four stages such as,
1. Preparation
2. Introduction
3. The test itself
4. Debriefing
Preparation: The experimenter should confirm that the testing room is prepared for the experiment, that the computer system is within the start state that was laid out in the test plan, and that all test materials, instructions, and questionnaires are available. For instance , all files needed for the test tasks should be restored to their original content, and any files created during earlier tests should be moved to another computer or a minimum of another directory.
Introduction: The experimenter welcomes the test user and provides a short explanation of the aim of the test. The experimenter may explain the computer setup to users if it’s likely to be unfamiliar to them. The experimenter then proceeds with introducing the test procedure, especially for the inexperienced experimenters.
The test itself: The experimenter should normally refrain from interacting with the user and may in no way express any personal opinions or indicate whether the user is doing good or not. The experimenter may make uncommitted sounds to acknowledge comments from the user and to stay the user going, but again, care should be taken to not let the tone of such sounds indicate whether the user is on the correct track or has just made a ridiculous comment. Also, the experimenter should refrain from helping the test user, even though the user gets into quite severe difficulties.
Debriefing: After the test, the user is debriefed and is asked to fill in any subjective satisfaction questionnaires. As to avoid any bias from comments by the experimenter, questionnaires should be administered before the other discussion of the system. During debriefing, users are asked for any comments they could have about the system and for any suggestions they’ll have for improvement.
…
References:
- Usability Engineering by JAKOB NIELSEN
- Parkhurst, Eva & Conner, Lindsay & Ferraro, James & Navarro, Mar & Mouloua, Mustapha. (2019). Heuristic Evaluation of A Tesla Model 3 Interface. Proceedings of the Human Factors and Ergonomics Society Annual Meeting. 63. 1515–1519. 10.1177/1071181319631336.
- https://www.tesla.com/sites/default/files/model_s_owners_manual_north_america_en_us.pdf
- https://abduzeedo.com/node/84312
- https://www.nngroup.com/articles/tesla-big-touchscreen/
- https://en.wikipedia.org/wiki/Usability_engineering
…
To get to know about the generations of user interfaces, visit my previous blog,
