cs349.txt

# Windowing systems

* What are the criteria the X system tries to satisfy?

- Implementable on a variety of displays
- Applications are device independent
- Network Transparent
- Supports multiple, concurrent displays
- Supports many interfaces
- Supports output to overlapping windows
- Supports a hierarchy of resizable windows
- High-performance, high-quality text, 2-D graphics, imaging
- Extensible

* Where does the BWS lay?

Between the hardware/OS and all applications.

* What is the BWS?

The lowest level abstraction for a windowing system.

* What does functionality does the BWS have?

- Routes input to the correct window.
- Has routines for creating/destory/managing windows
- Ensures only one application changes frame buffer
- Creates canvas abstraction
- Coordinate system
- Basic drawing routines

* How do you filter out events that you don't want in X?

Masks. XSelectInput( xinfo.display, xinfo.window, Mask1 | Mask2 | Mask3 );

* How does the coordinate system work?

Each window has its own co-ordinates, where (0, 0) is top left.
BWS translates between window co-ordinates and screen co-ordinates.

* What is the window manager?

A service that provides interactive components for windows.

* Where does the window manager lay?

On top of the Base Window System.

* What is the different between the application window and canvas?

Canvas includes the "border" around main screen (e.g. window buttons)
Window manager owns the window, application owns the canvas.

* What is X w.r.t the BWS and Window Manager?

X seperates the BWS from the Window Manager.

* What does OSX and Windows do w.r.t to the window manager and BWS?

They are combined into one system.

* Why would the BWS and window manager be combined?

- Performance increase.
- Restricts modification.

* What are the 3 conceptual drawing models?

Pixel, Stroke, Region

* What are some functions for the Pixel drawing model?

SetPixel(x, y, colour);
DrawImage(x, y, w, h, img);

* What are some functions for the Stroke drawing model?

DrawLine();
DrawRect();

* What is the graphics context?

A "global" object that is responsible for holding the drawing options.

* Where is the graphics context stored in X?

On the X Server, and it can switch between multiple saved contexts.

* What is the painter's algorithm, how do you implement it?

Drawing back-to-front, layering the image.

Keep ordered list of displayables: back -> front
Each class has a paint() method.
On repaint:
  Clear window
  Repainting everything in that order

* What is the X Server?

The client that handles rendering output and getting user input.

* What is the X Client?

The server that handles all application logic.

* What are the two event related functions in X?

XNextEvent(Display* display, XEvent* event);
XPending(Display* display); -> returns an int

# Events

* What is an event?

A message to notify an application that something happened.

* What are some examples of events?

Keyboard presses, pointer events, window crossing, input focus, window events,
timer events, sensor updates.

* What does the hardware, BWS, and WM do regarding events?

Collect event information, put it in a known structure.
Order events by time
Decide which applications should get events.
Deliver the event

* What model does Java use to register and handle events?

Listener model

* What is animation?

Simulating movement by displaying a series of pictures, or frames.

* What is flickering?

A screen artifact caused by displaying an intermediate image.
Happens when screen refreshes in the middle of rendering.

* How does double buffering work?

1. Create offscreen buffer
2. Render to buffer
3. Copy buffer to screen as quickly as possible
  Hopefully between refreshes.

* What is GUI toolkit?

A set of widgets that one can use to build applications.

* What is a widget?

An element of interaction in a GUI that have its own behaviour.
A generic name for parts of an interface that have their own behaviour.

* What is Event-Driven programming?

A paradigm where the flow of the program is determined by events, such as user
actions.

* What are the 3 steps in the event dispatch stage?

1. Event dispatch: Getting events to correct widget
2. Event handling: Running code for an event
3. Notifying view and windowing system: e.g. rendering

* What does X use for event dispatch?

An event loop.

* What is an interactor tree?

A hierachy that the widgets are laid out on.

* What do events do with the interactor tree?

Events traverse the interactor tree to find the correct widget.

* What is a lightweight widget toolkit?

Toolkit that draws its own widgets. e.g. Java Swing

* What is a heavyweight widget toolkit?

Each widget is its own window for eventing purposes.
Toolkit is a wrapper for OS widgets.

* What is positional dispatch?

Send events to widget that is currently being moused over.

* What is the difference with bottom-up and top-down approaches for positional dispatch?

Bottom-up:
  Event is routed to leaf node widget.
  Each widget can choose to act, or pass to parent.
Top-down:
  Event is routed to highest-level node.
  Each widget can choose to act, or pass to child.
  Used to create policies for events (e.g. disabling all events)

* What is the difference between bubbling and capturing?

Bubbling starts at the leaf node and goes to the root.
Capturing starts at the root node and goes to the leaf.

* What does each of lightweight and heavyweight toolkits have: bubbling and capturing?

Lightweight: Capturing. Since each event goes to the root node by the BWS.
Heavyweight: Capturing and bubbling, depending on the BWS.

* What are some limitations of positional dispatch and how do you get around that?

- Send keystrokes to scrollbar if it's moused over?
- Mouse can go out of the widget's area
- Mouse press in one button, but release in another button

Need to consider which widget is "in focus". Use focus dispatch.

* What is Focus Dispatch?

Dispatch all events to focused widget, regardless of mouse cursor position.

* How many focuses are there in focus dispatch?

One mouse focus, one keyboard focus.

* What event would a focused widget not receive in focus dispatch?

An event corresponding to an action which switches focus.

* What is Acclerator Key Dispatch?

Sending keyboard events based on which keys are pressed. e.g. For menu items!

* What is difference in event dispatching between heavy- and lightweight toolkits?

Heavyweight:
  BWS has visiblity into all widgets in the application.
Lightweight:
  BWS only has visiblity into the application window.
  Toolkit dispatches to widget.

* What is an event that's usually dispatched positionally?

Mouse-down events.

* What is an event that's usually dispatched focusly?

Keyboard events.

* What is the inheritance event binding style?

Each OO widget inherits from a base widget class.
Implements onMousePress, onMouseMove, etc...

* What are some problems with inheritance event binding?

Doesn't scale well, need to implement more methods.
no seperation between GUI and application logic

* What is the interface event binding style?

Define a interface for event binding. (Each interface for a specific type of action).
Create classes that implements behaviour for the events.

* What is the listener adapter pattern?

Define listener interfaces. Define adapters which implement all interface
functions with NOOPs. We can now override the adapters to implement our
behaviour.

* What is the delegate binding style for events?

Define a delegate. Override the delegate with a method.
That method is now called when the event is received.
Can have more than one function be applied on a delegate too!

* What happens when events are generated too quickly?

Not all events are guaranteed to be sent.
There may be an array of "skipped" events for each event.

# 2D Graphics

* What is 'computer graphics'?

The creation, storage, and manipulation of images and their models.

* What is a 'model' w.r.t computer graphics?

A mathematical representation of an image containing 'important' properties like
size of color.

* What is 'rendering'?

Using the properties of the model to create an image on the screen

* What is an 'image' w.r.t computer graphics?

A rendered model.

* What data might a "shape model" contain for lightweight GUI components?

Shape, location, and bounds.
If visible: outline, and interior colour.

* What is an affine transformation?

Using linear algebra to draw elements on screen, relative to their parents.

* What is the difference between vectors and points in 3d-matrix notation?

Vectors have 0 has the last value. Points have 1 as the last value.

* What is the formula for translation? Matrix formula?

x' = x + tx
y' = y + ty

x + tx     1  0  tx | x
y + ty =   0  1  ty | y
1          0  0  1  | 1

* What is the formula for uniform scaling? Matrix formula?

x' = x * sx
y' = y * sy

x'   sx 0  0 | x
y' = 0  sy 0 | y
1    0  0  1 | 1

* What is the formula for rotation? Matrix formula?

x' = xcos(T) - ysin(T)
y' = xsin(T) + ycos(T)

x'   cos(T) -sin(T) 0 | x
y' = sin(T)  cos(T) 0 | y
1      0       0    1 | 1

* What is the order to you should apply affine transformation routines in?

Scale -> Rotate -> Translate; or
Scale -> Translate -> Rotate

* What is the problem with scaling or rotating an already translated object? How do you fix it?

Rotation and scaling about about the original, so the translation co-ordinates
may change now. To fix this, translate back to origin before doing other operations.

* What are the two ways of handling mouse inputs with affine transformations?

Transform mouse to model coordinates.
  Only one transformation.
  Maintaining the inverse.
Transform model to mouse coordinates.
  Many transformations.
  Manipulations must be transformed back.

* What are the two toolkits in Java for affine transformations?

Graphics2D, AffineTransform.

# MVC

* What is a common characteristic of an app with multiple views?

When one view changes, the other(s) often change as well.

* What are the three parts of MVC?

Model: Manages the data and its manipulation
View: Manages the presentation of data
Controller: Managers user interaction

* By who and when was MVC developed?

Trygve Reenskaug, in 1979.

* Does the model, view and controller know about each other in MVC?

Model only knows about the view interface.
View and Controller know about the model.

* Give a typical declaration of a view interface in MVC.

interface IView {
  public void updateView();
}

* What is the difference between MVC in theory and in practice?

Theory:
  View and Controller both refer to Model directly.
  Model uses the observer design to inform View of changes.
Practice:
  Model is loosely coupled with UI using the observer pattern.
  View and Controller are lightly coupled, for responsiveness.
  Controller is an inner class of the View.

* What design pattern is used in MVC?

Observer pattern.

* Give a typical instanitation of an MVC framework.

Model model = new Model();
Controller controller = new Controller(model);
View view = new View(model, controller);
model.setView(view);

* What is a logical input device? Give some examples.

A graphical component, defined by their function.
e.g. Locator, picker, choice, valuator, string, stroke.

* What is a widget w.r.t a logical input device?

A logical input device with appearance.

* What is the main goal of MVC?

Seperation of concerns. Allowing multiple ways to render the same data.

* What is the advantage of a seperate model?

Makes it easier for different UI components to use the same data.

* What is likely to change more, the model or the UI?

The UI.

* What programming model do widgets use?

Event-driven programming model.

* What are the three main design goals of widget toolkits?

- Complete: GUI designers have everything they need.
- Consistent: Same look and feel. Same usage paradigms.
- Customizable: Developer can reasonably extend functionality.

* Other than programming languages, what is another language used to make UIs?

XML.

# Layout

* What are two tasks that layouts handle?

Designing a spatial layout of widgets in a container.
Adjusting that spatial layout when container is resized.

* What do dynamic layouts want to do when a window is resized?

Maximize the use of available space for displaying widgets.
Maintiains consistency with spatial layout, and preserves visual quality of
spatial layout though.

* What must dynamic layouts do when a window is resized?

- re-allocate space of widgets
- adjust location and size of widgets
- perhaps even change visibility, look, and/or feel.

* What is a responsive layout?

A layout that adapts to different devices or screen sizes.

* What can change in a flexible widget?

x, y position, width and height.

* What can you do to constrain properties in a flexible widget?

Constrain position (e.g. by anchoring to another widget)
Give layout a range of sizes.
  minimum size < preferred size < maximum size

* What does a layout manager provide?

A layout algorithm to size and position child widgets.

* What are some attributes of layout managers?

Does it respect a widget's preferred/min/max size?
  Always ignored? Always respected? In some dimensions?
How does it handle extra space?
  Add extra space? Equally? Unequally?
Additional constraints?
  Alignment? Share of additional space?

* What two design patterns does layouting in Java use?

Composite and Strategy patterns.

* What is the composite design pattern?

Composite objects in tree-like structures. Treat the group of objects as one
uniform object.

* What is the strategy design pattern?

Factors out an algorithm into a seperate object.
Allows client to dynamically switch algorithms.

* What are some general layout strategies?

- Fixed layout
- Intrinsic size
- Variable intrinsic size
- Struts and springs
- Constraints

* What is the main property of a fixed layout?

Widgets have a fixed size and fixed position

* What is the main property of an intrinsic size layout?

Query each tiem for its preferred size.
Grow the widget to perfectly contain each item.
Top-level widget's size is completely dependent on its contained widgets.

* What are some LayoutManagers in Java that are intrinsic size layouts?

BoxLayout, FlowLayout

* What is the main property of a variable intrinsic size layout?

Same as intrinsic size layout, except that the size and location are influenced
by the parent component's algorithm now.
Ask each children how big they want to be.
Set each child widget's size and location.

* What are some LayoutManagers in Java that are variable intrinsic size layouts?

GridBagLayout, BorderLayout.

* What are struts and springs layout managers?

Specify layout by marking aspects of widgets that are fixed vs those that can
stretch.
Strut is a fixed space. Springs stretch to fill space.
"Widget must be east of another widget"

* What are some struts and springs layouts in Java?

SpringLayout, BoxLayout (restricted form)

* What are common development tools for struts and spring layouts?

GUI editors.

# Design

* What is usefulness and usability?

Usefulness
  Meeting specific needs and supporting real tasks
  Quality of being of practical use
Usability
  The effectiveness, efficiency, and satisfaction with which users can
  achieve tasks.

* Give an example of poor usability?

Push doors with handles.

* What is a mental model?

What the user believes about the system
  If I do ___, the system will do ___.
  The system is ___.

* What are the three mental models?

Developer's model
  How the programmer believes system should be used.
System model
  How the system behaves
User's model
  How the user of a system believes system should be used.

* What is Norman's model of interaction?

Two parts: Execution and evaluation.
Execution: What we want to do to the system?
Evaluation: Comparing what happened with our goal.

* What are the execution stages in Norman's model of interaction?

1. Form an intention to act to achieve goal
2. Plan a sequence of actions to fulfill that intention
3. Execute planned actions with physical movements.

* What are the evaluation stages in Norman's model of interaction?

1. Physically percieve the current state of the system
2. Interpret that perception according to experience
3. Evaluate the interpreted state compared to our goal

* What is the Gulf of Execution?

Difficulty in translating user's intentions into actions allowed by system.

* What is the Gulf of Evaluation?

Difficulty in interpreting the state of the system to determine whether goal has
been met.

* What are some basic principles to reduce Gulf of Execution and Evaluation?

- Perceived affordance
- Mapping
- Constraints
- Visibility/Feedback
- Metaphor

* What is perceived affordance?

What people think you can do with an object, based on perceived properties.

* What is an affordance?

Something you can do with an object.

* What is a mapping in the context of design principles?

The relationship between the control movements and the results in the world.

* What are the two 'degrees' in UI mappings?

Degree of integration:
  Ratio of DOF of device of on-screen actions
Degree of compatibility:
  Similarity in action and effect.

* How do constraints work in the context of design principles?

Guide the user by preventing certain actions while enabling/encouraging others.

* According to Norman, what are the four types of contraints?

Physical, Logical, Semantic, Cultural

* How does visiblity apply in the context of design principles?

Make relevant parts visible and convey the correct message.

* How does feedback apply in the context of design principles?

Communicate what action has actually been done; what result has been accomplished.
Every GUI user action should give feedback.

* What is an example of bad feedback for the user?

Symbolic links in Linux.

* What is an example of good feedback for the user?

Search and replace in Sublime Text.

* How do metaphors apply in the context of design principles?

Make a set of unifying concepts.
Borrow concepts from one domain and applying them to another.
e.g. "Desktop", "Shopping Cart"

* What are some common metaphors in computer design today?

Window, Recycle Bin/Trash, Folders, Files.

* What were some inconsistent metaphors in visual design?

Mac trash bin would eject disks from drives.

* What is important to consider when designing a metaphor?

Analyze relationships between features.
  Too many features => conceptual baggage
  Too few features => confusion, poor mapping

* What are some guidelines for designing metaphors?

Integration
  Are metaphors consistent?
Unpacking
  Can people determine why each component of the metaphor was included?
Topology
  Is there a simliarity in structure between source and target domain?
Analysis
  Can users use the metaphor to infer functionality?
Visual Presentation
  Can objects and actions be presented to guide user to metaphor's concepts.

* What do design guidelines provide?

Well defined rules to follow, higher-level principles.

* What do design guidelines not provide?

A process.

# Design Process

* What is User Centered Design?

Understanding the people who use your software.

* Who developed User Centered Design?

Apple for the Mac.

* What is the process for User Centered Design?

User studies -> Implementation -> Usability studies

* What are the main problems for User Centered Design?

Adds time to the development cycle.
No "cutoff" for changes.

* What are the 5 User Centered Design principles?

1. Understand users' needs: Build a product that meets real needs
2. Design the UI first: Then design the architecture to support the UI
3. Iterate
4. Use it yourself
5. Observer others using it

* How does one understand the users' needs?

1. Observer existing solutions
2. List scenarios
3. List functions required by scenarios
4. Prioritize functions
5. List functions by frequency and commonality

* What steps should you follow to design the UI according to User Centered Design?

1. Identify and design components
2. Design component distributions
3. Test the design with users
4. Document the design

* What steps should you follow to refine the design according to User Centered Design?

1. Refine requirements
2. Add new scenarios
3. Walk through new scenarios
4. Adjust user interface design

* What is a scenario?

Specific use-case, written in a way that the end-user can understand it.

* What are the 3 priorities of functions when designing them?

Critical: Needed by early users to do something useful
Important: Required before shipping the product
Nice to Have: Nice, but unnecessary

* What are the 2 dimensions of usage patterns?

Frequency <-> Occasional
By Many People <-> By Few People

* What is temporal distribution? What is spatial distribution?

Temporal
  When components appear, do they flow from one interface to another.
  How to get from one UI element to the next
Spatial
  Where components appear on an individual interface section.
  Where to go to get to the UI element.

* What are interaction sequences? Are they micro or macro structure?

Macro-structure, convey the "big picture" of system interaction.
Can be drawn like a flow chart.

* What are interface schematics? Are they micro or macro structure?

Micro-structure, convey essential content, and functionality at individual steps
of interaction.
Graphic design mockups.

* What is a prototype? What is the goal of prototyping?

A limited representation of a design that allows people to interact with and
explore its suitablity.

Goal: Maximum feedback for minimum effort.

* What are some examples of prototypes?

Paper sketches, slide show, physical models, mock software.

* What are some objectives of prototyping?

- Aid in discussions with stakeholders
- Help communication of ideas among team members
- User evaluation
- Technical/layout testing and tuning.

* What areas should be prototyped?

Difficult, controversial, and critical areas?

* What is the difference between a low and high fidelity prototype?

Low:
  Prototype doesn't look like final product. Easy to make. Simulated or slower.
  More iteratible. Encourages experimentation.
High:
  Prototype looks and acts like real product. More creative. Faster.
  More accurate. May give a false sense of completeness.

* What is paper prototyping?

Users perform realistic tasks by interacting with a paper version of the software.
Another person is "playing computer".

* What is the Wizard of Oz technique?

Evalute unimplemented technology by using a human to simulate the response.

* When should you use low fidelity prototypes? High fidelity?

Low fidelity prototypes should be used early on in the design phase.
High fidelity prototypes should be used late in the design phase.

* What is breadth and depth in the context of prototypes?

Depth: Functionality. Amount of interactivity.
Breadth: Features. Amount of features.

* What are the three dimensions of prototyping?

Functionality, features, fidelity.

# Visual perception

* What is psychophysics?

The relationship between external stimuli and internal sensations.

* What is temporal resolution?

The ability to see images that are changing over time.

* What is the Critical Flicker Frequency, or Flicker Fusion Threshold? Common value?

The frequency at which flickering light is percieved as continuous light.

Usually 45 Hz.

* What can the Critical Flicker Frequency be used for?

Mimicking continuous motion. e.g. 24 FPS film, zoetropes.

* What is spatial resolution?

The ability to see images that are close enough or with a certain size.

* What is visual acuity?

A measure of the spatial resolution of the visual processing system.

* What is the definition of 20/20 vision?

Seperate lines 1 arc minute (1/60 degrees) at 20 feet.

* Are what point are two lines indistinguishable on a monitor?

2 arc minutes (1/30 degrees)

* What determines colour?

The wavelength of light.
Infrared has lower frequency -> Ultraviolet has higher frequency.

* What are additive colour models? Name some models.

Coloured light is added to produce white.
RGB, HSV, YUV.

* What are subtractive colour models? Name some models.

Coloured light is absorbed to produce black.
CMY/CMYK

* What is the HSB colour model?

Hue
  Determines the colour
  Going around the circle in HSB
Saturation
  Determines how much hue (gray to colour)
  Vertical stripes in HSB
Brightness
  How much light (black to white)
  Horizontal stripes in HSB

* What sensors does the human eye have to percieve colour?

Cones and rods.

* What are the three types of cones in the human eye? What is their relative quantity?

Blue, green, and red (or yellow)
Fewer blue cones.

* What affects our ability to discriminate between colours?

Paleness, size, seperation.

* What are the two -chromacity's of colour blindness?

Monochromacity: 2 or 3 types of cones missing
Dichromacy: 1 type of cone missing.

* What are the 3 types of dichromacy?

Protanopia: Missing red cones.
Deuteranopia: Missing green cones.
Trianopia: Missing blue cones.

* What are the 3 functions of peripheral vision?

1. Guides fovea (provides cues to guide eye movement)
2. Detects motion
3. Helps us see better in the dark

* What does the weakness of peripheral vision imply in visual design?

People may not notice important information, like error messages.

* What are ways to make messages visible to the user?

Place it where users are looking
Place the error message near the error
Use an error symbol
Reserve the colour red for errors
"Heavy Artillery": Pop-ups, sound, motion.

* What is the common idea behind all display technology?

Each pixel is three sub-pixels: red, green, and blue.

* How many colours are available in 16, 24, and 32 bit RGB colours?

16: 5 red, 5 blue and 6 green bits => 65536 colours
24: 8 bits per colour => 16777216 colours
32: 8 bits per colour + 8 bit alpha channel => 16777216 colours + transparency.

* How do CRT monitors work?

Electrons are shot into a phosphor coating, which lights up when hit.

* How do Plamsa displays work?

Make a coloured gas glow.

* How do LCD displays work?

Shine a bright white backlight, use crystals to bend light, and use a colour
filter to generate RGB pixels.

* What are LED displays?

LCD displays with an LED backlight.

* What are OLED displays?

LED displays with no backlight, and ability to bend.
Expensive though.

* What are the various LayoutManagers in Java?

NestedLayout: Layouts in a layout
SpringLayout: Springs and Struts
BorderLayout: NESW and center
BoxLayout: Top to bottom, or left to right
GridLayout: Evenly space things out
FlowLayout: Put everything at preferred size, make a new row if not enough room.

* What does it mean when we say that "model is very loosely coupled with the UI"?

The model is just a generic object that any view can access to render its
information.

* What is in a typical event loop?

XEvent event;

long lastRepaint = now();
while (true) {
  if (XPending() > 0) {
    XNextEvent(xinfo.display, &event);
    HandleEvent(event);  // switch statement
  }

  doNextTick();
  repaint();
  sleep(1000 / 30 (now() - lastRepaint()));
  lastRepaint = now();
}


* How long should the event loop sleep for?

The time per frame - the time it took to compute everything before.

* Why do we use XPending instead of XNextEvent when checking for an event?

XNextEvent blocks until an event is present, so the whole rendering loop would
stop until an event is received. Also if use the XNextEvent in the while
condition, we would never actually get out of the event checking.

# Visual design

* What are some things that the user needs to know when seeing a UI?

- What can I do?
- Where is foo?
- What is expected of me?
- What is related to what?

* What does good visual design strive for?

Want to impose as little thinking as possible on the user.
Allow them to concentrate on their task, not the interface.

* What is the higher level goals of visual design?

Create desired relationships and avoid undesired relationships.

* What should a presentation try to be and not be?

+ Attractive look
+ Easy to understand
+ Has a distinctive look across many windows

- Not cluttered or hard to organize
- Not hard to perceive clearly
- Doesn't contain excessive idiosyncrasy
- Doesn't make the user stop and think to avoid errors

* How do you make a good visual design?

Keep the human brain's conscious and unconscious capabilities in mind.
  - Keep things simple
  - Leverage pre-attentive processes (i.e. subconscious responses)

* What is simplicity?

Presenting the minimum amount of information to achieve maximum effect.

* What does simplicity lead to?

Quickly recognized and understood functionality.
- Less information = less time to process
- Can more quickly produce correct mental models

Easier ability to recall information
- Less to remember

* What are some examples of simplifying things?

TV remotes, subway maps.

* How do you achieve simplicity?

Reduce, reduce, reduce! ... And reduce.

* What are Gestalt principles?

Theories of visual perception that describe how people tend to organize visual
elements into groups or unified wholes, when certain principles are applied.

Clues about how the brain groups raw visual impact.

* What are the 7 (8) Gestalt principles?

Proximity, similarity, good continuation, closure, figure/ground, law of
Pragnanz, uniform connectedness, (alignment)

* What is the Gestalt principle of proximity?

Individual elements are associated more strongly with nearby elements than with
those further away.

* What is the Gestalt principle of similarity?

Elements are more strongly associated with each other when they share basic
visual characteristics, such as: shape, size, color, texture, orientation.

* What is the Gestalt principle of good continuation?

Our visual system is biased to perceive continuous forms rather than
disconnected segments.
Elements arranged in a straight line or a smooth curve are perceived as being
more related than elements not on the line or curve.

* What do proximity, similarity, and good continuation describe?

How we group objects.

* What do closure, figure/ground, law of pragnanz, uniform connectedness, and alignment describe?

How we deal with ambiguity and fill in missing data.

* What is the Gestalt principle of closure?

We tend to see a complete figure even when parts of the information is missing.
We tend to perceive a set of individual elements as a single, recognizable
pattern, rather than multiple individual elements.

* What is the Gestalt principle of figure/ground?

Our mind separates the visual field into the figure and the ground.
- Figure is the visual element that is interpreted as being the object of interest.
- Ground is the area on which the figure rests (and everything else).

* What are the properties of the figure and ground w.r.t Gestalt principles?

The figure has a definitive shape, the ground is shapeless.
The ground continues behind the figure.
The figure seems closer with a clear location in space.
Elements below the horizon are more likely to be perceived as figures.

* What is the Gestalt principle of the Law of Pragnanz?

We tend to interpret ambiguous images as simple and complete, rather than
complex and incomplete.

Images recalled from memory are simplified.

* What does the Law of Pragnanz advise you to do?

- Minimize the number of elements.
- Symmetrical composition is perceived as simpler.

* What is the Gestalt principle of uniform connectedness?

Elements connected to one another by uniform visual properties are perceived to
be more related than elements that are not connected.

* What are two typical strategies for achieving uniform connectedness?

Connecting lines, and common regions.

* What is the "Gestalt principle" of alignment?

Keep text/content aligned.

* What are the 4 techniques for creating organization and structure?

Grouping, Hierarchy, Relationship, Balance.

* What is the grouping technique for creating organization and structure?

Group elements into higher order units. Use the Gestalt principles to create groups.
Proximity, Similarity, Alignment, Uniform connectedness.

Reserve powerful techniques such as colour and uniform connectedness for
explicitly telling the user something.

* What is the hierarchy technique for creating organization and structure?

A visual hierarchy guides and allows information scanning.
- Create the hierarchy to support intended reading sequence.
- Useful techniques: Size, position, spacing, white space, color
- Build on usual reading direction for the culture

e.g. Structure data with separators.

* What is the relationship technique for creating organization and structure?

Establish relationships between elements to communicate that they are connected.
- Alignment is very effective at creating relationships.
- Similarity (size, color, shape, etc) is also very effective.

* What is the balance technique for creating organization and structure?

Try to create a stable composition by balancing elements
- Stability can be achieved by manipulating properties such as position, size
- Symmetric layouts naturally achieve balance.

* What are the 4 principles to make pleasing layouts?

Contrast, Repetition, Alignment, Proximity.

* What is the proximity principle for making pleasing layouts?

Items relating to each other should be grouped close together.
When several items are in close proximity to each other, they become one visual
unit rather than several separate units. This helps organize information and
reduces clutter.

* What is the alignment principle for making pleasing layouts?

Nothing should be placed on the page arbitrarily.
Every element should have some visual connection with another element on the
page. This creates a clean, sophisticated, fresh look.

* What is the repetition principle for making pleasing layouts?

Repeat visual elements of the design throughout the piece.
You can repeat color, shape, texture, spatial relationships, line thicknesses,
sizes, etc. This helps develop the organization and strengthens the unity.

* What is the contrast principle for making pleasing layouts?

Avoid elements on the page that are mearly similar.
If the elements (type, color, size, etc) are not the same, then make them
very different. Contrast is often the most important visual attraction on the
page.

* What are some common mistakes in visual design?

Haphazard layout. Aligning labels, not controls. Bounding boxes creating clutter
and competing for attention.

* What is the squint test?

Squinting at your visual design to get a broad impression of it.

# Responsiveness

* What are the two ways to achieve responsiveness?

1. Designing the UI to meet human deadline requirements
2. Loading data efficiently so that it's available quickly

* What factors affect user's perception of responsiveness?

The system's ability to:
  1. Keep up with the user
  2. Keep them informed about its status
  3. Not make them wait unexpectedly

* What do researchers find to be the most important factor in determining user satisfaction?

Responsiveness

* What is the difference between responsiveness and performance?

Performance = computation per unit of time
Responsiveness = compliance with human deadline requirements

* How can low performance systems still be responsive?

- By providing feedback about what the user has done.
- By keeping users informed if they cannot fulfill the requests immediately.
- By providing feedback about what is happening (e.g. loading bar)
- By allowing the user to do other things while waiting

* What are some things that make applications appear unresponsive?

- Delayed feedback for button press, scrolling, etc.
- Ignoring user input completely. (e.g. wait-cursor)
- Providing no clues on how long the operation will take
- Jerky or hard to follow animations
- Ignoring user input while performing time-consuming operations

* What is the shortest gap of silence we can detect in a sound?

0.001s

* What is the shortest time a visual stimulus can be shown and still affect us?

0.005s

* What is the duration of saccade during which vision is suppressed?

0.1s

* What is the maximum interval between cause-effect events?

0.14s

* What is the time for a skilled reader's brain to comprehend a printed word?

0.15s

* What is the visual-motor reaction time to inspected events?

1s

* What is the duration of unbroken attention to a single task?

6-30s

* What are some human deadlines at 0.001 seconds?

- Minimum Detectable Silent Audio Gap

* What are some human deadlines at 0.01 seconds?

- Shortest noticeable pen-ink lag
- Preconscious perception: Can induce unconscious familiarity of images by
  making them disappear within 0.01s of appearing.

* What are some human deadlines at 0.1 seconds? What are the implications?

- Perceiving the number of 1-4 items
- Involuntary eye movement
- Audiovisual "lock" threshold
- Perception of cause and effect
- Perceptual-motor feedback
- Object identification
- Perceptual moment

If software waits longer than 0.1 second to show a response. The perception of
cause and effect is broken. The software's reaction will not seem to be a 
result of the user's action. Therefore you should display a busy indicator.

* What are some human deadlines at 1 second? What are the implications?

- Max conversation gaps
- Visual-motor reaction time for unexpected events
- Attentional blink

Necessary to display progress indicators for long operations.
Humans will take at least a second to react to an on-screen object.

* What are some best practices for progress indicators?

- Show work remaining, not work completed
- Show total progress, not progress on current step
- Show percentage of an operation that is complete
- Start at 1%, not 0%
- Display 100% only very briefly at the end of the operation
- Show smooth, linear progress, not erratic bursts of progress
- Use human-scale percision (e.g. "about 4 minutes" > "240 seconds")

* What are some UI tricks to improve responsiveness?

- Busy indicators
- Progress indicators
- Render/Display important information first
- Fake heavyweight computations during hand-eye coordination tasks
    e.g. Scrolling, dragging, resizing windows
- Working ahead, pre-compute responses to high probability requests.

* What are the goals w.r.t responsiveness for long tasks?

- Keep UI responsive
- Provide progress feedback
- Allow long task to be paused or cancelled.

* What should you do when a task will take significant time?

Run the task in another thread.

* What is the "run in the UI thread" strategy for handling a long task?

1. Task object keeps track of current task progress
2. Task object adds a short running subtask to the UI event queue
3. At the end of the subtask, check if there is more work. If so, add another
   short-running subtask to the end of the UI event queue
4. Handle any events that accumulated while the subtask was running.

* What are the advantages and disadvantages of the "run in the UI thread" strategy for handling a long task?

+ More natually handles "pausing" the task since it maintains information on
  progress of overall task.
+ Can be run in Swing event thread or separate thread.
+ Useful in single-threaded platforms.

- Tricky to predict length of time for subtasks
- Not all tasks can easily break down into subtasks (e.g. blocking I/O)

* What is the "separate thread" strategy for handling a long task?

1. Long method runs in a separate thread (e.g. Runnable object)
2. Method regularly checks if task should be cancelled and reports back to UI
about progress (by updating views)

Use the synchronized keyword to protect critical sections.

* What's the problem with Swing and separate threads?

Swing is not thread-safe, so you need to use invokeLater instead of calling
Swing methods directly.

Don't call Swing methods or access Swing components from outside the Event
Dispatch thread.

* Why aren't GUI toolkits thread-safe?

We have two sets of abstractions that flow in opposite directions:
  1. User-initated threads that travel "down" to the hardware.
     (e.g. start a thread to calculate primes)
  2. Events that travel from hardware to higher-level abstractions.
     (e.g. button clicks to cancel finding primes)

Any locking protocol for these two kinds of abstractions will conflict.

* What are the advantages and disadvantages of the "separate thread" strategy for handling a long task?

+ Conceptually, the easiest to implement.
+ Takes advantage of multi-core architectures.

- Extra code required to be able to pause/restart method
- All the usual threading problems (e.g. race conditions and deadlocks)

* What are the 3 types of threads in Swing?

1. Initial Thread
2. Event Dispatch Thread (i.e. UI Thread)
3. Worker Threads (i.e. Background Threads)

* How do you schedule code to run on the UI thread in Swing?

Use invokeAndWait() or invokeLater().

* What is the standard way to create worker threads in Java Swing and Android?

Java Swing: SwingWorker
Android: AsyncTask

* What is the complication for MVC when there are long tasks?

Long tasks start to break clean separation of MVC:
  - Model's methods need to be designed to allow user to stop the tasks.
  - May need methods to inquire about length of task completion.
    - Not part of "model", part of interaction.

Usability concerns are directly influencing design of model to accommodate user
interaction.

* What is the "classic", static web architecture?

- Thin client, fat server.
- Requests to the server are synchronous HTTP requests. (GET and POST)

* What are the disadvantages of the "classic", static web applications?

- Poor distribution of processing
- High user response latency
- Difficulty programming user interfaces
- Increased security risk (from mixing UI and application code)
- Heavy state management on servers
- Offline difficulties
- Reduced opportunities for interoperability

* What is the dynamic web architecture?

- Use AJAX to issue calls to web servers.
- Server handles requests and returns a data feed asynchronously.
- Client receives the feed and updates the UI via Javascript.

* What are the advantages of the dynamic web architecture?

- Minimized bandwidth
- Requests are faster (since they are asynchronous)
- Separation of concerns:
  - UI on browser, data on server

# Undo

* What are the benefits of undo?

- Enables exploratory learning
- Lets you recover from errors
- Lets you evaluate modifications

* What are some design choices w.r.t undo?

- What can't/isn't undone?
- What part of UI is restored after undo?
- How much should be undone at a time?
- Is undo/redo global in scope, local, or someplace in between?

* What actions should be undoable?

- All changes to the document (i.e. the model).
- Changes to the view, if they are extremely tedious or require a lot of effort

* What should you do if you have an undoable action?

Ask for confirmation.

* What should happen with the UI after an undo?

UI state should be meaningful after undo/redo.
- Change selection to object(s) changed as a result of undo/redo
- Scroll to show selection, if necessary
- Give focus to the control that is hosting the changed state

* What defines one undoable "chunk"?

The conceptual change from one document state to another state

* What are some suggestions regarding the granularity of undo?

- Ignore intermediate states when under continuous interactive control
    - e.g. resizing an object
- Chunk all changes resulting from an interface event
    - e.g. Dialog settings, find and replace all
- Delimit on discrete input breaks
    - e.g. Words, or pauses in typing

* What are some possible levels of undo scope?

System, Application, Document, Widget

* What is forward undo?

1. Save complete baseline document state at some past point
2. Save change records to transform baseline document state into current state
3. To undo last action, apply all the change records except the last one

* What is reverse undo?

1. Save complete current document state
2. Save reverse change records to return to previous value
3. To undo last action, apply last reverse change record.

* What is the memento pattern w.r.t change record implementation?

- Save snapshots of each document state
- Could be complete state or difference from last state

* What is the command pattern w.r.t change record implementation?

Save commands to execute (or "un-execute") to change state.

* How does undo/redo work with the reverse undo command pattern?

- User issues command
  1. Execute command to create new current document state
  2. Push command onto undo stack
  3. Clear redo stack
- Undo
  1. Pop command from undo stack and un-execute it to create new current state
  2. Push command on redo stack
- Redo
  1. Pop command off redo stack and execute it to create new current state
  2. Push command on undo stack

* What does Java Swing use for undo and redo?

Interfaces:
  UndoableEdit: Implemented by command objects. Key methods: redo() and undo()
  StateEditable: Implemented by methods that can save/restore their state.
                 Key methods: storeState(), restoreState()
Classes:
  AbstractUndoableEdit: Convenience class for UndoableEdit
  StateEdit: Convenience class for StateEditable
  UndoManager: Container for UndoableEdit objects (using the command pattern)
               Key methods: addEdit, canUndo, canRedo, undo, redo
  CompoundEdit: Concrete subclass for AbstractUndoableEdit, to assemble
                little UndoableEdits to big ones.

* How do you handle undoing destructive commands?

Option 1: Use forward command undo.
Option 2: Use reverse command undo, but store the previous state for destructive
          commands.

# Cut, Paste, Drag-and-Drop

* Why do we want to support cut and paste and drag-and-drop?

Easy data transfer between within and between applications.
It's expected behaviour of any application.

* What is the clipboard?

A ubiquitous data transfer method.
- Copy information (or pointer to info) to clipboard
- Other applications can read data from the clipboard

* What is a potential security risk of using the clipboard?

Any application can read clipboard information.

Therefore, browsers make the clipboard inaccessible to web apps.

* What is a requirement for clipboards to work seamlessly?

Requires common data formats.

* What should the application do when it places data on the clipboard?

Indicate the formats in which it can provide the data.

* What do the Mac Human Interface Guidelines indicate?

- Put either plaintext or an image on the clipboard
- Accept both plaintext and images from the clipboard
- Should always be able to cut/paste something

* Why is data not always copied to the clipboard immediately? What are the implications?

Why not always copied:
- Data may be available in many formats, wasteful to put all formats on
  clipboard at once
- Data may never be pasted, therefore it could be wasteful

If data is not immediately placed on clipboard:
- App must create a copy if user changes data locally.
- Must put it on the clipboard if application exits

* What is the relevant package and the relevant classes for clipboards in Java?

Package: java.awt.datatransfer
Classes:
- Clipboard
- DataFlavor
- Transferable
- Toolkit

* What is the difference between a local and system clipboard?

Local clipboard are named clipboards holding data only accessible by current
application. e.g. new Clipboard("My clipboard") in Java

System clipboard is the OS-wide clipboard.
e.g. Toolkit.getDefaultToolkit().getSystemClipboard()

* How do you copy data to a clipboard in Java?

1. Get clipboard
2. Create a Transferable object
  a. Defines methods for responding to queries about what DataFlavors are available.
  b. Defines method for getting data of specified type
3. Set clipboard contents to new Transferable object

* What is a Transferable object in Java?

Encapsulates all data to copy in an object.
Similar to UndoableEdit
Important methods:
  DataFlavor[] getTransferDataFlavors()
  boolean isDataFlavorSupported(DataFlavor flavor)
  Object getTransferData(DataFlavor flavor)

* How do you paste data from a clipboard in Java?

1. Get the clipboard
2. See if it supports the desired DataFlavor
3. Get the data, casting it to the proper Java object.

* What does the TransferHandler class do in Java?

Used for handling drag-and-drop.

* How do you support dragging in Java?

1. Set a transfer handler for each component that supports dragging
2. In the source of the drag, define a mouse listener that knows when a drag
   has started
3. When a drag has started, get the component's transfer handler and call its
   exportAsDrag method
4. Override createTransferable and exportDone to do the right things

* How do you support dropping in Java?

1. Set a transfer handler for each component that supports dropping
2. Override the importData method.

# 8 History

* What is an interface?

What the system presents to the user.
It's what you can manipulate and what the system uses to present feedback

* What is an interaction?

The sequence of actions a person expresses and the corresponding system responses.

* What is the history of interaction about?

Making the input and output languages of the machine closer to the input and
output language of the user and their tasks.

* How have the forms of interaction evolved?

Interaction has evolved from forms that favoured the machine (since machine time
was more valuable) to those that favour the user (hardware is cheap, people
are expensive).

* What are batch interfaces? Who were the users?

- Set of instructions prepared, and fed to computer via punch cards, tape, etc.
- Response typically received via a paper cutout.
- No real interaction possible as system executes instructions
- Responses received in hours or days

Users were only highly trained individuals.

* What are conversational/command-line interfaces?

- User types command, waits for resonse
- Programs usually run to completion before response, but
- Feedback can be given during execution
- User is guided through heavily scripted/structured interactions

* What are the advantages and disadvantages of command-line interfaces?

+ High flexible: Can combine commands to create sophisticated sets of commands.

- Users need to understand the computer
- I/O is in system language, not task language
- Requires recall rather than recognition

* What's with problem with both batch and command-line interfaces?

They require an interaction language closer to the system than the task

* What did Ivan Sutherland do?

Early direct manipulation, light pen, graphics.

* What did Douglas Englebart do?

Mouse, cut/paste

* What did Alan Kay do?

Smalltalk (OO), windowing GUIs

* What is Vannevar Bush's "memex"?

A device in which an individual stores all his books, records, etc.
An enlarged imtimate supplement to his memory.

* What is the Xerox Star Information System?

The first commercial computer with a GUI.

* What are modern GUIs like?

Windows, Icons, Menus, and Pointers

* What is the modern GUI interaction style?

User in control: System waits for input, then responds
Recognition over recall: Enables discovery of options and experimentation
Metaphors: "desktop", "folders"

# 8.2 Touch Interfaces

* What is the look and feel?

Look: How manipulatable objects are presented in the interface
Feel: How user expression of these objects is translated into commands to
manipulate underlying data

* What is an interaction model?

A set of principles, rules, and properties that guide the design of an interface.

* What is an interface composed of?

Domain objects: The things of interest, e.g. data
Interaction instrument: A necessary mediator between the user and domain objects.

Domain objects are manipulated using interaction instruments.

* What are the two ways to activate WIMP objects?

Spatially: Caused by moving mouse into a control area
Temporally: Caused by a former action.

* What are the costs of having spatial and temporal activations?

Spatial: Cost of the space occupied
Temporal: Time to activate

* What is the degree of indirection?

Spatial/temporal offset between instrument and action on object

* What is the degree of integration?

Ratio of degree of freedom of instrument / degree of freedom of input device

* What is the degree of compatibility?

Similarity of action on control device/instrument to action on object

* What are some examples of low and high degrees of spatial indirection?

Low: Drag to translate, handles to size rectangles
High: Scrollbars, dialog boxes

* What are some examples of low and high degrees of temporal indirection?

Low: Direct dragging
High: Waiting until exiting dialog

* What are some examples of low/high degrees compatibility?

Dragging (high), scrolling (medium), dialog (low)

* What is a NUI?

A natural user interface, term for promoting direct manipulation techniques like
touch.

* What is a resistive touch interface?

1. Two transparent, conductive layers are separated by a gap.
2. When pressure is applied the two layers are pressed together.

* What is a capacitive touch interface?

1. Senses the conductive properties of an object (e.g. finger)
2. Location of touch is determined indirected from sensors in the four corners.

* What is a mutual capacitance touch interface?

1. Capacitors are arranged in a grid.
2. Touch location is determined by measuring capacitance change at every point
  on the grid.

* What is the advantage of a mutual capacitance touch interface?

Allows detection of simultaneous touches in multiple locations.

* What is an inductive touch interface?

Uses a magnetized stylus to induce an EM field in a sensing layer on the display

* What is an optical touch interface?

Cameras watch the surface.

* What is an indirect input device?

Device is moved on one surface to indicate a point on another

* What is a direct input device?

Unified input and display surface.

* What are some of the differences between a stylus vs touch?

Stylus:
  - Can only have 1 point
  - Doesn't occlude the screen very much
  - High precision
  - Can only use one (preferred) hand
  - Elementary inputs: Tap, Drag, Draw Path
  - Requires a mechanical intermediary
  - High acquisition time, since you have to get the pen out
  - Can have physical buttons
  - Needs non-zero pressure to activate
  - Needs to detect palm.

Touch:
  - Can have up to 10 inputs
  - Occludes the screen a lot
  - Moderate precision
  - Can use any/both hands
  - Elementary inputs: Tap, Hold, Drag Finger, Pinch
  - No intermediary needed
  - Low acquisition time
  - No buttons
  - Can have zero pressure to activate
  - Needs to account for accidental presses
  - Needs to account for phantom presses

* What are some design questions to ask about touch inputs?

- How many points of contact?
- Is the touch reported as an (x, y) position, or is the contact area/pressure
  reported?
- Is a stylus required?
- Can pen contacts be distinguished from touch contacts?
- Can pen contacts and touch contacts be sensed at the same time?

* What is the difference w.r.t input device states between touch and mouse?

Touch interfaces have no hover state. (or at least, no co-ordinates for it).
It's all just touch or no touch.

* What are the challenges of allowing touch input?

- Reduced precision
- Finger occlusion
- Touch feedback ambiguity problem

* What is the fat finger challenge w.r.t touch interfaces?

Occlusion: The user's finger occludes the target before touching the display.
           You can display the cursor at a fixed offset, but that breaks the
           direct manipulation paradigm.

Imprecision: The touch area of a finger is bigger than a pixel.

* What is the ambiguous feedback challenge w.r.t touch interfaces?

Users feel a physical click when they press the mouse.
No such feedback on touch devices.

* What is the "lack of hover state" challenge w.r.t touch interfaces?

Hover state allows the user to preview an action.
This is not possible with touch devices.

* What is the multi-touch capture challenge w.r.t touch interfaces?

Multiple fingers may capture a control at the same time, leading to ambiguity.
This is resolved differently depending on the OS.
Over-capture is possible.

* What is over-capture?

Multi-touch controls are captured by more than 1 contact at the same time.

* What is the physical constraints challenge w.r.t touch interfaces? What are some solutions?

Direct touch breaks when movement is constrained. (e.g. moving beyond bounds)

Solutions: Elastic effects, snapping, "catch-up" zones.

* What is WIMP?

Windows, Icons, Menus, and Pointers.

- Applications objects are displayed in document windows
- Objects can be selected and sometimes dragged and dropped between windows
- Commands are invoked through menus or toolbars.

* What is direct manipulation?

Acting directly on a set of objects in an interface.
e.g. dragging and dropping to the trash bin.

* What is an affordance?

A quality of an object or environment that suggests its use.

* What does direct manipulation try to do with affordances?

DM tries to make affordances in the interface like affordances for the same
actions in the real world.

* Do WIMP GUIs leverage direct manipulation consistently? Why or why not?

No.

Many commands are invoked indirectly. e.g. menus, dialogs, toolbars.
Lots of objects in the interfaces are not objects of interest, e.g. containers

* What are the principles of direct manipulation?

There should be clear affordances. The interaction should be intuitive and obvious.
The task objects are manipulated by physical actions e.g. clicking or dragging.
Almost all actions are reversible, users can explore without severe consequences.

* What are the benefits of direct manipulation?

Users focus more on the task than on the technology.
Feeling of direct involvement with the world of task objects.
Can learn the interface via exploration.

* What are the challenges of direct manipulation?

- Accessibility issues. (especially blind people)
- Switching between keyboard and pointer is time consuming.
- Analogies may not be clear
- Not all interactions are valid.

* What are some things to consider about gestural interactions?

- How should gesture map to system functions?
- Should gestures map to common or complex functions?
- Does each command require it's own gesture?
- How many commands can we expect users to learn?
- How can hand gestures co-exist with pointer interactions?

* What are some problems with direct manipulation a tabletop?

- The fat "body part" probelm
- Content orientation
- Multiple multi-touch input

* What are some things you have to consider when designing for touch devices?

- Displays are smaller
- Large variety of sizes and resolutions.
- Orientation changes
- Only one app in the foreground, others are suspended. Fills entire screen.
- Variable bandwidth (for CPU, memory)
- No hints via hover
- UI needs to be intuitive

* What are some things you can do to make your touch device UI better?

- Help users enter data quickly (e.g. choosing the right keyboard)
- Help users find correct actions (e.g. easier access to frequent actions)
  (make actions obvious)
- Utilize real estate and avoid clutter (e.g. find metadata)

* What does the each javascript touch event contain?

3 lists of touches:
  touches: a list of touches currently on the screen
  targetTouches: a list of fingers on the current DOM element
  changedTouches: a list of fingure involved in the current event

* What are the 3 event types in the javascript Touch Event API?

touchstart, touchmove, and touchend.

# 9.1 Touchless interfaces

* What are touchless interfaces?

Using sensors as the interface.

* What is the main challenge with implementing touchless interfaces?

Hard to implement.

* What are some things we might want to sense in a touchless interface?

Occupancy and motion, Range, Position, Movement and Orientation, Gaze,
Speech, Brain-wave Activity

* What are some things we can do with sensed data in a touchless interface?

- Support in-air gestures: e.g. Hand poses
- Identify objects or people: e.g. Face recognition
- Determine context: e.g. "The user is in the car"
- Determine affect: e.g. Detect emotions

* What are some design considerations for touchless interfaces?

- High computation cost of processing data from sensors
- Use traditional or non-traditional interfaces?

* How do you implement a signal to command touchless system?

1. Preprocess data
2. Selection relevant features
3. Classify

* What is explicit interaction w.r.t touchless systems?

User takes an action and expects a timely response.

* What is implicit interaction w.r.t touchless systems?

Based on users' existing pattern of behaviour.

* What are some challenges of implementing touchless systems w.r.t errors? What are some ways to reduce this problem?

- Need to balance false positive/false negative errors
- Users want to be able to correct errors.
- Users are intolerant of errors

Solutions:
- Graceful degradiation
- Seek deliberation confirmation from users
- Give control: revert to manual mode

* How do you solve the problem that touchless devices are always-on?

Reserved actions: Make a set of actions that are always used for commands.
Reserved clutch: Make an action that starts and stops gesture recognition.
Multi-modal input: Other form of input for controlling device.

* What are some challenges of speech recognition systems?

- Users want to speak their natual language.
- When to ask for confirmation? What if they did not say yes or no?
- High rate of recognition errors.
- Lack of visual feedback
- Speech is easy to produce, but hard to absorb
- Silence can be ambiguous

* What are the 3 types of recognition errors w.r.t speech?

Rejection error: No idea about what the user said
Substitution error: Substituting one word for another
Insertion error: Recognize noise as a word

# 9.2 Perception and Cognition

* What is cognition?

The faculty for processing information.
Attention, memory, learning, reasoning.

* What are the two types of memory?

Short-term, long-term.

* How are memories formed?

Via long-lasting/permanent changes to neurons.

* What is short-term memory?

A combination of perception (makes it available) and attention (focuses on it).

* What are the characteristics of short-term memory?

Low capacity, volatile.

* What are the implications of short-term memory for UI?

UI should help people remember information from one moment to the next.

e.g. Keep search terms on the screen. Keep instructions on the screen.

* What are the characteristics of long-term memory?

Error-prone, weighted by emotions, retroactively alterable

* What are the implications of long-term memory for UI?

Don't burden long-term memory more than you have to.

* What biases perception?

Your expectations, your experiences, your current context, your goals.

* What are the two states of cognitive processes?

Unconscious: Processes we're not aware of.
Conscious: Prcesses we are aware of.

* What is the Locus of Attention?

The feature, object, or idea you're currently thinking about.

We cannot completely control where our locus will be.

* How long does it take for humans to context switch?

10 seconds.

* What is change blindness?

When a change in stimuli is unnoticed.

* What is absorption w.r.t locuses of attention?

You can be more or less absorbed in your locus of attention.
More absorbed = more difficult to transition to another locus.

* What is an automatic action?

Any activity done without conscious thought.

* How do automatic actions affect humans?

We cannot avoid developing automatic actions.
We cannot interrupt a sequence of actions of less than 1-2 seconds.

* How do automatic actions affect UI?

If we make consistent UI, we can build automatic actions on it.
However, we don't want to make some things (e.g. file deletion) be an
automatic action.

# 10 Android

* What are some design considerations when developing for Android?

- Limited battery, processing power
- OS aggressively flushes memory
- Limited background computation
- Small screen
- Multiple orientations and layout
- Single window running at a time
- Multi-touch input

* What are the 3 main invokable components of an Android app?

- Activity: A single screen in an application
- Service: Something that runs in a separate background thread.
- Intent: A way for a component to start another component.

* What is an Android Activity?

Represents a single screen, the application class.
They have an explicit lifecycle and manage their own state.
There have 3 states: Running, paused, stopped.
It's common to switch between activities.

* What is an Android Intent?

Data passers between activities.

* What are the main UI components in Android?

ViewGroups (i.e. layout managers) and Views.
They are typically described in XML.

* What are Android fragments?

Portions of a UI. An activity can contain multiple fragments.

# 10.2 Accessibility

* What is there to said about the concept of an "average person"?

Just a statistical ideal.

* What is there to be said about temporary or situational disabilities?

We all have them.

* What are some examples of temporary or situational disabilities?

Sick, injured, driving a car, underwater diving, texting.

* What are some impairments on you while walking?

- Reduced reading speed
- Reduced reading comprehension
- Higher cognitive load
- Fragmented attention
- Impaired dexterity

* How can you make your UI accommodate the impairments caused by walking?

Varied saliency (e.g. font sizes) helps address limited attention.
Larger visual cues helps address reduced reading ability
Larger inter actors helps address reduced dexterity


* What are the effects of aging?

- Reduced motor coordination
- Visual impairments
- Hearing impairments
- Loss of memory

* How many people have a disability?

10-20% of population. 2-6 million Canadians, 30-60 million Americans

* How many people have a visual disability?

1 in 100.
1 in 475 are legally blind.
1 in 2000 are totally blind.

* How many people have a hearing disability?

1 in 10 have significant hearing impairment
1 in 125 are deaf

* How many people are wheelchair users?

1 in 250

* How does the OS make things accessible?

Have many options, such as:
- Control cursor with keyboard
- Magnify portions of the screen
- Captions and subtitles
- Screen readers

* What is the curb-cut phenomenon? What are some more examples?

Helping a disabled person do one thing may help many others too.

e.g. Cassette tapes, close-captioning, screen-readers

* What are some legal obligations w.r.t accessibilty?

All U.S. government software must be accessible.

* What are some tools for implementing web accessibility?

Alt-text, accessibility-testing tools.

* What are 4 reasons why we should make things accessible?

1. We all have temporary or situational disabilities.
2. Designing technologies to be inclusive benefits everyone.
3. It's the law.
4. Web accessiblity is essential for equal opportunity.

# 11 Input

* How can you classify computer inputs?

Sensing method, continuous vs discrete, degrees of freedom.

* What are the 4 classifications of computer input based on sensing methods?

Mechanical: e.g. Switches
Motion: e.g. Accelerometer
Contact: e.g. Touchscreen
Signal processing: e.g. computer vision

* What are specific and general input devices? What are the advantages and disadvantages?

Specific: Optimized for specific tasks
  e.g. Hand tracking glove
  Good for specific purpose, but fails horribly at others.
General: Adapted for many tasks
  e.g. Mouse
  Good for general purposes, but poor in edges cases (e.g. high DOFs needed)

* What two tasks is input based on now?

Text input and spatial input.

* What is QWERTY designed for?

Space "typebars" to reduce jams in typewriters.

* What are the principles of Dvorak?

- Letters should be typed by alternating between hands.
- Most common letters and digraphs should be the easiest to type.
- The least common letters should be on the bottom row.
- The right hand should do more typing.

* Outside of key layout, what are other problems to consider with keyboards?

Size of keyboard. The smaller the size the harder it is to type on it.
Virtual keyboards have ergonomic problems and size problems too.

* What are some drasticly different keyboard layouts?

Thumb keyboards, one-handed keyboards, Englebart's NLS keyboard, Twiddler.

* Other than a keyboard, what other ways are there to enter text?

Natural handwriting recognition/graffiti gestures.
Predictive text.
Gestural text input, e.g. Swipe and 8 key

* What are some properties of positional input devices?

Force (e.g. joystick) vs displacement (mouse) sensing
Position (e.g. mouse) vs Rate (e.g. joystick) control
Absolute (e.g. touchscreen) vs relative (e.g. mouse) positioning
Direct (e.g. touchscreen) vs indirect (e.g. mouse) contact
Dimensions sensed e.g. 1 = dial, 2 = mouse, 3 = wiimote

* What is the relationship between force/displacement sensing and position/rate control?

Force (isometric) sensing should map to rate control.
Displacement (isotonic) sensing should map to position control.

* What do you need to make relative positioning work?

You need a "clutch".

* What is the relationship between absolute/relative positioning and direct/indirect contact?

Direct contact is usually paired with absolute positioning.
Indirect contact is usually paired with relative positioning.

* What is the Control-Display (CD) Gain?

A scale factor when mapping the control device to the display.
A ratio of display movement to control movement, often in terms of input device
velocity, so it can work with both rate and position controls.

CDgain = V_pointer / V_device

* What is pointer acceleration?

For relative pointing, changing the CD Gain based on current velocity.

* What are gestural inputs?

Movements that are mapped to commands.

# Input performance

* What is the Keystroke Level Model (KLM)?

1. Describe each task with a sequence of operators.
2. Sum up times to estimate how long the task takes.
Operator types:
- K Keystroke = 0.08 - 1.2s
- P Pointing = 1.10s
- B Button press on mouse = 0.1s
- H Hand move from mouse to/from keyboard = 0.4s
- M Mental preparation = 1.2s

* When should you insert an M operation in KLM?

When you need to:
- Initiate a task
- Make a strategy decision
- Retrieve a chunk from memory
- Find something on the display
- Think of a task parameter
- Verify that a specification/action is correct
- Do any action if they're a novice.

* What are the benefits and drawbacks of doing KLM analysis?

Benefits:
- Pretty easy to model
- Can be done from just pictures and ideas

Drawbacks:
- Some time estimates are out of date (due to different input types)
- Some time estimates are inherently variable (e.g. typing speed)
- Doesn't model errors or learning time

* Why doesn't KLM model pointing very well?

- Some pointing devices are faster than others
- Intuitively, it should take longer to move the mouse a long distance, or point
  at a small button

* What is Fitts' Law?

A predictive model for pointing time considering device, distance, and target size.

Main concepts:
- The larger the distance, the longer the time
- The smaller the target size, the longer the time

MT = a + blog_2(D/W + 1)

MT = movement time
D = distance between starting and center of target
W = constraining (e.g. minimum of width or height) of target.
a, b = characteristics of input device.

* What is the index of performance in Fitts' law?

1/b

* What is the index of difficulty (ID) in Fitts' law?

log_2(D/W + 1)

* What is the advantage of Pie Menus over Context Menus w.r.t Fitts' Law?

Both Context Menus and Pie Menus are near the mouse, so they lower D.
However each element has variable D for a context menu.
Each element has the same D for a pie menu.

* What does Fitts' Law say about OSX's dock expansion?

The target space is not increased, so it's not easier to press.

* What is the difference between motor space and screen space?

If you make a button move slower when mousing over a button, you increase
the motor space while taking up the same screen space.

* What is Steering Law?

An adaptation of Fitts' Law for constrained paths.
We represent a fixed path as a series of minor goals of Fitts' Law.
Instead of D, we have A = the distance/length of the constrained path.
After solving a limit, we get

Index of Difficulty (T) = b * A / W

And we sum the parts of the path to find the total cost.

* In general, what does Fitts' Law tell us about UI design?

- Keep things close.
- Make things larger.
- Manipulate motor space to make intended targets sticker.