Launcher Reveal Prototype

Download the prototype application and source code

Launcher Reveal Considerations

With a Launcher available on each display, we have chosen to hold the cursor briefly at the edge of any display which does not sit on the left-most edge of the extended desktop, allowing the user to push against the edge to reveal the Launcher. Important considerations here are:

• The Launcher reveal must be reliable and easy to achieve when required

• The Launcher reveal must not be too sensitive: there is already an issue with false positives for the reveal, when targeting items near the left edge of the screen (eg. the browser Back button)

• The user should be able to pass quickly and easily onto the display to the left – they will not always be looking for a Launcher reveal.

• Related to the previous point, if a user overshoots onto a display to the right (when targeting items such as scrollbars on the far right of a left-located display), it should be quick and easy for them to correct their position back onto the other display.

Running the Prototype

The prototype is a C++ Qt application. Download the archive and unpack it. You may be able to launch the application by just double-clicking on it. If this doesn’t work, you will need to launch the executable from the terminal. The steps are as follows:

• Unpack the downloaded archive to a suitable location. In this example, we unpack onto the Desktop.

• Open the LauncherReveal folder on the Desktop and try double-clicking the LauncherReveal executable to launch it.

• If this doesn’t work, launch the Terminal application.

• Type ‘cd Desktop/RevealLauncher’ to change to the directory which contains the executable (replace ‘Desktop’ with the directory you have unpacked the archive contents into if necessary)

• Type ‘sudo ./LauncherReveal’ to launch the executable. To grant permission for the system to run the executable, you will then be prompted to enter your login password.

We have used quite a low-level language and framework for the prototype because it needs to create windows across multiple displays and to manipulate the position of the mouse cursor.

You can run the prototype on a PC with a single display, and try out the Launcher reveal. However, the prototype really becomes interesting when you run it with more than one display attached, and check out the Launcher reveal across all displays.

This prototype concentrates on the Launcher reveal only, so there are lots of things (windows, top bar) which do nothing. The prototype will only work properly for multiple displays with the same height, organised in a row. Being a prototype, this is essentially throw away code, which lives just to explore a very specific set of interactions, for a limited configuration of hardware.

In order to reveal the Launcher, you push the cursor into the left edge of a display for a fraction of a second (100ms by default). This works on any display (not just the far-left display), as we hold the cursor at the edge of a display when it crosses from either the left or right. Push a little more (a further 150ms by default) and the cursor will break through onto the next display.

The images of the browser window are used to test for false positives and overshooting problems when trying to target the Back button on the far left, and the vertical scrollbar on the far right.

Tuning the Parameters

You will notice a panel with lots of parameters to tweak. We have chosen defaults which work well for the small sample of people and hardware we have tested the prototype with so far – informal ad-hoc testing at this stage. Here is an explanation of each parameter and the trade-offs they represent:

Launcher reveal: push for 100 ms
The length of time the user must push against the left edge of the display to reveal the Launcher. For lower values, you get a more responsive-feeling Launcher, but you also get more unwanted reveals when targeting items on the left of the display (eg. the browser back button). Try tuning this value up to 200ms for less unwanted reveals, but you’ll need to push a little harder for those reveals that you want.

Pass display edge: push for 150 ms
Once the Launcher has revealed, we don’t want the cursor to break straight onto the next display, so we continue to hold the cursor for a little more time on a left-most edge. This gives the user the opportunity to stop pushing and move to target a Launcher icon. Lower values make it easier to move from one screen to another, but more likely that you will break though onto the next display when you wish to target something in the Launcher.

Event sampling period: 50 ms
The event sampling period is the size of the time-slices which are used to determine when the user has stopped pushing. A time-slice which collects no mouse events will result in an ‘end-of-push’ condition, cancelling a Launcher reveal or movement across a display edge. Lower values will increase the chances of an unwanted ‘end-of-push’ condition (for gentler pushes or older hardware). This period must be large enough to divide the previous time values into multiple time-slices, otherwise they just degrade into timer delays.

Cursor travels freely after crossing display edges for 1000ms
In order to allow the user to quickly correct an overshoot onto another display (when targeting items to the left or right extremities of a display), we temporarily drop the hold-at-edge behaviour once the cursor crosses an edge. Lower values give the user less time to make any corrections, but making this value too large results in missed edge-holds and Launcher reveals.

Cursor travels freely at velocities over 1000000 pixels/sec
In the prototype, we have prioritised easy, predictable Launcher reveals over travelling very quickly across the extended desktop. The user can still travel across the desktop fairly quickly, although they will be detained for a fraction of a second at any edges in the way. If you want to try out an alternative prioritisation (quick travel across the extended desktop, requiring a slower, more careful and deliberate targeting of the edge for a Launcher reveal), then drop this value down from the very high default value (which effectively disables this feature), to something in the region of 2000-5000 pixels/sec.

Hold cursor at right edge of displays (true)
We found that symmetric behaviour, with respect to holding the cursor on display edges, feels more natural, and also makes it easier to target items near the right edge of a display. However, this feature can be disabled, in order to evaluate whether less intervention on cursor movement might be preferable.

Show Launcher proximity shadow (true)
A shadow will appear at the left edge of a display to improve the discoverability and feedback for the Launcher. It grows out as the cursor approaches the edge, and then grows further still when the user pushes against the edge, providing feedback that the push is being recognised and the Launcher is about to reveal.

How It Works

You can check out how the prototype works by looking at the source code. If you know some C++ and Qt, you should hopefully be able to make some sense of it.

Fundamental to the interactions here, is the ability to determine when the user starts to push against an edge, and when they stop. We measure the duration of the push to see if we should reveal the Launcher or let the cursor through onto the next display. It is straightforward to determine when the push starts: we identify the first mouse move event which crosses a display edge. But how to determine when the push ends is more difficult. Jason Smith, on the DX team, came up with the neat solution of splitting the entire duration of the push into smaller time slices. For each time slice, we count the number of mouse move events coming in. As soon as a time slice expires which has collected no mouse events, then we have the end of our ‘push’.