The Multitouch Aquaria Project

Tracking Down the Touchscreen Bug

Thursday 2011-03-17

I mentioned in my last post that I have been struggling to diagnose what I call the “Touchscreen Bug”. The following video shows one symptom of the problem, where the cursor flies to the extremes of the screen, limiting Naija’s range of motion to the semi-cardinal directions.

In other cases, the cursor is stuck in the bottom right corner of the screen, making it impossible to move at all.

I was unable to find anything in the Aquaria source code that would produce the sort of behavior I observed, but suspected that it was due to the use of an absolute input device instead of a mouse. Seeking the root cause I attempted to reproduce the issue with the SDL library that Aquaria uses to handle input on Linux. Interestingly, the SDL mouse motion event documentation explains:

If the cursor is hidden (SDL_ShowCursor(0)) and the input is grabbed (SDL_WM_GrabInput(SDL_GRAB_ON)), then the mouse will give relative motion events even when the cursor reaches the edge fo the screen.

We can replicate these conditions using the Ruby SDL bindings like so (download):

#!/usr/bin/env ruby
require "sdl"
SDL.init(SDL::INIT_VIDEO)
screen = SDL::Screen.open(1024, 600, 0, SDL::FULLSCREEN)
SDL::Mouse.hide
while event = SDL::Event.wait
    case event
    when SDL::Event::KeyUp
        if event.sym == SDL::Key::ESCAPE
            SDL.quit
        end
    when SDL::Event::MouseMotion
        puts "pos = #{event.x}, #{event.y}"
        puts "rel = #{event.xrel}, #{event.yrel}"
    end
end

Running this script (which requires libsdl-ruby) under Ubuntu Natty and dragging your finger around on the screen reveals that SDL is reporting the absolute coordinates of the mouse pointer as relative motion, while the absolute coordinates are held at (1023, 599)—the bottom right corner of the screen. Running it under Maverick gives the behavior in the video above, with the cursor “sticking” in the corners.

But does this happen for other input devices? A mouse or trackpad works as expected. To test this theory, I obtained a Wacom pen tablet. This produced results similar to the touchscreen, both in Aquaria and the test script. Armed with this knowledge, I was able to locate another project with this problem:

I wanted to play BOS Wars using a Wacom Bamboo CTE-650 tablet. Unfortunately, the mouse cursor is practically impossible to control with that when BOS Wars is in full-screen mode. It seems the position of the pen on the tablet is controlling the velocity of the mouse cursor, rather than the position. There is a tiny spot on the tablet where the cursor stays stable. Move the pen even a millimeter and the cursor wildly swings to the edge of the screen.

The Debian bug thread goes on to discuss several possible solutions. The least invasive of these is to add the environment variable SDL_MOUSE_RELATIVE=0, which disable the relative motion “feature”. This is the fix that I have implemented in Aquaria.

At this point single-touch navigation works, though it is a bit finicky. Aquaria’s buttons don’t respond well to touch—they must be held slightly too long. In the coming weeks I will undertake such minor interface tweaks. First, however, I need to add gesture support to allow singing and changing forms.

©2011 Tom Most, Team Modjeska   •   Part of freecog.net