Research project on rollover (for Qukeys)

(Michael Richters) #1

I’m hoping to find some volunteers to help me with a little research project. A recent conversation on the forum gave me a few ideas for possible improvements to Qukeys. Unbeknownst to me when I started writing that plugin, a substantial number of people roll over from modifiers to other keys – or at least tend to release the two keys simultaneously when chording – rather than holding the modifier for the whole duration of the modified keypress. Qukeys works by deciding which keycode to use based on the order in which the two keys are released (if the subsequent key is released first, the qukey becomes a modifier). This means that “rollover-style chorders” often get the wrong keycode when typing, because they tend to release shift (for example) before T when typing T.

There is a system that is intended to help with this, but it hasn’t been working as well as it might, and I now have a new idea that might be better. I have a guess about how keys are often typed in rollover, but I would like to gather some data from real typists in order to test my hypothesis, and perhaps find a good default value for the configuration parameter that would be used. In order to do this, I’ve put together a new plugin: Kaleidoscope-RolloverLogger.

This plugin is a kind of keylogger. It writes one line to the serial port for every keypress, reporting the time it toggled on, the time it toggled off, and whether or not that key was a modifier (including layer shift keys). It does not (by default) log actual characters typed, nor keyswitch coordinates, so it should be extremely difficult for anyone to extract from such a log the actual input that was typed.

If you are interested in contributing some data, feel free to add the plugin to your sketch briefly (I strongly encourage you to remove it as soon as you’re done), set up whatever system for piping the serial port output to a file you find convenient, review the file, and send it to me: I’m happy to help anyone who has trouble setting up something to record the output.

There is more detailed information about the plugin in its README file, but I want to emphasize one point about it here: It is a keylogger. Please don’t type anything sensitive (e.g. passwords) while it’s installed in the firmware of your keyboard. I’ve done what I can to make the default output as useless as possible for malicious purposes, but RolloverLogger does have modes that output keyswitch coordinates and even key values to the serial port (feel free to use that functionality, if you have a use for it). I would not leave it installed in my keyboard’s firmware more than an hour or so at a time, and I would limit myself to doing things like typing practice in that time.


(Noseglasses) #2

I had to provide the VID and PID info to make capturing serial output work. For the Model01, e.g.

cat $(Kaleidoscope/bin/find-device-port-linux-udev 1209 2301) | tee /tmp/RolloverLogger.log
1 Like

(Michael Richters) #3

Oops! I forgot that those scripts aren’t quite the same as the the one that I use for debugging. Danke!


(Michael Richters) #4

I didn’t find the time yesterday to elaborate on my idea for the possible improvement(s) to Qukeys that constitute the raison d’etre of this project, so I’ll try now.

The Qukeys rollover problem

When I first wrote Qukeys, I assumed that people ~always hold modifiers for the full duration of the modified keypress (as I do). In other words, to send ctrl + S, I would press ctrl, press S, release S, then release ctrl. This is how Qukeys determines which state a qukey ends up in – by delaying both keypress events until one of them is released.

I then learned that my assumption was faulty, and there are plenty of people who frequently roll over from modifiers to other keys (i.e. press ctrl, press S, release ctrl, release S). Some people tend to release the two keys simultaneously, which results in a race condition. My solution was to introduce a configuration parameter, allowing the users to select a “grace period” for qukey release, so that if the S in the above example was released just a few milliseconds after the modifier, it would be treated as if it was released first.

That grace period idea helped, but it also made it possible for very fast typists to get unintended modifiers if they set up home-row qukeys. Typos from unintended letter keys can be frustrating, but unintended modifier keys have a much greater potential for destruction, so I always caution people to keep the release delay parameter set fairly low, and some users haven’t been able to find a setting that works well enough.

A new solution (or two)

It occurred to me after a recent discussion that I was not using all of the input data available, and that I might be able to do better. In a rollover situation, there are four points in time that can be measured:

  1. qukey press time
  2. other press time
  3. qukey release time
  4. other release time

…but I was only using two of them (the two releases) to decide whether the qukey should take on its primary or alternate Key value. In general, my goal with this research project is to get a data set showing how people use modifiers in combination with other keys – particularly when they roll over, and try to find the best algorithm to use all the available information to produce the intended output.

Specifically, I have one idea that I want to test, using just three of the timestamps above. This idea is to compare the duration of the overlap (between points 2 & 3, when both keys are held) to be at least x percent of the total duration of the second key press (between points 2 & 4). This would make the effective release delay dynamic, so it would be longer or shorter depending on how long the two keys overlap. I’m hoping that this will produce better results than the fixed-length release delay, and as a side benefit, it’s actually simpler in the code.

Of course, that only uses three of the timestamps available, and it’s possible that, for people who regularly roll over from modifiers (rather than those that release the two keys ~simultaneously), it might not provide a clear distinction between typing overlap and modifier use. In that case, there might still be a pattern that can be used based on the relative timing of the two key presses, or the duration that the first one is held, as well. And for people who really do roll over when using modifiers in the same way they do when typing normally, there still might be one possible distinguishing factor I could find by looking at the key press(es) preceding the modifier – I’m guessing that it’s more likely to have a gap before pressing a modifier key than a printable character key, though that would probably be somewhat less true of qukeys on the home row, especially ones with an alternate Key of shift.

A related problem

I’ve got one other Qukeys improvement in the works, which is related. A few times, people have reported rollover-related errors when using qukeys for what would normally be modifier keys (e.g. the thumb keys on the Model01). In these cases, I have usually recommended SpaceCadet instead, because it is intended for the purpose (using what is normally a modifier as a printable character if it is tapped alone), whereas Qukeys is the reverse (using printable character keys as modifiers if chorded). However, as a result of the release delay conversation, it occurred to me that a very small change would allow Qukeys to do both.

The idea is that if a qukey’s primary Key value is a modifier (including layer shift keys), it will be treated like a SpaceCadet key instead. It will always produce that modifier, unless it is tapped on its own. In other words, if there’s any rollover from a shift / X qukey, it will be shift, but for a X / shift qukey, its value will depend on release order and timing.

This might be useful because Qukeys offers more flexibility than SpaceCadet in defining which keys are affected. With SpaceCadet, if Key_LeftShift produces ( on tap, every Key_LeftShift will do so, regardless of where it is in the keymap. Qukeys specifies the exact coordinates of its keys, so you could have two different Key_LeftShift keys (probably on different layers), one of which has an alternate key value, and one that doesn’t.

One last note: If I can get a variety of contributors, I might even be able to produce a script that can be run on a future version of the data file produced by RolloverLogger that would recommend certain settings for the users to minimize Qukeys errors.

Again, I will be very grateful to anyone who helps me out by contributing some typing data so I will be able to make some better choices in the design of Qukeys. Even if you’re not a Qukeys user, it could still be very helpful to get typing rollover data.


Unexpected characters using Qukeys
(Michael Richters) #5

I implemented my new Qukeys variable rollover delay protocol in Kaleidoglyph, and my initial impression is that it might work even better than I hoped, even with a minimum overlap requirement of only 50%. I’d still like to get some real-life rollover data, but I’ll port my changes to Kaleidoscope when I get the time, so other people can give it a try.