Keyboard report - clear in every cycle vs. report on key event


(Noseglasses) #1

tl;dr

What is the advantage of bundling key events and sending USB keyboard reports once per loop cycle, thereby clearing the report in every cycle, against an alternative approach that issues a key report whenever a key state changes, without ever clearing the keyboard report data structure?

Lengthy explanation why this is relevant

Finally I got around to port my QMK firmware-mod and keymap to Kaleidoscope. When I started (months ago), I expected it to be much easier than it had been using QMK. Kaleidoscopes’s design, with its plugin interface and hooks appeared to be pretty well suited. That was one of the major advantages that attracted me coming from the QMK world.

Eventually, it turned out that the port was almost as hard as it was to implement all the stuff for QMK in the first place. Why was my primary assumption that wrong?

One of the issues that cost me the greatest amount of time to solve where the various sorts of rollover issues that were mostly related to the way USB keyboard HID reports are dealt with by the firmware.

I’d like to start with a brief explanation of how USB keyboard reports are currently generated. This is for all those who have not delved into the firmware core yet.

The USB HID keyboard report is essentially a data array that is passed to the host system once in a while (usually once per loop cycle after keys have been pressed and released) to inform the OS about key events. The key report thereby contains information about keys and modifiers that are currently pressed. It is the host OS’s task to compare consecutive reports to find out which key states actually changed. The key report array can be modified programmatically, e.g. from plugins’ hook methods or macros, but it is mostly changed by the firmware core when changes of key states are are detected during key matrix scans.

In every cycle of the firmware loop, first the keyboard matrix is scanned for changes. Event handler hooks enable firmware plugins to react on newly pressed/released keys and modify the keyboard report accordingly. In the next stage, another set of hook methods, the pre-clear loop hooks, may again modify the keyboard report before it is finally send to the host. After the report was send, the report data structure is cleared. Then, at the end of the loop cycle, another set of hook methods, the post-report loop hooks are allowed to pre-populate the cleared keyboard report before the next loop cycle commences.

For most applications this approach of bundling key events and sending and clearing the report data structure is sufficient. But for more complex applications like the plugin I recently developed, this is a quite problematic approach. The plugin I am speaking of - working title Kaleidoscope-Papageno - does some complex types of key event pattern recognition (clusters, chords, arbitrary key sequences, and tap dances) with the minimum possible memory (RAM and PROGMEM) footprint. This is quite a complex thing (ok - frankly, metaphorically it’s close to Frankensteins’ Monster :wink:). But it is very powerful.

However, there were many corner cases I had to eliminate, which I finally managed with the help of a firmware simulator that I developed, mostly for this purpose. Without its ability to simulate the exact timing of the firmware loop, and a huge amount of regression tests I would have been lost. But even with this nice tool I spend hours banging my head against whatever hard objects I could find.

Apparently, I am not alone with this sort of problems. In his posts here in the forum @merlin mentioned he also had to deal with similar problems to make his popular Qukeys plugin work. And maybe there are others who experience the same.

Most of our problems come from the fact that key events are bundled into keyboard reports, which are send in each loop (only if the report data structure changed compared to the previous send attempt). The report data structure is thereby cleared before the key matrix processing of the next cycle happens. This has several drawbacks

  1. When keys are pressed and released in the same cycle, the report does not reflect it.
  2. Most hooks see incompletely populated reports.
  3. A loop hook cannot see if keys changed unless it stores the completed report (which it cannot as the report is only complete in the moment when it is send).

These are only some of the aspects that might cause problems.

I heard rumours about serious attempts to change the current key event handling towards an event driven approach. This is a complex task. It will take some time until it will be production ready and, of course, only if the gods of Kaleidoscope are well meaning towards such changes.

For the meantime, maybe it would be possible to change the current implementation a little bit to make it easier to work with by no more clearing the key report in every cycle and by sending a report after every key event (key press and release), thereby staying compatible with the existing plugins?

I am not sure if I am missing important details here that would explain why things are done the way they are done right now. Also, the approach I proposed might have some severe drawbacks that I overlook (although my firmware mod works nice and fast based on the proposed changes).

In any case I would be happy about explanations, comments and discussion.

cheers


(Noseglasses) #2

Oh, I forgot to mention, coming from ErgoDox and Planck I was within hours at my normal typing speed after switching to the Model01.

Thanks to J&K! This is really a nice piece of wood!


(Jesse) #3

The very, very short explanation is that maintaining a key report’s state across cycles rather than rebuilding it in every cycle tends to lead to ‘stuck’ keys with alarming frequency. It’s my understanding that this was the cause of many of the complaints about Kinesis’ USB Advantage (I) keyboard.

The implementation you describe is indeed, the “obvious” right way to do it. And it’s how I did it at first. It was…painfully unreliable. If, -anywhere- in the pipeline, a key change event gets dropped, you end up with really angry-making misbehavior.

That sounds correct to me. A key pressed for 0 cycles is, by definition, not pressed for any cycles.

Many plugins don’t care. But some desperately need to be able to see the whole keyboard state. And right now, that’s painful (or impossible)

@algernon is almost done with an update of your plugins proposal. By moving from a preallocated array of hooks, we’ll be able to expose additional hook points at very little cost. Like, say, hooks AFTER all the keys have been scanned.

I agree that this is a problem and I believe that getting you more hook points will solve this in a simple and relatively elegant way.


(Noseglasses) #4

Unless some of those key events are issued programatically by hooks. The INJECTED flag is not of much help here as it is meant to prevent infinite hook loops.


(Noseglasses) #5

More hooks are definitely desirable. :+1:t5: But I am not quite sure that they can solve all the problems I had to face. But I will see once they are available.


(Noseglasses) #6

I experienced some of this. But it was pretty easy to track down using the simulator. But I understand and appreciate your intention to provide a robust and reliable product.


(Noseglasses) #7

…or is it that some of the reports got lost on their way between the keyboard and the OS?


(Michael Richters) #8

[I haven’t read the whole topic yet (I will, soon), but I want to add something real quick before the discussion gets away from me completely.]

In my experimental Kaleidoscope fork, I don’t keep the HID report structure from one report to the next, but I do store a representation of the keycode values for each key, from which the next report is built. So there is definitely the possibility of getting keys “stuck”, and I did, predictably, have a bug or two of that form as I was building it. However, there are plugins that explicitly work by making keys “stuck” (e.g. OneShot), and although there are fewer places in which that can happen if the report is built fresh from the scan in every cycle, it’s still possible to get “stuck” keys, and there are plenty of other problems that are harder (or nearly impossible) to solve (e.g. scan order bugs).


(Michael Richters) #9

More detail, informed by my experiments:

Keeping and modifying the HID report data structure instead of clearing it (or creating a new one for each report, which is what I decided to do) doesn’t preserve enough information. It would work on a keyboard where each key was restricted to sending one unique keycode, but Kaleidoscope lets us define keys that send multiple keycodes, and more than one key can be responsible for the same keycode. Most typically, if we define a key with a modifier flag (e.g. control + s), but also hold a key that sends just that modifier (control), releasing the former key should not remove the modifier from the report, because the user is still holding a key that sends that keycode.

So I’ve experimented with an array of Key objects — one for each key on the keyboard, storing their current values. Without any plugins active, each key’s value is Key_Transparent if it’s not pressed, Key_NoKey if it’s masked, and some value looked up from the keymap if it is held. The obvious danger, as @jesse pointed out is getting “stuck” keys by failing to clear the values in this array when a key is released. Plugins with event handlers can obviously cause that type of bug, and it’s a very bad type of bug — much worse than most. On the other hand, most plugins don’t need to do anything on key release events, and can just let the core controller process them normally. Then there are the plugins that work by temporarily getting keys “stuck” (e.g. OneShot), and I found this to be a very simple (and therefore easily debugged) mechanism for doing so. So far, I’ve written two plugins for my experimental fork, and found both to be much simpler than their counterparts for Kaleidoscope proper. Qukeys is about half the amount of code, with more functionality. Unshifter (a version of TopsyTurvy + Shapeshifter) was so difficult to implement for Kaleidoscope that I gave up in the design stages, but took only a few hours to get right for my fork. It’s very short, with dead simple hook functions.


Regarding specific hooks — I’ve tried out a different structure than what Kaleidoscope has, with (so far) four different plugin hook points:

  • pre-scan
  • event handler
  • pre-report
  • post-report

The pre-scan hooks run every cycle, before the key scan. Useful for timers and anything that changes regardless of key events (e.g. LED effects).

Whenever a key is pressed or released, the event handler hooks get called.

Whenever a report is about to be sent (probably due to a key press or release event, but not necessarily), the pre-report hooks get called.

After a HID report is sent, the post-report hooks get called.

Unlike Kaleidoscope’s event handler hooks, these ones get called for single events, and (generally) a new report is sent for each one. So if two keyswitches change state in the same scan cycle, event handlers will get called twice, and two reports will be sent before the next keyswitch scan. The event’s Key value get looked up in the keymap, and is then passed by reference to the event handler hooks, which can change that value before the next plugin gets it. Only after all the event handlers are done does the controller update the Key value in the cache. This way key events don’t result in HID reports unless all event handlers allow it, so any event handler that returns false (abort the event) must be very careful about key release events.

I currently have the pre-report hooks work the same way, but I’m starting to think this isn’t worth the trouble. The post-report hooks are handy for bookkeeping; they also get passed the event Key, but not by reference, and they run unconditionally any time a report is sent. I added this hook when writing Unshifter as a simple way to keep track of the number of “real” shift keys currently being held.

Altogether, I’ve found this system to be reliable so far, and a good compromise; trading scan order bugs and related complexity for the hazard of possibly getting (unintended) “stuck” keys.


(Michael Richters) #10

I don’t think this is a problem for either system. I admit that I don’t know how the USB HID events are transmitted, but I’m guessing that the USB controller on the ATmega32u4 caches the last report it gets, and provides it to the host on request, so one report explicitly sent by the firmware could be several reports from the keyboard to the host. Am I close?


(Noseglasses) #11

This became a reeeaaallllllyyyyyy long post, sorry for that. But there is quite a lot to say about key reports and event handling.

@merlin: Just to check if I understand correctly how your approach works: You do clear the report in every cycle but you preserve the state of the keys’ keycodes in a separate state array. This array is modified by the event handlers and other hooks. Then, whenever an event occurs, the actual report is composed based on the current state of the keycode state array. Reports are send immediately after every key event.

Did I get it right?

What about the extra runtime costs of checking the keycode state array? Are they significant? The state array can in most cases be expected to be sparsely populated (different from Key_NoKey and Key_Tranparent) with only those keycodes that represent keys that are actually held.

In any case, this data structure (AFAICC, 128 bytes RAM) seems like a great improvement for those plugins that want to keep track of what actually changed between reports/events and about what modifiers will end up in the keymap.

One could even think about a slightly greater dimension of this state array in order to add auxiliary keycodes that are not related to actual keys on the keyboard but can be used by plugins to store additional information that is not supposed to be overwritten during event handling. I am just brainstorming here…


Right now, I am starting to understand how your proposed changes would affect my Papageno plugin. Still I am not sure if these changes would solve all of my problems. To give you a hint about what these problems are, I will describe the Papageno plugin’s basic algorithm.

The plugin’s main task is to do key event pattern recognition. If hit in a specific order (possibly multiple keys concurrently) specific actions are triggered, similar to what the macro or the tap dance plugins do.
To make this possible, the plugin keeps track of the order of key events. Pattern matching hereby operates on matrix key positions (not keycodes). This requires a handler that is passed row/col values (like the current eventHandlerHook - not sure if this is possible with a handler of your new design). The plugin’s key event handler adds key events of specific keys (those that are explicitly registered from within the sketch) to an event queue. After an event was added to the queue, the event handler blocks other handlers by returning false. Some key events are thus temporarily swallowed. If a key was not registered in the sketch to be part of any pattern, it is just passed through to the consecutive handlers and pattern matching, if currently in progress, is aborted.

While pattern matching is going on, after an event was added to the queue, the pattern matching engine checks if it causes a pattern match, based on the other events that are already waiting in the queue. If a pattern matches, an action is triggered that is specific for the matching pattern. Actions can cause the generation (not injection - see below) of artificial key events, thereby passing matrix positions or keycodes. An action can also mean a registered user function to be called. All keys in the queue that belong to the pattern match are discarded afterwards.

Whenever a key event arrives that leads to a pattern match failure, the oldest event at the front of the queue is removed and passed back to Kaleidoscope (flushed). Then, pattern matching is restarted based on the remaining events. The resumed pattern matching may or may not immediately fail, which in turn causes more events to be flushed. There is also a user defined timeout that causes flushing of the whole event queue. You see, there are different reasons why key events (key matrix positions & press/release) must be flushed.

When flushing events, Kaleidoscope must be fooled to believe that the flushed key events just arrived during a matrix scan (they cannot be INJECTED, see explanation below). At first glance this seem(s/ed) straight forward. It works well with simple keys, like e.g. Key_A. It terribly fails with most other keys, such as controller keys (e.g. OneShot modifiers or layer changes) that are processed by plugins’ event handlers. The simple reason is that based on such keys (eventually mapped to keycodes) some plugins require their loop hooks to run, e.g. to populate the key report before the next firmware cycle starts (OneShot). Unfortunatelly, this means, to be on the safe side, I have to simulate an individual loop cycle after every flushed event.

In an attempt to solve this problem I started to call the lower parts of Kaleidoscope.loop() that come after the matrix scan part after every flushed event. A flushed key event would then behave the same as if it just arrived during a matrix scan that is normtally immediately followed by loop hooks and report.

While this worked in some cases, in other cases it caused empty reports to be send. This is because I flush keys from within my plugin’s event handler. This can cause a simulated loop hook/send report as part of an event flush being followed by the loop hooks/send report of the remaining current loop cycle of the real firmware loop. Such empty reports sometimes go unnoticed, sometimes they cause strange side effects. Under some circumstances they e.g. can cause double characters if a key is held for more than one cycle (which is almost always the case).

I tried for ages to find a good solution that would work with Kaleidoscope’s current implementation. And I really tried LOTS of different approaches. Finally, after a long desperate fight - man against firmware - I gave up and changed small parts of Kaleidoscopes loop handling and event processing. The most prominent change is that I had to remove the report clearing after the pre-clear hooks inside of Kaleidoscope::loop(). Also I had to trigger additional reports from within handleKeyswitchEventDefault. When keys are toggled (on/off), reports are now send immediately.

Another problem I had to deal with is related to INJECTED keys. In the current implementation handleKeyswitchEventDefault checks for the INJECTED keyword and ignores toggle-off/key-release if an event is not INJECTED. No idea why this is required in the first place? Anyway, this check I had to remove as well, to allow multiple taps of the same key, represented by a series of flushed events. This is inevitable as I cannot make flushed keys INJECTED. I need to have them being treated like if they just arrived during matrix scan. Most key event handlers (like e.g. OneShot’s), however, ignore INJECTED keys in an attempt to prevent infinite recursion.

As a sidenote: The INJECTED approach used by some plugins has the shortcoming that if a plugin injects a key, this key will not only be ignored by its own handler but also by all those other plugins’ handlers that check and ignore injected keys. This is in some cases very undesired behavior. IMHO, it would be more robust/flexible if every plugin would use its own boolean flag to prevent undesired reentrance of its own handler instead of using the keycode for this bit of information.

It broke my heart, having to modify Kaleidoscope’s core. I desparately would like to find a way to get my stuff going with stock Kaleidoscope, not to have potential users being required to rely on a custom Kaleidoscope-mod.

Kaleidoscope-Papageno now works fine together with OneShot but I did not try any other plugins, yet. Any plugins that work based on specific assumptions about the call-order of handlers might fail.

As mentioned before, I had some hanging keys (with the restricted number of plugins I currently use). But all of it was easy to track down using simulator and debugger together. Now everything works like a charm and I have a bunch of regression tests that I can use with travis and appveyor.


(Michael Richters) #12

[I’ll try to write short replies to each point. For the sake of brevity, I’m calling my experimental fork “Kaleidoglyph” to distinguish it from the mainline Kaleidoscope.]

Re: Kaleidoglyph design

You’ve got the basic idea correct about how the Kaleidoglyph main loop works, except for some unimportant details. One thing I left out is that it’s not just the current event’s Key value that gets passed to the event handlers; I use a structure that includes the Key, the KeyAddr (equivalent to row & column), and the keyswitch state (toggled on / toggled off / held / idle – in practice only the first two are used).

Comparatively, the cost is insignificant. In Kaleidoscope, there is essentially one pass through the array, and each plugin’s event handler is called for each non-idle key (until recently, there were a lot more calls). In Kaleidoglyph, there’s only one set of event handler calls per event, then one pass through the active_keys array to populate the report, which is only done in response to a keyswitch event. I haven’t measured the time it takes explicitly, but I have measured the average cycle time, with a stripped down version (no plugins active). Average cycle time with Kaleidoglyph was ~427µs when idle, and ~435µs when mashing the keys as fast as I can. Kaleidoscope, similarly stripped down, was ~500µs idle, and ~560µs in its worst case condition (lots of keys in different columns held).

I’ve been planning to write a Macros plugin at some point that has its own supplemental Key array of virtual key values (I think 8 would be sufficient, but I’m guessing here). It’s pre-report hook would be responsible for adding those keycodes to the report. This would only be necessary for plugins that would need more than one Key active at a time from a single physical key.

Re: Papageno

Papageno is very similar to Qukeys in how it works (storing a queue of keyswitch events to be released later). Qukeys has a much narrower purpose, but I think I can offer you some ideas. Kaleidoglyph greatly simplified Qukeys; it’s less than half as much code, and much easier to follow. That’s not entirely due to the event-driven design changes, but that is the bulk of it.

I had the same problems you did with flushing keys from the queue from within the event handler, since the new HID report was not (necessarily) complete, but my approach was to copy the previous report (from KeyboardioHID, instead of using the HID façade, thus breaking an abstraction barrier), modify that report, send it, then restore the current (incomplete report). I think this was more straightforward than calling parts of the main loop function, but that idea didn’t occur to me, so I can’t say that for sure.

None of that is necessary with Kaleidoglyph, because there’s one report per event, rather than one per cycle, so a lot of the code just disappeared.

Re: “injected” events

In Kaleidoscope, Qukeys sets an internal flag (flushing_queue) before (re-)calling handleKeyswitchEvent() rather than using the INJECTED state bit, precisely so that the event handlers of other plugins wouldn’t also ignore the event. This works well enough, but if both Qukeys and Papageno (for example) are running in the same sketch, and both use flags like that, it will easily cause an infinite loop as each plugin enqueues events flushed by the other, then later flushes the event.

I added a simple function to Kaleidoglyph’s Plugin class to better solve this problem – a method defined like so:

  virtual bool checkCaller(Plugin*& caller) const final {
    if (caller == nullptr)
      return false;
    if (caller == this)
      caller = nullptr;
    return true;
  }

Just about every plugin should probably then contain this at the beginning of its event handler to guard against repeat handling of (re-)“injected” events:

  if (checkCaller(caller))
    return true;

…and if it needs to re-start handling of an event (e.g. when Qukeys or Papageno flushes a key from its queue), it calls handleKeyswitchEvent() with an optional caller argument of type Plugin*:

handleKeyswitchEvent(event, this);

Both the parameters to handleKeyswitchEvent() are passed by reference, so every plugin that comes before the caller has already seen that event, and will ignore it. The caller will also ignore it, but will change the caller parameter to nullptr, causing subsequent plugins to process the event normally.

In the case of entirely new events (e.g. Macros sending a sequence of key events), handleKeyswitchEvent() would get called without the second parameter, and all plugins would handle it normally. It might turn out that this isn’t really useful, though; even in my example, it’s likely that all the plugins should ignore the event. And maybe it would be useful to have a bogus Plugin pointer value (0xFFFF?) that could be used to force all the plugins to ignore the event.

So far, Kaleidoglyph doesn’t have any concept of an “injected” event as different from a normal event, but I’ve got six bits unused in the KeyswitchState structure that may well come in handy for that.


(Jesse) #13

You shouldn’t feel too bad about that. Kaleidoscope is code. All code is imperfect. The problems you describe are real, even if I’m not sure I’m sold on your proposed solution.

(FWIW, I’ve never been a huge fan of the INJECTED system. It’s a workaround for issues, not a desirable feature.)


(Noseglasses) #14

This is really an exciting discussion!

After I spend so much (exhausting) time trying to find a way to work around or adapt to the given event handling/keyboard report system, it is a lot of fun thinking/dreaming about an implementation that would have made things really easy :grinning:

Let’s use our experience to make it easier for future generations!


Let me start with a scenario that I found difficult to solve:
Can Qukeys emit a tapped OneShot modifier when the Qukey is tapped? If so, how does it work in (roll over) cases where the Qukey is toggled off in the same cycle where another printable key is toggled on that is supposed to be affected by the OneShot modifier?

Using a copy of the previous report works well for printable keys and modifiers.

For other keys, like those that control plugins, I expect it to fail (if there are no additional workarounds). It might or might not work, depending on the way other plugins are implemented. This assumption is based on an analysis of what the OneShot plugin does (I did not try other plugins yet). If tapped, a OneShot modifier key requires the OneShot post-clear loop hook to run in order to add the actual modifier to the next report.

Because of this and because I am wary about what other plugins might do, I decided, to stay on the safe side and call the pre-clear loop hooks, send key-report and post-clear loop hooks after every time I flush a key event via handleKeyswitchEvent.

I am well aware that this solution is not only overkill but it additionally comes with some potential side effects. One major drawback is that all plugins’ loop hooks are executed, including those that are unrelated to key processing, like e.g. LED effects. This might confuse plugins that do assumptions about loop timing, i.e. those that operate under the assumption that their loop handler is called once per cycle and that a cycle has a typical average duration. With Kaleidoscope-Papageno, LED animations are, thus, likely to behave jerky while typing.

Your reworked set of hooks would nicely solve some or even all of this. LED mode plugins could use the pre-scan hook for everything that needs to be done once per cycle, including animations. The pre- and post-report hooks might then be used for key report related stuff exclusively.

I really like this approach :+1: It nicely replaces the problematic INJECTED mechanism and saves some permanently used RAM that would otherwise be needed for multiple plugins’ variables like flushing_queue.
[FWIW, I think checkCaller will even work without being virtual.]

I am not sure about the setting-to-nullptr part and the logic behind it. With Papageno I only would want to process events that have not been flushed (caller == nullptr).

With your suggested approach, when Qukeys would flush an event, thereby tagging it with its this pointer, event handler hooks of plugins that were listed before Qukeys in the sketch would ignore the event. Am I right? How does that help? And how does it solve the problems in a scenario where two plugins A and B both run an event queue, and events are prone to bounce back and forth - which reminds me on the closed loop letter chute from the movie Brazil :grinning:.
Could you, maybe, provide an example of the event-flow in such an A-B scenario?

Although I did not (yet) completely understand what you are up to, the general idea of the caller pointer is great.

I had bad experiences with requiring users to add boiler plate code to their implementations. If I would be forced to do it today, I would wrap it up in a macro, so I (i.e. the interface maintainer) could change it later or could even make it a NOOP if necessary.

But as the keyswitch event handler hooks are called by keyswitchEventHooks, the task of checkCaller(...) might be taken over by keyswitchEventHooks. That way, the boiler plate code could be avoided.

bool keyswitchEventHooks(KeyswitchEvent& event, KeyArray& active_keys, Plugin*& caller) {
   for(plugin: plugins) {
      if(caller) {
         if(&plugin == caller)
            caller = nullptr;
         continue; 
      }
      ...
   }
   return true;
}

Does that make sense? Again, I’m just brainstorming here.

You mean, to skip the event handler hooks and just use the rest of Controller::handleKeyswitchEvent? That sounds reasonable.

Isn’t an event with caller != nullptr automatically an “injected” event?


(Noseglasses) #15

I will keep you updated about hanging keys issues once I play with other plugins that are related to key processing.

After I typed for several hours with my personal setup on the M01 I did not experience any hanging keys.
Typing really feels great :surfing_man:.
My ErgoDox is already casting jealous glances at me :eyes:


(Michael Richters) #16

I’m pretty sure that’s not true. OneShot calls activateOneShot(idx) in its event handler, which ultimately results in a call to handleKeyswitchEvent(). However…

To my knowledge, I haven’t explicitly test this, but I’m pretty sure it works correctly, and the OneShot modifier will get applied to the other key. The OneShot injected event has a row, col of UNKNOWN_KEYSWITCH_LOCATION, and the first thing the Qukeys event handler does is check for events with out-of-bounds row or col values (and let them pass through unchanged). Qukeys only cares about physical keypresses, so it’s fine to ignore those events. It does mean that there could be a problem if a plugin calls handleKeyswitchEvent() with valid row & col values, but in that case, a physical keypress might be intended, so maybe the result will be what’s expected, anyway.

I don’t know if this would work for Papageno, as well, but I’m guessing it would. (I also think both Qukeys and Papageno probably work best when they are the first plugins in the event-handling order.)


(Noseglasses) #17

Exactly. That’s something I’m quite unhappy about. What if a user wants to combine “queueing” plugins?


(Michael Richters) #18

I’ve done a lot of general thinking about this, but mostly with respect to Qukeys as one of the two. I’m fairly certain Qukeys will work with another queueing plugin as long as it goes first. Qukeys never aborts key events; it only delays them. And, for all intents and purposes, it also preserves the order of those events. Last, it also effectively defers keymap lookups for those events until they are released from the queue (mostly so that layer changes take effect at the right point in the sequence). Qukeys could follow another plugin that makes the same guarantees, though timeouts could still be an issue.

To the specific question of Qukeys/Papageno compatibility, the only issue I think there might be is one of timing. If a Papageno configuration depends on specific timing of key events on a small scale (e.g. t < 200ms), there could be issues with using both together. I’d probably have a better idea about the potential problems if I had any specific use cases for Papageno, but I haven’t thought of any yet that aren’t covered by other plugins (or a combination thereof).

The timing issues could be addressed by having the controller record a timestamp for each event and passing that to the event handlers, but I’m not sure it’s worth the trouble (and the extra data storage), and it might not solve all of the timing problems (if something is delayed long enough by one plugin, the next might have already passed its timeout, making things worse instead of better).


(Noseglasses) #19

And you are damn right here, its actually not true. My assumption about the OneShot plugin was wrong. When I was playing with different solutions to get Papageno working with stock Kaleidoscope, I could fix it by adding parts of the main loop, including the pre- and post-clear hooks. But that was before I switched to sending reports after every event. Now, after I remove the additional calls to pre- and post-clear hooks on flush, it still works :beer: So it was useful in the end, to bring up the subject.

But even though I was wrong with respect to the OneShot plugin, I still suspect that there might be plugins that perform specific action in their pre-clear hooks that might be based on information gained during the key event handlers’ execution, even though I cannot think of an application right now. If necessary, I can still return to calling the loop hooks after flushing.

I am quite sure that all those assumptions hold for Papageno as well. An Papageno does not lookup keycodes for flushed keys but passes them with UNKNOWN_KEYSWITCH_LOCATION to the event handling system, which will then deal with the keycode lookup.

No, the only thing it does is to measure the time that elapsed since the last call of the key event handler. It it exceeds a timeout, the queue is flushed. A timeout < 200 ms is not useful in my experience. One could also replace the timeout with pressing a user defined escape key, like one shot escape does.

Agreed that some of the services provided by Papageno are (partially) covered by other plugins, among those TapDance(s), leader sequences and chording. But those are the ones I added when I was almost finished with Papageno. Their sole purpose is to not require mixing plugins, which would potentially mean a waste of resources. The leader sequences are actually more general as they are actually arbitrary key sequences that are assigned actions. The only differences are that they are directly translated from keyword strings to the search tree.

The most prominent feature - neither available in QMK, nor in Kaleidoscope - are Papageno’s key sequences (I call them single note lines as I tend to use analogies with music terminology in Papageno’s documentation). Those are the first feature I added and the reason I developed Papageno in the first place. They combine the idea of tap dances with leader keys. You can define any arbitrary sequence of keys that trigger an event.

I use key sequences and tap dances to excessively load my thumb cluster with commands. I figured that it can be faster and less tiring to use gestures with multiple keys hit by different fingers than repeated tapping the same key.

If I e.g. type cmd->alt on the Model01, it triggers enter. Do I type alt->bksp, it triggers delete, do I type alt->alt, it triggers shift+tab. And so on… I try to do as much work as I can with the thumbs [The reason why I ended up with ergonomic and programmable keyboards is that I kind of wrecked my pinkies]. Even if the M01 has more reachable thumb keys than other programmable keyboards, in average I still have to assign more than two function per key.

I figured that my muscle memory is very fast in memorizing those multi-key sequences. On the ErgoDox I trigger around eight different commands (additional to the keymap meaning of the keys) with the four reachable thumb keys by combining multi-key sequences and ordinary tap dances.

I already mentioned that it is more efficient to let Papageno do the work of other plugins like TapDance if Papageno’s core functionality is required. This is even more resource efficient as Papageno comes with its own compiler Glockenspiel that compiles readable definitions of key sequences, clusters, chords, into highly efficient C/C++ code. It is thus very easy to define gestures and to keep track of them. The generated code represents Papageno’s internal data structures. No dynamic allocation is required during firmware execution and all data structures in RAM have only the size that is absolutely necessary. Other plugins limit the number of keys/functions/macros available and must allocate static memory to guarantee this amount. So there’s always a little waste of memory.

Here’s a gist of my current Model01 sketch. From line 170 on you find the Papageno/Glockenspiel definitions. Here is the C/C++ code the Glockenspiel compiler generates.

– End of the commercial :wink:


Recently, I am thinking about getting back to add modifiers to the home row. With DualUse this approach failed when I tried it on the ErgoDox, half a year ago. But now that there’s Qukeys on the M01, I probably will give the idea another try. By then I will definitely see how Papageno and Qukeys get along with each other.

As long as there are no Qukeys’ key positions registered as part of Papageno patterns, there shouldn’t be any negative interaction expected.


(Michael Richters) #20

I don’t think so, based on your use-case description. One guarantee that Qukeys makes, but Papageno does not appear to, is that every keyswitch event eventually gets passed through to the next plugin (and, for human-detectable purposes, preserves the order of those events); it’s just that some of those events might be delayed, and the resulting Key values can vary (but row & column values are intact). Unless I misunderstand Papageno, it doesn’t make this guarantee if it finds a matching event sequence, but instead aborts those matching events and replaces them with some other sequence/combination of events.

My best guess is that things should work well with Qukeys first, then Papageno, even if some of the Papageno sequences include positions defined as qukeys. It’s certainly possible that complications could arise, of course, though. With the order reversed, Qukeys might get a rapid sequence of events that was actually more spread out in time on the physical switches. Most of the time, this shouldn’t matter, but it could make it difficult to use the release delay feature.