What do people want from the firmware?

Programming
1

I’m working on a set of libraries with which to extend the Keyboardio firmware with, and it looks like it will satisfy my own needs very, very soon now. But chances are, I won’t be able to play with the hardware yet by that point, so to keep myself occupied during the - hopefully - final stretch of the wait, I thought I’ll ask all of you people here, for ideas about features that could be useful in the firmware. These will be additional libraries, not part of the core firmware itself, so don’t hold your imagination back!

Ideas are listed below, for progress, check the milestone on GitHub.

  • A keylogger, and a tool to make pretty heatmaps. The tool is not ready yet, only the keylogger part.
  • Magic combo framework: Press any combination of keys at the same time, and run custom actions. This is a building block for other things, primarily for plugin authors, not end-users.
  • One-shot keys: which all support holding and double-tapping for sticky behaviour. Only one-shot modifiers are implemented so far.
  • Generic macros: Nothing really useful in itself, just a framework for making it possible to build macros with. Not very user-friendly just yet.
  • Run-time toggleable one-shots: One-shot modifiers that start out as normal keys, but if a magic sequence is tapped, the modifiers turn into one-shots. (Needs Macro, OneShotMods, and MagicCombo)
  • Tap-dance keys: different functionality, depending on how many times the same key has been tapped without interruption.
  • Unicode input: A way to programmatically trigger the Unicode input system of he operating system. Works on Linux only so far, but OSX and Windows support is coming, too.
  • Dynamic macros: trigger recording, press a key to assign the macro to, tap other keys, and trigger stop. The macro is saved to EEPROM, and the assigned key will replay the recorded sequence.
  • Keymaps in EEPROM: Instead of having to reflash the keyboard every time one wants to change the keymap, provide a way to upload a keymap via Serial, and have it stored in EEPROM. This makes it much much easier to update the layout, for a non-programmer.
  • Leader keys, a’la Vim.
  • Symbolic unicode input: trigger the mode, enter a symbol name, and the keyboard will delete what you wrote and input the unicode symbol instead. For example, Fn+u, kiss would result in 😙 ending up on your screen. This would also make it possible to use macros to enter unicode stuff, so you could have a macro that says typeUnicode(Unicode::Lambda) in the code, and inputs λ when tapped.
  • Dual-use modifier/keyand layer/key pairs: Tap the key, and get a normal symbol, hold it, and get a modifier, or a temporary layer switch. Thanks to @james.nvc for the idea.
  • A diacritic key, to cycle through diacritics over the previously input symbol. Thanks to @michael for the idea.
  • Built-in heatmap with the LEDs, thanks to @merlin for the idea.
  • One-shot layers.
  • Space Cadet Shift.
  • Examples and helper libraries that combine some of the other plugins, presented in a friendlier way.
  • Documentation.
  • A way to communicate with the keyboard, for editor integration and such. Idea from @amos.
  • An easy way to shift-invert keys. Idea from @merlin.
  • An easy way to remap the shifted symbol of any key. Idea from @merlin.

As of this morning (EU time), the official firmware has enough hooks to implement most - if not all - of the above, rendering my previous attempt at making an extensible keyboard library obsolete. So I’ll spend the weekend porting the features to the official firmware, and likely start implementing the missing ones. If you have any other ideas, please do let me know. The more the merrier, and no idea is too wild!

This will be a set of libraries, not a single one, so we’ll get to pick and choose what we like. And composing them looks simple enough so far:

That’s all there is to it, thanks to some clever wizardry going on behind the scenes.

Anyway, don’t hold your imagination back, I’d like to know even the wildest ideas you may have regarding the firmware, and I’ll see if it can be made reality.

Edit#1: Added dual-use keys and a keylogger.
Edit#2: Added diacritic key.
Edit#3: Added the built-in heatmap.
Edit#4: Added a link to the source code repo of the libraries.
Edit#5: Rearranged the list a bit, to make it more up-to-date.
Edit#6: Another rearrangement of the feature list.
Edit#7: Yet another rearrangement. Just list the ideas, not progress. Link to the GitHub milestone for progress.
Edit#8: Added the communication idea.
Edit#9: Added Space Cadet Shift.
Edit#10: Added the shift-invert and the shift-replace ideas.

9 Likes
2

Is there a way to have a key act as a modifier when held & another key is pressed, but send a different key when pressed by itself? e.g. control key is like a control when holding & pressing another key, but tapping it sends Escape.

I’m currently using xcape on Linux & Karabiner on OS X to do this, but it’d be much nicer if the keyboard itself could do this.

2 Likes
3

It is possible, and I’ll add it to the list, because it’s not even hard to implement: you press a Control/ESC key, it does nothing, but starts a timer. If you release it before the timer ends, you get an ESC, because it is considered a tap. If not, you get a Control. If you press any other key before the timer ends, you get both Control and the key.

Same can be done for layers, so you can have modifier/key and layer/key pairs too.

Thanks for the reminder!

2 Likes
4

Another thing I’ll add: a keylogger, and a tool to generate pretty heatmaps out of the logs.

5 Likes
5

Having Keylogger functionality as a plugin is great. But I want to be very, very careful about it for the obvious privacy reasons.

2 Likes
6

Indeed. That will be its own separate library, with big, bold warning letters and the like.

7

Thank you @algernon for all the great ideas and effort you are already putting into the extension libraries. Since you asked: I am looking for a library to implement a “diacritics” key.

The idea is, that instead of typing a dead key followed by the letter you want the diacritic to appear you first type the letter followed by a fixed “diacritics” key that then steps through different diacritics the more often you type it.

For example you would press an ‘e’ followed by the diacritics key which would delete the e and type an é. If you would press it one more time it would delete the é and type an è and so on. Depending on the language you would have different maps for the diacritics key.

In German you would for example have o -> ö -> ô -> ó -> ò in French you would rather have o -> ó -> ò -> ô -> ö

The idea origins from ADNW - Optimierung Für Die Gerade Tastatur Mit Daumen-Shift.

3 Likes
8

An interesting idea! This will be possible with the help of tap-dance. I’ll not go into the gory details of what tap-dance is, but one thing it allows you to do, is run a custom function each time a tap-dance key is tapped. This custom function is what would do the diacritics magic in your case. It just has to remember the keycode of the previous key. Entirely doable, and I can provide a plugin that comes preconfigured to this, and all you need to do is select a language. (And of course, you’d be able to reach into the lower levels and add custom sequences, but that’s a bit more advanced.)

I’d try this approach too, to see how it differs from the way I currently input Hungarian accents.

For those interested, the way it would work, is roughly like this:

  • Install an event handler hook when you include the plugin in your Sketch.
  • If it sees a key that is not the diacritic key, it does nothing, and passes control over to the next handler, after storing the keycode for that particular key.
  • If it sees a diacritic key, it looks at the previous key, and decides what to do, based on the tap count, and the previous key.
  • Once it figures out what to input, it sends a backspace, and does some magic to input the diacritic key you want.

If you tap any key other than the diacritic key, the sequence is interrupted, and it resets the counter, so it can be used next time too.

The hardest thing here would be figuring out how to input the diacritics themselves, as that is OS, and OS-side layout dependent. So I suppose, I will have to make that part configurable too…

Once I get to implement this feature, I will likely ask for hints, to see how each of those symbols need to be entered.

1 Like
9

Thank you for your fast response. That sounds wonderful.

I would like to suggest thinking about a similar idea: A “compose” key. The idea is to type a sequence of characters which get replaced. The compose key might actually be “shift”+“diacritics”.

The difference to the diacritics key is, that the sequence of keys gets only replaced once:
“1/4” + “compose” -> ¼
"co" + “compose” -> ©
"L-" + “compose” -> £
ect.

In order to be able to have both functionalities you would have the firmware to remember the last n characters typed.

In case the compose key is pressed you need to check wether it is a valid compose sequence. If not, nothing happens. The buffer keeps its data. In case a valid compose sequence is detected the matching key-sequence will be sent and the buffer will be erased. I imagine that the challenging bit will be to get the character Sequence in the buffer right, because you will actually have to keep track of backspace, modifier-Keys and maybe even of caret moving actions. Maybe it’s best to start of with a functionality that only allows you to press the keys in strict sequence for it to work. Though the addition of deleting the last character by pressing “backspace” would be great.

In case the diacritics-key is pressed we need to keep track of the number of presses of the diacritics-key and of the last character typed. The challenge I see here is to distinguish between lowercase and capital letters. Because “a” + “diacritics” -> “á”; “A” + “diacritics” -> “Á”. It sounds like you are already having a good idea of how to solve that.

Let me know if I can help. I could supply keycode-sequences for the mac and windows (I already researched a bit on input methods for both). I can also offer my help for beta-testing or code-reviewing.

1 Like
10

Compose-key is already implemented as an input method in all major OSes. There’s no need to duplicate that functionality here.

1 Like
11

The currently implemented compose-key is a dead-key. So you have to press “compose” + sequence. Right?

12

While this is certainly possible, it will have to rely on the OS side, and all you save is a dead key. Using layers for this purpose sounds like a better approach to me. Or, using leader keys, but that’s a dead key too, except a bit more… flexible, perhaps, than layers.

Or, as an alternative: it’s reasonably easy to do chord-based things: hold L and - together, and get £, co for ©, and so on. This too, can be implemented in at least two ways:

  1. Register the original keys, and delete back. This is simple to implement, but unreliable, because of OS side repeat.
  2. Delay registering the key, and register only after a timeout, or when a combination is found, or when no combo has been found.

Obviously, the second way is the way to go, in which case, to input ©, you’d hold c and o, and neither of them would be sent to the host, only ©. If you type c, release it, and tap o, then they would be sent in this order, because they are never pressed at the same time. If you hold c, for a little time, without pressing anything else, it would time out, and register c. So it should not impact normal typing too much, and is much easier to implement than remembering the last few keys, and more reliable to use, too.

But it still does not save much compared to the OS-side compose key, in my opinion. I can add it to my list, though, because it sounds interesting, and is a fun way to demonstrate the magic combo feature I already ported to KeyboardioFirmware. :slight_smile:

1 Like
13

If you are interested in a keyboard that implements all the diacritics and special characters/symbols for all languages using a Latin based alphabet (with dead keys and AltGr) see the Gelatin keyboard I posted in the Colemak thread.

While it would be very nice to not have to implement a custom language layer for each OS the firmware would have to send Unicode characters directly to the OS, I don’t know how hard this is or if that functionality will be available from day one?

2 Likes
14

I can see your point and think that @andrewg probably wanted to point out the same. That makes sense to me.

My priority is clearly on the diacritics key. It would certainly help me more then the “not-dead-compose” key since I’m using äöü a lot plus éáúóèàùòâôû and ñ every now and then. ^

I think it is a convenient way to write the letter followed by one additional key to modfiy the letter. Especially considering the space restrictions on a 64 key layout.

Reliability is certainly an issue, even with only deleting one character and sending a sequence to produce a new one.

1 Like
15

On the major OS-es, it is possible to enter unicode codepoints, up to 4-5 hex digits (OSX and Windows allow more, iirc, Linux is the most restricted here) - that should be enough for Latin based diacritics, I hope. It’s not the smoothest thing, but it works, and all you have to change in the firmware is which OS input method to use. This is something that will be part of the extra libraries I’m building, and should be ready by the time the first keyboards ship. Using unicode input will require some extra work, to set up the firmware, but we can build language packs, to make things easier.

16

If you need someone to do the unicode work I’m your girl, I’ve had to build layouts for all 3 major OSes for all the Latin based languages and I’d be quite comfortable adding/creating that in firmware (I’ve also done some firmware work and I’m proficient in C). I don’t think a single diacritics key would do the trick, you will need at least two or three and then a lot of special symbols.

See the following layout with two full diacritic keys (each has four different diacritics),four keys that have one diacritic each and then a bunch of special character keys using Alt/AltGr.

3 Likes
17

That is certainly an alternative to having a diacritics key. But you have to have an additional layer for the diacritics. That might be ok if you are using them seldom. Writing German texts you need “äöü” a lot. Therefore those letters are on layer one of the German iso-layout. Trying to find a new layout for a 64 key matrix keyboard, it is really inconvenient to have to use another three keys plus a diacritics layer / more dead keys for the other diacritics.

Therefore I find the idea of having a diacritics key very compelling. It saves you keys and layers and allows you to write in a very intuitive way. Even if it is a diacritic that you need very seldom.

There is OS specific methods for entering Unicodecharacters. On Windows it is quite easy because you only have to send “AltGr” + the code for the Unicode sign. On Mac you have to change the layout before you can do the same. That would probably mean to have to send different unicode sequences all the time, but I think I still would prefer that over installing additional software, as it is a standard layout that is available in every mac.

1 Like
18

I think you might be misunderstanding how the dead-key works, if I want a diacritic from that layout I’ll just press the diacritic key and then the letter I want (´ and a make á and ¨ and o make ö), granted some diacritics are less comfortable than others because you have to press either Shift and the diacritic dead-key, Alt and the diacritic key or possibly with the least frequent keys you have to press Shift, Alt and the diacritic dead-key.

With a composite key you will have to press the composite key itself, then a second key and a third and you have to remember which one composites with which one and there are thousands of permutations making it very hard for anyone to learn the layout.

I write regularly in English, Danish, Icelandic and German and rather than having to switch between all, learn all the special characters for programming on both my laptop and ErgoDox I found creating a single keyboard was much easiers.

1 Like
19

Ahh so you are still sending scan codes, just sending special sequences which result in unicode characters and depending on the OS it might or might not have a keyboard layout to interpret those into the Unicode Characters.

20

Mind if I ping you on GitHub, once I get to a stage with the library, where your help would be appreciated? There are a number of questions I will likely have, and quite a number of helper thingies I could use some help with. But I need to draft an outline first, and explain how I imagine to build this up. It’s a bit of a mess in my head right now.