Chrysalis - layout & keyboard settings editor

Programming
1

I reached another milestone recently, and am now able to have bi-directional communication with the keyboard. At the moment, there is a small Python tool, that presents a REPL one can use to talk with the keyboard:

focus-repl.png1411×624 34.8 KB

This is obviously not friendly, so I’m also working on a cross-platform application that will present a nice GUI to fiddle with your keyboard. It’s in a very early stage so far (read: I can fire up an Electron app, and verify from the Chrome developer console that I can talk to the keyboard via USB Serial - nothing more), so I’m trying to collect ideas, to see what people would expect from an application that is supposed to help them create their own layout, tweak settings, flash firmware updates, and so on.

I want it to look and feel simple, even if lots of neat things are in it. Before I go into a lot of detail, I’d like to ask You: what would you want from this application?

At a minimum, there will be three things provided: a way to talk with the keyboard directly (kind of a console, or REPL access, for advanced uses), a way to flash a pre-compiled firmware HEX, and a layout editor, with support for layers & everything.

I have plenty more ideas, but would love to hear Yours first =)

5 Likes
2

One idea that I’ve always thought would be interesting is a built in password manager but that might require a flash card to store enough encrypted data for it to be useful.

3

While a password manager is an interesting idea, there are a number of obstacles:

  • low storage on the device, both for data, and for program code
  • securing the keyboard->host transfer of sensitive data, so that one can’t steal your keyboard, and extract all your passwords.
    • We either do decryption on the keyboard, but that’s a deep rabbit hole: we have to lock things down hard, to prevent easy stealing of passwords.
    • Or we just use the keyboard for storage, but that’s not much more useful than using an USB stick - perhaps a bit more obscure, but not more secure, or useful.

What could be useful - and possible with the planned application! - is to have a macro, that reads certain parts of your EEPROM, and you’d hide master password for your KeePassX database there. The way I imagine it, is that you’d have, say, 64 bytes for the master password. Now, having it in plain-text in EEPROM is wrong, so… you could store the first N bytes XOR’ed by X, then the next M bytes rotated Y to the right, and skip every Zth byte, and so on. So to unlock the master password, you’d first need to tell the keyboard how to decrypt it. We can provide a special command you can enter on the console, and then your macro that reads from EEPROM, and types the password, would start working. For updating, we’d send the “encrypted” bytes, and have a tool that can generate that - perhaps even provide a way to extend the app with plugins… now that’s a good idea I have not thought of before!

Not the most secure solution, but perhaps not too trivial, either. But then I’m not a cryptographer, so the above scheme may be easily brute-forceable.

But yeah, something like this can be done, and the app can provide some support for it. I don’t want to build a password manager, though. There are plenty of those around, by people who understand the field much better than I do :slight_smile:

1 Like
4

I agree that it’s tricky, for one the password database would have to be encrypted and running decryption routines on the keyboard itself is probably not feasible.

The advantage of having the database on the keyboard rather than on the computer is that even if someone breaks into your computer they still can’t access your password database and can only possible intercept passwords in flight if they have access to the bus or protected memory. Of course if the storage on the keyboard is accessible to your computer this advantage is negated.

It’s just a thought that’s been bumbling around in my head for a while, not sure there is any practical way to implement.

1 Like
5

The way things work now, if the keyboard is plugged in, and the plugin that makes bi-directional communication possible is installed, everyone who can talk to the serial port, will have full access to your keyboard. So the secrets therein have to be protected, and that’s the tricky part.

And I’m going to assume that whoever has physical access to the keyboard, can re-plug it into another computer where they have admin access. So… yeah… with more powerful hardware, and a lot more space, we could build something to offload secrets onto the keyboard, but we’re not there yet.

1 Like
6

The ability to set LED “themes” of some sort with a GUI would be handy

2 Likes
7

There are a number of options here, with different implications:

  • Set a LED theme, from a list of available ones included with the firmware.
    • This limits you to the themes that come with the firmware, obviously.
    • On the flip side, it requires at most a single byte of EEPROM to persist it, so the theme remains in effect when you reboot the keyboard.
    • You can even opt to not save it, and just set it once.
  • Set a fully custom LED theme
    • If not saved onto the keyboard, it will get lost with a reboot, or whenever you change the LEDs in any way.
    • If persisted, it either eats a lot of EEPROM (when not using a palette), or requires setting up a palette first. (But still eats 64 bytes + palette then)

If selecting a theme from pre-built ones, do you want to be able to preview it within the app?

1 Like
8

Previewing might be neat, but probably not vital, I would think.

As to how it’s saving, I’d be okay with it not persisting custom themes & just having them saved in the app (so it’d still be a simple matter to restore after reboot or other change).

1 Like
9

This is the very definition of “roll your own encryption”, and the response is always DON’T.

If you’re going to store a secret in a removable device, the removable device itself must be secured to at least the same level, otherwise you’ve just opened a back door. What’s to stop someone reading your “decryption” algorithm from the EEPROM?

There are secret storage systems that are hardened against logical and physical attack, such as smartcards. If you want to use one, go ahead. Building one into a keyboard is a cool idea, but the model01 hardware as currently designed is not a secure storage device and we shouldn’t be encouraging users to treat it as such.

4 Likes
10

When I first heard about the LED’s Keyboardio, I though integration with IFTTT would be spectacular.

4 Likes
11

so many cool ideas…

12

My addition to the list of ideas:
The ability to have an on-screen keyboard in the shape of the Model 01 that shows which keys I press as I type (and updates to display the correct layer as modifiers are pressed). If this description doesn’t make sense, think of the Keyboard Viewer on macOS.

I think this would be a great addition to learning new layouts as using a pre-existing solution (for example, the aforementioned macOS Keyboard Viewer) is going to have the keys in the wrong location and would almost certainly have no concept of the Model 01’s layers.

I also second @andrewg when he says [quote=“andrewg, post:9, topic:253”]
This is the very definition of “roll your own encryption”, and the response is always DON’T.
[/quote]
Seriously, don’t! The Model 01 (from what I can see) is not designed to be a cryptographically secure device, it is far too late in the design process to think about making it one, and while a hardware password manager is a neat idea — it is a neat idea for another product (it’s a feeping creaturism).

As for the issue of LED themes, how about a hybrid approach:

  1. A unique ID gets saved to the keyboard.
  2. On keyboard boot up, the keyboard requests the OS driver to hand it the LED theme matching the given ID.

This would then allow the “storing” of 252 themes in a single byte in the EEPROM (themes 0 and 1 can be built-in default themes, with theme 2 being a “loading spinner” theme which would give visible feedback on where the keyboard is in the process of communicating with the OS driver). The downside to this idea is a custom LED theme will look like the “loading spinner” theme until the OS driver instantiates and responds to the polling request for the custom theme.

Yes, please! And not just pre-built ones, any LED theme should be previewable in the app.

3 Likes
13

I agree with the above from @cal and @andrewg; I would be strongly against trying to shoehorn a secure password manager in to the firmware.

1 Like
14

It looks like there’s already a keyboard with ifttt integration: Das Keyboard 5Q

Here are some examples

What would we need to do to get this for the Keyboard.io?

1 Like
15

My personal take on IFTTT: ugh :angry:

If it must exist for the Model 01, let it be in a plugin so I can banish it to the circle of hell reserved for cloud marketer-speak (if others want death by a thousand paper cuts, that is their choice — I want nothing to do with it).

Cloud connectivity is a non-starter for me to the point that if a product has any form of requirements on cloud anything, then I will not buy said product.

4 Likes
16

There’s no requirement for Keyboardio to use the cloud. Plugins for a bi-directional communication tool is an excellent idea.

I suggested IFTTT because it’s an obvious choice to integrate with as it’s something a lot of people are familiar with and gives an idea what could be done. If there’s better, more open alternatives to IFTTT, then I would like to hear about them.

A boring use case I can think of is a notification from a friend on a social media site, (maybe flashing a key matching the friend’s initial,) the Any Key turns green and clicking on it opens that app. Or another could be a notification of a failing build script and now the Any Key could open the log to the failing script.

1 Like
17

This should be possible, there already is a Keylogger plugin, just have to teach Chrysalis (likely via a plugin, too) to do something with that information.

The keyboard doesn’t even need to ask: Chrysalis can push it down when it starts up, and then all we have to do is make sure it starts with the OS. Kind of like an agent of some sort :slight_smile:

No need for a “spinner” theme then, either. Pushing down a LED effect is a couple of milliseconds, so your keyboard will just stay with LEDs off, until Chrysalis starts up.

The harder part of that is that currently the LED themes can’t tell the host their names, nor how they look - the former can be fixed with reasonable ease, the latter - not so. This means that if we want Chrysalis to preview built-in themes, we need to pretty much re-code them in it, too. Or at least provide a static image (this would be a reasonable compromise, in my opinion).

Oh dear.

** algernon rolls a saving throw* * - phew, crisis averted, I won’t be ranting about the 5Q!

The keyboard itself can’t connect to the internet, and there is no reasonable way to add that functionality. I don’t think anyone wants to, either. Chrysalis could act as a bridge, but by default, it will not. Anyone’s free to write a plugin to do that, of course, but you will never need to install it, if you don’t want to (I wouldn’t, either, except perhaps to prove a point).

5 Likes
18

To make a few things clear, here are some of my plans for Chrysalis:

  • I want it to be similar in spirit to Kaleidoscope (the firmware that powers the Model 01): a small, lean core, and a horde of plugins.
  • There will be very little that will need to be mandatory. So much so, that you will be able to disable everything, including bi-directional communication in the firmware, and use Chrysalis as a nice looking flashing application.

The plugins then, would add the bulk of the functionality:

  • Flash compiled firmware.
  • Build a firmware from sources (will need Arduino IDE installed, and a bit of configuration).
  • Provide a REPL (read-eval-print loop) for advanced users.
  • A keyboard layout editor & uploader (to be stored in EEPROM).
  • LED effect editor. With previews & whatnot.
  • One to tell the keyboard the OS of the host, so it doesn’t have to guess.
  • Tell the keyboard the active program, so it can switch layers, or do something else.
4 Likes
19

I could live with that for built-in themes. :slight_smile:
Or, for extra credit, they could be APNGs so the user can get a sense of what a given theme does… :stuck_out_tongue:

1 Like
20

Over the weekend, while finishing up the current Diablo3 season (using a Shortcut prototype, powered by Kaleidoscope; of course), my unconscious spent some time thinking about Chrysalis and its architecture, while the rest of my brain conveniently switched off while hunting demons. While I will present a number of ideas here, unfortunately, there will be no pictures, no mockups, or anything like that, so we’ll have to use our imagination. And I hope I can describe the ideas good enough to be imaginable. Mind you, some details may end up being a bit technical, this is pretty much an unedited brain dump now.

Starting up

So, we want to support different hardware with the application, so it should try to discover the device as soon as possible: it would scan the USB devices, and if it sees something it recognises, it would try to connect to it via Serial, and issue a version command to figure out the details. For the time being, until we figure out a way to do plugins, this would be built into the application, just like it all started with Kaleidoscope, too. If it finds no device, it would present a way to manually select one.

Home screen

Once we have a device, it’s time to load up the starting screen! Up until now, the user was presented with a logo, and a progress bar. With the starting screen, we’d still have the logo, the keyboard manufacturer + product under it, and then a set of boxes with which we can reach the various functions of the application. When selecting one of the functions, a new screen loads. Each of these would have a little Home (or Back) icon in the top left corner, to allow navigating back to the starting screen. I think this is a better approach than the permanent sidebar, that takes up a considerable amount of space, yet, it is unlikely to see frequent use.

Layout editor

One of the most used parts, and the first one we will likely develop, is the layout editor. The latest backer update showed a preview of it:

https://ksr-ugc.imgix.net/assets/015/983/708/3bb9653e3006795de1e20bb1d49c9fb5_original.gif?w=639&fit=max&v=1490319942&auto=format&gif-q=50&q=92&s=ffd901a100c8ffce0122a08e166db518

You basically click around, or otherwise navigate the keyboard, and once you found a key you want to replace, click it. Then you are presented with a menu - either a popup, or one in a fixed place on the right - where you can configure that particular key. What I’m not sure about, is how we can discover features of the firmware? Like, if one-shots are not compiled in, we likely should not present those as possible key actions, either. I was thinking, that we could have a Features plugin (or bake this into Kaleidoscope-Ranges, which sort of does something related), which would keep a set of flags, whether a particular plugin is enabled or not. Plugins could then poke their flag on when they are included and used, and Chrysalis could ask the keyboard for a list of features, and get a bit mask, which it can then translate back to actual features. This would not take much space on the keyboard, and still be reasonably useful. The downside is that we have a central registry of features now, and both the firmware-side list, and Chrysalis must be in sync for this to be useful. But then, we already have something like that with Kaleidoscope-Ranges (not going into detail what that plugin is for, feel free to ask me if you are curious).

Profiles and devices

For different keyboards, a different layout would load, and they’d be saved in different “profiles”. Question is, how do we figure out the configuration on the device? As in, if you have two Model 01s, plug one in, configure it, then unplug it and plug in the other, how will Chrysalis know it is not the same device? We could hash the EEPROM contents, I suppose. Every unconfigured keyboard would look the same then, but once you configure one, that one would be different.

And this leads to a next issue: what if multiple Kaleidoscope-powered devices are plugged in at the same time? I do that all the time: both the Model 01 and the Shortcut are plugged in. We’ll need a device chooser, I suppose. Could reuse it in the initial discovery too, when no device, or multiple devices are found.

Layout editor, take two

But, back to the layout editor! When you configure an action, it would internally store it in a structured fashion, one that is easy to work with, to export, and so on. In a format independent of the codes used under the hood. When applying the layout, Chrysalis would do so via a shared “compiler” that translates its internal format to keycodes Kaleidoscope works with. We’ll have to come up with an extensible internal representation. Doing the compiling/transformation afterwards is a walk in the park in comparison.

LED theme editor

The LED theme editor would look somewhat similar to the layout editor, except we’d have a permanent palette on the right (itself editable, of course). You click a color, then click all the keys you want to apply that color to, and done. Repeat until you have a theme you like. In the future, we may want to allow selecting a group of keys to apply colors to. But perhaps not in the initial version.

Below the palette, would be the color chooser: the selected color could be altered by selecting a different one from the chooser.

Firmware flasher

Now, this is simple. Select file, hit upload, wait, done. I don’t think we need more than that?

REPL

This is something I want to have a lot of polish too, because I will use this the most. What I have in mind, is not only a very dumb thing that I can use to send and receive data to the keyboard, by speaking its low-level protocol (though that should be an option too!), but something higher level.

It would turn the keymap.map dump (which is a series of 16-bit integers) into a proper layout, one that looks the same as if it were on the layout editor. Even better, if it could load up, and embed the editor, that would be best.

Same goes for the colormap and the palette, and possibly other things too: Chrysalis should apply as much formatting and presentation, as is needed to represent a particular thing to the user.

However, this will only work well, if we have a reasonably modular architecture, so initially, the REPL may do some formatting, but it will not embed any other parts of the application yet.

Closing thoughts

This is all I had in mind for now. In the next few days, I’ll try to come up with a few mockups, to see if those help. Other than that, I think that pondering about the plugin architecture at this point is not necessary. It would take too much time and effort to do so, and shipping something is more important. We can - and will - rewrite parts when necessary. I did that with Kaleidoscope plugins, some of them have been rewritten more than three times. While it feels sad, and perhaps a waste to do that, these things are small enough to be rewritten in a week or so. I hope Chrysalis will end up similarly.

6 Likes