Managing core and contributed Kaleidoscope plugins

I think everybody is right. (Though @algernon, there’s enough plugin traffic that adding a category is pretty easy. I’ve been wanting to split ‘programming’ up a bit anyway. There’s “user sketches”, “kaleidoscope core” and “plugins” and they’re all pretty different.)

It’s not wrong to have a place for people to discuss plugins, with a thread per plugin. I think that the canonical plugin list being on the wiki makes sense. But I totally see the need for people to have threads to discuss individual plugins. (Especially the user-written ones that the Kaleidoscope authors don’t support or know about.)

1 Like

thread per plugin sounds perfect! I was imagining an announcement only category, with discussion squelched, and that sounded wrong. :slight_smile:

I would l like to see a plugin category like we have a programming category. I think these things are related but they are not the same thing. It is nice to be able to search a specific category when looking for something and I think that the programming category would have more noise to sift through.

I would love to see a plugin manager like Plug for vim or even a full blown package manager that people can add plugins and dependencies too. Something like hex, rubygems, or npm.

Jesse created it here:

https://community.keyboard.io/c/programming/Discuss-Kaleidoscope-Plugins-one-thread-per-plugin

1 Like

One of my plans is to do this within the context of Chrysalis (and also provide a CLI tool for those who do not want to use a GUI). Details are TBD, we’re not there yet.

2 Likes

Does such a plugin page in the Wiki exist by now? I just read this topic over. The wish for some sort of plugin manager was expressed several times.

Wouldn’t it be a good idea to collect the users informations about their plugins in a data format that could both be rendered as a table on a wiki page and parsed/read by a future plugin manager?

Are there any ideas for a suitable format, platform, etc? Of course it would be best if users could generate new entries by themselves. I find it important to define the information that describes a plugin (name, description, url, whatelse?).

Names might become a problem once there are many user plugins around. The likelyhood of naming-conflicts grows with the total number.
Any ideas about a way to introduce namespaces, domains, etc.?

Here is another topic where the naming of plugins is briefly discussed

A closely-related question about third-party (or, more specifically non-Arduino-Boards) plugins is where one should put them when building custom firmware. In the libraries directory, along with the official ones? Symlink’d there? As submodules of one’s own firmware sketch? In a separate library directory? Does the build system already have a place for them?

If there isn’t already a system for this, let’s make one.

“libraries” in the Arduino sketch directory is traditional (~/Arduino/libraries or ~/Documents/Arduino/libraries depending on the OS.)

I’d thought that you could also drop them inside lib/ in your sketch directory. (Though now I wonder if that’s libraries/ inside your sketch dir.) If the IDE supports one of those and the commandline build doesn’t, that’s a bug and something we should fix.

1 Like

I’ll experiment when I get a chance. If $SKETCH_DIR/lib or equivalent works, I think it would make sense to include third-party plugins as git submodules, but that’s probably not a great solution for people who are not git-adept.

Just something I threw together here:

It has not been maintained at all.

1 Like

Nice. That will be very useful to newbies.

Maybe we can add some sort of table for user plugins? Is it possible to use a data format that is machine parsible to generate markdown tables?

I started a new topic

Well, the first line of rules.mk is:

EXTRA_BUILDER_ARGS="-libraries ."

…so it looks like the anything added to the sketch directory itself would get found, but I’m not 100% certain.

Yup. The big question that should get answered is what the precedence order is, since Arduino has…surprised us on that front before. If you put something in the sketch directory and it’s also in Arduino-Boards, what happens? And does the coommandline behave the same as the IDE?

Arduino has, we believe, changed the build order on us before. The current documentation for search path ordering is buried in the commentary on some pull requests that proposed changing it.

I suspect the right way to figure this out is to enumerate all the possible places a library might get placed and write a script to compile a sketch with different versions of the library in each place, testing out every possible permutation. That’d let us autogenerate truthful docs and also catch potential changes.

1 Like

I started to explore how to best bundle plugin libraries, and the short version is that downloadable zip/tar files are the most straightforward way.

A work in progress still, but you can find my notes here, along with proposed categories (same as @jesse proposed before, with the addition of an Experimental category), and current plugins proposed for each.

1 Like

I’m wondering how this plan will affect a developer’s setup. Right now, I’ve got an Arduino-Boards clone symlinked to $SKETCHBOOK_DIR/hardware/keyboardio/avr, with multiple remotes for some of the submodules, as well as Arduino-Boards itself. Separate from that, I have my own Model01-Firmware fork cloned elsewhere, with third-party plugins set up as submodules from the top level so the build system will find them when I run make from my Model01-Firmware/ dir (not the one under Arduino-Boards; I just use that one if I want to send a PR to keyboardio/Model01-Firmware).

Basically, I’m interested in knowing how you see the restructuring affecting how we organize and build in development, or if the bundling you’re discussing is unrelated entirely.

I imagine Arduino-Boards will remain somewhat similar to how it looks now, a combination of Core+Extra+Experimental. So I suppose nothing will change there.

If not using Arduino-Boards, then there’ll be a small script or a wiki page that explains how to merge the different repos locally…

That just gave me another idea: what if we had one repo, but with branches? And master would be a merge of all of them. (Not including third-party stuff, that’d still be a separate thing). Releases can be tagged on branches, zip files generated, and so on.

Not sure if this’d be better, or more confusing.

I’m not sure I follow. Do you mean some branches of Arduino-Boards would have some of the submodules, but not others? If that’s what you mean, I think I’d prefer not to do it that way; the reason I put third-party plugin clones under my own Model01-Firmware fork is so I wouldn’t have to deal with them cluttering up the output of git status in Arduino-Boards. I suppose .gitignore could take care of that problem, though.

I’d use a separate repo, that has nothing but the submodules (and Arduino-Boards would include it as a submodule). Lets call that repo Plugins. It would have a plugins/core, plugins/extra, and a plugins/experimental branch, each with their own set of plugins. Then, it would have master, which would be the merge of all that. As a developer, you’d work with master most of the time, so no cluttered git status. The rare times one would want to check out any of the other branches, is to move plugins between the repos, or update one, or something along those lines. Things that should be rare enough. (We could automate updates one way or the other, or delegate that to the set maintainers, and just open an issue if an update is desirable). Or we could track tagged plugin releases, kinda halfway between automation and manual requests.

In any case, what I wish to end up with, is developers having to care very little about the branches or repos.

2 Likes