Lately there has been some discussion in the forum about a collection/database/installer of/for user plugins.
I am pondering for a while now on how that could be done and I am interested to discuss the matter with you.
Let’s start with the information I would regard important about a user plugin:
name
description
required/supported Kaleidoscope version
git URL
current maintainer (directions for issue reports, PRs, etc.)
dependencies (other plugins)
compatibilities (can replace other plugins)
known incompatibilities with other plugins
I heard about ideas towards a future installer for user plugins. Maybe there are already ideas about that in the Crysalis team?
Such an installer would certainly required the above plugin information to
be machine readable/parsable. Thus, we would best have some sort of database
or at least use a pre-defined data format to store the given information.
At first I thought that we might want to collect the information directly in the
Kaleidoscope wiki. But that is probably not a good idea as, to my knowledge, markdown does not
provide an appropriate way to define data in a machine readable fashion.
Another important point is that such a solution would probably be too brittle. Every user could
accidentally render the information unparsable by introducing formatting errors when adding or modifying
information about her/his plugin.
An appropriate solution could be to have an independent
dedicated GitHub repo that basicaly stores some sort of yaml/xml/json or whatever
file that represents the user plugin database. This information would
then be machine readable and we would have travis tests that ensures the
well definedness of the data.
The GitHub repository would have a maintainer that would act as gatekeeper
to ensure that the necessary information for a plugin would be provided and the formatting is ok.
Users could, thus, update the information regarding their plugins via PRs.
Instead of yaml/xml/json/etc. we could also use html tables which are
also quite simple to parse. This would have the great advantage that
we could include the current version of the html file in the Kaleidoscope wiki (GitHub wiki markdown can handle embedded html, no?).
I agree that the wiki is probably not the right place to build this, though you could potentially create the wiki page as output from your database. GFM has limited html support, but there is a table building syntax. If you look at the wiki home page you will see an example. I first tried to build that table as a straight html block, but it wouldn’t render properly. I was able to create bulleted lists in the cells, though.
It would be nicer, though, to have a searchable directory with tags and pointers to the community thread for a given plugin, recommendations, notes, etc. Not sure where such a thing would be hosted, but I expect we could find someplace. This may be more work than you want to get into for this theoretical installer; I’m just spitballing here.
At a minimum, I would like to end up with some clear way for the user to know if they are looking at core plugins or user contributed plugins, but ideally they would be able to search both groups in the same directory.
Thanks for working on this. Please let me know if I can help in any way.
I have built and maintained a system like this in the past and helped others attempt to bootstrap similar things. If we’re going to reinvent this particular wheel, I’d like us to have a really strong case for why we shouldn’t use the existing infrastructure provided by the platform.
Just to clarify this. I did not intend to work on any kind of installer but I read other posts in this forum where people expressed thoughts about an installer, e.g. here
I am mostly interested to have a place where users can inform about their plugins, even in a way, that they list it with status under construction. There are so many good ideas, this is really a field where there is a lot to do and there is much fun in it. That’s why I thought it to be important, to have a place to look what is already there or what is in progress. Thus users can contribute to growing plugin projects instead of accidentally reinventing the wheel.
Nice. So, I was not that wrong with my idea about about a database and a data format like json. Fortunately, I am not the first that came up with this idea, so no Heureka this time
The only thing that I am not sure of is how to make the json information available to the wiki, in a way that would make it as simple as possible for users to only browse Kaleidoscope stuff and, moreover, to hide all the information that is irrelevant for a quick search, like architecture, types, etc.
There’s one thing, where I am not sure of. People might sometimes develop things that are not actually a Arduino library but something related, like any type of auxiliary scripting, etc. Information about such miscellaneous stuff would be at least as valuable as news and info about plugins.
How could we handle this? And could we reuse on of the entries of the json format for information that is relevant for us, say the distinction between user and stock, etc. plugins? Are we we free to add additional entries?
I’m afraid, I disagree. One reason why I started this topic is that I find it very inconvenient, having to use discourse`s search engine to determine if someone is already working on something or not.
Nobody writes forum posts with the clear intention that the provided information is supposed to be found easily, later on. At least, I don’t.
I just spend approx. 5 minutes to dig up algernon’s post about his ideas towards a plugin manager. Just as an example how hard it can be to find something that you even know it is there.
I would really appreciate some sort of database with nice search features. That does not mean that I wouldn’t want to discuss plugins, ideas, etc. here. I like open discussion and I am really fond of other peoples contributions about fascinating topics. I even read a lot of posts about hardware-hacking a keyboard that I do not physically have, at least no just yet.
In my experience databases of ‘work in progress’ projects quickly become…graveyards of good intentions and placeholders for things that never got started. Can you show me a project that has done what you’re describing and had it work out well? That might help me understand better.
Alternatively, would turning on discourse’s ‘tags’ support help make it easier to find what you want?
Not sure. As long as it is obviouse that a project is or was in progress, the information about it is still worth something. Even a dead project whose GitHub page I can visit, can be inspiring. Hope that didn’t sound too nekrophiliac
I could e.g. imagine to setup rules that say that if the URL of a listed project becomes unavailable for a certain amount of time and the the maintainer does not react, the project is kicked out of the list. Just as an idea.
No, due to a lack of experience. This is the first project I am contributing to. But you sound like you could name a good negative example.
To me tags sound generally attractive. Having the ability to assign multiple tags might be helpful. A topic would still live inside a category. But, it seems that we can only tag topics, not posts, or did I miss something? I often search for posts where someone mentioned something in particular. Thus, in my opinion, it would help to tag an individual post within an otherwise untagged topic.
I want to second @noseglasses’s opinion regarding the usefulness of Discourse as an information storage and retrieval system. I have found it quite difficult and time-consuming to find both topics and posts that I want to refer to if they’re not actively being discussed at the time, to the point that I often give up or don’t even try.
I do think it would be helpful if someone maintained a current list of plugins in a pinned topic on this forum, but having something that’s machine-readable would be even better, so I could list and search them from the command line, à la aptitude (I never use apt-get directly…).
Alternatively, would turning on discourse’s ‘tags’ support help make it easier to find what you want?
New users will want to read through a list of all the noncore plugins people have published. Reading all the threads in this category seems to be the best bet right now, but half of the threads aren’t an announcement of a plugin, which is annoying and just feels like it should be fixed as the problem will grow. I suggest either: 1. enable tags and make one for plugin releases (i haven’t used the tag functionality, so not sure if this would work), 2. split this category into 2, one for announcements of plugins, one for everything else programming + plugins, and move the existing threads into the appropriate category. 3. Start a wiki page with a list of non-core plugins. 4. An alternate wiki for making a list of plugins is https://directory.fsf.org, just make a category for keyboardio plugins.
The original purpose of the Plugins sub-category within Programming was to have only plugin announcement threads. (See About the Plugins category and this other thread beginning at about post 8.) Although I’m totally in favor of having a wiki page with a list of plugins, brief descriptions, links to threads here, etc - another (stopgap?) measure we could take would be to kick out all the threads other than plugin announcement threads back into Programming instead of the Plugins sub-category. (Threads can be very easily recategorized, even retroactively.)
I suppose we could let the non-plugin-announcement threads have the ‘Plugin’ category, and create a new category ‘Plugin Announcements’ or ‘Available Third-Party Plugins’ for plugin announcements. This would be clearer, as the original name ‘Plugins’ clearly isn’t communicating the intended purpose of third-party plugin announcements to average users.
Looking at the existing threads in Plugins, other than plugin announcements we seem to have:
Questions about usage of particular plugins
Questions about creating plugins
Questions about which plugin to use for a particular task
This thread, re: plugin database
As far as I’m concerned, I’m not sure there’s a meaningful difference between the threads listed above and threads that belong in the ‘Programming’ category in general. Indeed, sometimes the question about which plugin to use ends up being a programming question more than a plugin question.