A closely-related question about third-party (or, more specifically non-Arduino-Boards) plugins is where one should put them when building custom firmware. In the libraries directory, along with the official ones? Symlink’d there? As submodules of one’s own firmware sketch? In a separate library directory? Does the build system already have a place for them?
If there isn’t already a system for this, let’s make one.
“libraries” in the Arduino sketch directory is traditional (~/Arduino/libraries or ~/Documents/Arduino/libraries depending on the OS.)
I’d thought that you could also drop them inside lib/ in your sketch directory. (Though now I wonder if that’s libraries/ inside your sketch dir.) If the IDE supports one of those and the commandline build doesn’t, that’s a bug and something we should fix.
I’ll experiment when I get a chance. If $SKETCH_DIR/lib or equivalent works, I think it would make sense to include third-party plugins as git submodules, but that’s probably not a great solution for people who are not git-adept.
Just something I threw together here:
It has not been maintained at all.
Nice. That will be very useful to newbies.
Maybe we can add some sort of table for user plugins? Is it possible to use a data format that is machine parsible to generate markdown tables?
I started a new topic
Well, the first line of rules.mk is:
EXTRA_BUILDER_ARGS="-libraries ."
…so it looks like the anything added to the sketch directory itself would get found, but I’m not 100% certain.
Yup. The big question that should get answered is what the precedence order is, since Arduino has…surprised us on that front before. If you put something in the sketch directory and it’s also in Arduino-Boards, what happens? And does the coommandline behave the same as the IDE?
Arduino has, we believe, changed the build order on us before. The current documentation for search path ordering is buried in the commentary on some pull requests that proposed changing it.
I suspect the right way to figure this out is to enumerate all the possible places a library might get placed and write a script to compile a sketch with different versions of the library in each place, testing out every possible permutation. That’d let us autogenerate truthful docs and also catch potential changes.
I started to explore how to best bundle plugin libraries, and the short version is that downloadable zip/tar files are the most straightforward way.
A work in progress still, but you can find my notes here, along with proposed categories (same as @jesse proposed before, with the addition of an Experimental category), and current plugins proposed for each.
I’m wondering how this plan will affect a developer’s setup. Right now, I’ve got an Arduino-Boards clone symlinked to $SKETCHBOOK_DIR/hardware/keyboardio/avr, with multiple remotes for some of the submodules, as well as Arduino-Boards itself. Separate from that, I have my own Model01-Firmware fork cloned elsewhere, with third-party plugins set up as submodules from the top level so the build system will find them when I run make from my Model01-Firmware/ dir (not the one under Arduino-Boards; I just use that one if I want to send a PR to keyboardio/Model01-Firmware).
Basically, I’m interested in knowing how you see the restructuring affecting how we organize and build in development, or if the bundling you’re discussing is unrelated entirely.
I imagine Arduino-Boards will remain somewhat similar to how it looks now, a combination of Core+Extra+Experimental. So I suppose nothing will change there.
If not using Arduino-Boards, then there’ll be a small script or a wiki page that explains how to merge the different repos locally…
That just gave me another idea: what if we had one repo, but with branches? And master would be a merge of all of them. (Not including third-party stuff, that’d still be a separate thing). Releases can be tagged on branches, zip files generated, and so on.
Not sure if this’d be better, or more confusing.
I’m not sure I follow. Do you mean some branches of Arduino-Boards would have some of the submodules, but not others? If that’s what you mean, I think I’d prefer not to do it that way; the reason I put third-party plugin clones under my own Model01-Firmware fork is so I wouldn’t have to deal with them cluttering up the output of git status in Arduino-Boards. I suppose .gitignore could take care of that problem, though.
I’d use a separate repo, that has nothing but the submodules (and Arduino-Boards would include it as a submodule). Lets call that repo Plugins. It would have a plugins/core, plugins/extra, and a plugins/experimental branch, each with their own set of plugins. Then, it would have master, which would be the merge of all that. As a developer, you’d work with master most of the time, so no cluttered git status. The rare times one would want to check out any of the other branches, is to move plugins between the repos, or update one, or something along those lines. Things that should be rare enough. (We could automate updates one way or the other, or delegate that to the set maintainers, and just open an issue if an update is desirable). Or we could track tagged plugin releases, kinda halfway between automation and manual requests.
In any case, what I wish to end up with, is developers having to care very little about the branches or repos.