This is about trouble I have with the command-line build system (kaleidoscope-builder).
I would like to know what’s the currently best way to work with several independent copies of the Keyboardio-Bundle without having to reset symbolic links in Arduinos $SKETCHBOOK_DIR/hardware directory as suggested by kaleidoscope/README.md?
Recently I started again working on the firmware and third party plugins. Is it possible that there where any changes committed to the build system in the recent half year that nowadays prevent the directory Kaleidoscope-Bundle-Keyboardio/avr/libraries from being searched for 3rd party Arduino libraries (plugins) if no symbolic link has been set?
The thing is that I am pretty sure that half a year ago or so the following would have been possible (although with Arduino-Boards instead of Kaleidoscope-Bundle-Keyboardio) on Ubuntu Linux with Arduino 1.8.5 without having to add a symbolic link to the Kaleidoscope-Bundle-Keyboardio to Arduino’s own library folder.
export ARDUINO_PATH=<place where your Arduino installation lives>
mkdir -p $HOME/kaleidoscope/hardware
git clone --recursive https://github.com/keyboardio/Kaleidoscope-Bundle-Keyboardio.git \
$HOME/kaleidoscope/hardware/keyboardio
cd $HOME/kaleidoscope/hardware/keyboardio/avr/libraries
git clone <some 3rd party plugin>
cd $HOME/kaleidoscope/hardware/keyboardio/avr/libraries/Model01-Firmware
# Modify the firmware to use the 3rd party plugin
make
With the latest version of Kaleidoscope-Bundle-Keyboardio this results in the 3rd party plugin’s headers not to be found.
Is there maybe a simple way that I am completely overlooking that also works without setting symbolic links? Changing those links is pretty tedious and error prone when working with multiple copies of the firmware in parallel.
Not that I recall, no. Mind you, the searching is done by Arduino, no Kaleidoscope or kaleidoscope-builder. We only tell Arduino where to search, but everything past that point is its territory.
You can put the additional libraries anywhere, and tell Arduino (via kaleidoscope-builder) to search for them there too. Perhaps even easier would be to dump the plugins into a place arduino searches by default (such as ~/Arduino/libraries).
Personally, I usually put everything under the lib/ directory of whatever I’m working on: lib/hardware/keyboardio will be a symlink to Kaleidoscope-Bundle-Keyboardio, 3rd party plugins will be dumped into lib/ directly. Then, in .kaleidoscope-builder.conf, I have these two magic lines:
everything works fine again. Probably I used to do it that way before.
I guess I became accustomed to this build workflow (putting all plugins in a common libraries directory) in the pre-monorepo age when all plugins were treated kind of equal.
In any case that seems to me to be the fastest way to do a command line build to try things out without setting up other stuff.
Would it be possible to add $PWD/../../../.. as a fallback hardware path to the build system, to enable builds from the Model01-Firmware or any other subdirectory of libraries without the need for further setup?
What is SOURCEDIR and where is it defined? How does the build system know which lib directory is referred to when specifying flag -libraries?
I assume .kaleidoscope-builder.conf lives in $HOME? Is its somewhere documented?
SOURCEDIR is something kaleidoscope-builder ties to figure out, defaults to $(pwd). lib is relative to the current directory too.
It lives in the current directory, but a global one in $HOME is included as well, if it exists. The order is $HOME/.kaleidoscope-builder.conf, ${SOURCEDIR}/.kaleidoscope-builder.conf, and ${SOURCEDIR}/kaleidoscope-builder.conf and finally etc/kaleidoscope-builder.conf from Kaleidoscope itself. These will all be included in this order, if any of them exist, and the later ones can override stuff in the earlier ones.
(Yes it is a tad confusing, not well documented, but we’re going to replace the CLI build system with something much simpler and far less confusing, and that will be documented properly.)
I tend to have several trees that I’m working on concurrently. Originally, I found it very difficult to work with a single tree because switch from one branch to another was unmanageable with the submodule structure, then I started writing my own version of the firmware, and needed to switch back and forth frequently. My solution was to write a short shell script to manage the symlink and always keep them pointing to whichever tree my current directory is a part of. I’ll share it here when I get a chance later today.
Here’s my link-avr-libs script. If I execute it from anywhere in a Kaleidoscope-Bundle-Keyboardio source tree, it makes the symlink I need to build with the libs in that tree.
#!/bin/bash
KALEIDOSCOPE_SOURCE_TREE=${1:-$(pwd)}
# Search for a directory with a file named "boards.txt"
while [[ ! -e ${KALEIDOSCOPE_SOURCE_TREE}/boards.txt ]]; do
KALEIDOSCOPE_SOURCE_TREE=$(dirname ${KALEIDOSCOPE_SOURCE_TREE})
if [[ ${KALEIDOSCOPE_SOURCE_TREE} == / ]]; then
echo "Kaleidoscope source tree not found" >&2
exit 1
fi
done
KEYBOARDIO_AVR_DIR="Arduino/hardware/keyboardio/avr"
if [[ $(uname -s) = "Darwin" ]]; then
KEYBOARDIO_AVR_DIR=Documents/${KEYBOARDIO_AVR_DIR}
fi
rm "${HOME}/${KEYBOARDIO_AVR_DIR}"
ln -sf "${KALEIDOSCOPE_SOURCE_TREE}" "${HOME}/${KEYBOARDIO_AVR_DIR}" && {
echo "${HOME}/${KEYBOARDIO_AVR_DIR} -> ${KALEIDOSCOPE_SOURCE_TREE}"
}
[Edit: This script doesn’t currently work properly for builds of sketches that are outside the Kaleidoscope-Bundle-Keyboardio tree. I’ve made a new version that fixes this problem, and put it in a GitHub repository that I will add a few other related tools to.]
That makes resetting the link more comfortable. Thanks for sharing this.
I would love a solution that works without setting a link. Not because I am lazy, which I definitely am, but because forgetting to reset it can yield all sorts of strange behavior.
I considered wrapping the build with a script that sets the link first (and possibly connects to the serial port for debugging after flashing), but since the sketch directory might not be within the bundle’s tree, that’s not guaranteed to work. That’s a problem that seems unsolvable – if the sketch is outside the Kaleidoscope tree, there’s no way for it to know which Kaleidoscope tree to use.
@noseglasses I think I’d be happiest if the $PWD/../../../.. got wrapped in a tiny check that makes sure something that looks like what we want is actually in that directory. (Checking for existence of a kaleidoscope-specific file, say.)
(Also, please add some comments about why the fallback is there, so that future code archeologists don’t get too confused
