No, it also fires when the key is released. It also fires every cycle while the key is held. It also used to fire when the key was idle, which had its uses, but we removed that feature, because it made the main loop a lot faster. Always fast > marginally useful use-case.
So with an up-to-date Kaleidoscope, macros fire when the key toggles on, while it is pressed, and when it is toggled off. This is by design, and is useful, I don’t see it changing anytime soon. If you’re curious about why it is useful, let me know, and I’ll present some examples 