I accept this doesn’t work and will just use a workaround, but I’m curious about why it doesn’t work.
A while back I experimented with changing the key to activate the numpad layer (which I would seldom use). I tried fun + N which works fine to activate the layer, but mostly doesn’t work to toggle back, even though N is a transparent key in that layer. Very occasionally it will work, but most of the time it won’t.
Basically, the layer toggle moves the target layer to the top, unless it’s already at the top of the layer stack, in which case it’s deactivated. Try holding fun down and tapping N twice to unlock the numpad layer to work around the problem in the meantime.