Elf_diff - a tool to compare firmware binaries

Programming
,
1

If you are looking for a comfortable tool to check possible improvements to firmware binaries in terms of size, symbols and assembly, the following result of my this mornings work might help you.

Invoke it as

elf_diff --bin-dir <path to where avr-gcc lives> \
         --bin-prefix "avr-"  \
         --old <full path to old firmware ELF binary> \
         --new <full path to new firmware ELF binary>

Output might look somewhat like this:

ELF binary comparison

   (c) 2019 by noseglasses (shinynoseglasses@gmail.com)

Comparing binaries
   old: /tmp/kaleidoscope-noseglasses/sketch/17810223-Model01-Firmware.ino/output/Model01-Firmware-0.0.0.elf
   new: /tmp/kaleidoscope-noseglasses/sketch/18182958-Model01-Firmware.ino/output/Model01-Firmware-0.0.0.elf

Binary size:
   overall: 26216 -> 26306 bytes (+90 bytes) 
   text: 25960 -> 26038 bytes (+78 bytes) 
   data: 256 -> 268 bytes (+12 bytes) 

Static RAM consumption:
   overall: 1434 -> 1390 bytes (-44 bytes) *
   data: 256 -> 268 bytes (+12 bytes) 
   bss: 1178 -> 1122 bytes (-56 bytes) *

text: code instructions
data: initilized global or static variables
bss: uninitialized global or static variables

589 symbols found in /tmp/kaleidoscope-noseglasses/sketch/17810223-Model01-Firmware.ino/output/Model01-Firmware-0.0.0.elf
598 symbols found in /tmp/kaleidoscope-noseglasses/sketch/18182958-Model01-Firmware.ino/output/Model01-Firmware-0.0.0.elf

24 Symbols changed size:
   LEDBreatheEffect: 6 -> 4 bytes
   _GLOBAL__sub_I__ZN12kaleidoscope6plugin14ColormapEffect9map_base_E: 14 -> 10 bytes
   kaleidoscope::Hooks::onSetup(): 4 -> 102 bytes
   kaleidoscope::plugin::BootGreetingEffect::afterEachCycle(): 170 -> 182 bytes
   kaleidoscope::plugin::ColormapEffect::onLayerChange(): 40 -> 44 bytes
   kaleidoscope::plugin::LEDControl::activate(kaleidoscope::plugin::LEDMode*): 30 -> 72 bytes
   kaleidoscope::plugin::LEDControl::beforeReportingState(): 116 -> 102 bytes
   kaleidoscope::plugin::LEDControl::get_mode(): 20 -> 8 bytes
   kaleidoscope::plugin::LEDControl::next_mode(): 56 -> 26 bytes
   kaleidoscope::plugin::LEDControl::onSetup(): 82 -> 90 bytes
   kaleidoscope::plugin::LEDControl::prev_mode(): 54 -> 36 bytes
   kaleidoscope::plugin::LEDControl::refreshAll(): 54 -> 40 bytes
   kaleidoscope::plugin::LEDControl::set_mode(unsigned char): 22 -> 54 bytes
   kaleidoscope::plugin::LEDMode::onSetup(): 34 -> 20 bytes
   kaleidoscope::plugin::LEDSolidColor::LEDSolidColor(unsigned char, unsigned char, unsigned char): 18 -> 10 bytes
   kaleidoscope::plugin::NumPad::setKeyboardLEDColors(): 290 -> 276 bytes
   solidBlue: 5 -> 3 bytes
   solidGreen: 5 -> 3 bytes
   solidIndigo: 5 -> 3 bytes
   solidOrange: 5 -> 3 bytes
   solidRed: 5 -> 3 bytes
   solidViolet: 5 -> 3 bytes
   solidYellow: 5 -> 3 bytes
   toggleLedsOnSuspendResume(kaleidoscope::plugin::HostPowerManagement::Event): 84 -> 68 bytes

15 symbols dissappeared (527 bytes, see details below):
   _GLOBAL__sub_I__ZN12kaleidoscope6plugin10LEDControl5modesE: 8 bytes
   _GLOBAL__sub_I__ZN12kaleidoscope6plugin16LEDBreatheEffect6updateEv: 26 bytes
   kaleidoscope::EventHandlerResult kaleidoscope_internal::EventDispatcher::apply<kaleidoscope_internal::EventHandler_onSetup>(): 166 bytes
   kaleidoscope::plugin::ColormapEffect::onActivate(): 30 bytes
   kaleidoscope::plugin::ColormapEffect::refreshAt(unsigned char, unsigned char): 32 bytes
   kaleidoscope::plugin::LEDBreatheEffect::update(): 46 bytes
   kaleidoscope::plugin::LEDControl::LEDControl(): 20 bytes
   kaleidoscope::plugin::LEDControl::mode: 1 bytes
   kaleidoscope::plugin::LEDControl::mode_add(kaleidoscope::plugin::LEDMode*): 48 bytes
   kaleidoscope::plugin::LEDControl::modes: 48 bytes
   kaleidoscope::plugin::LEDControl::refreshAll() [clone .part.2]: 48 bytes
   kaleidoscope::plugin::LEDSolidColor::onActivate(): 12 bytes
   kaleidoscope::plugin::LEDSolidColor::refreshAt(unsigned char, unsigned char): 18 bytes
   vtable for kaleidoscope::plugin::ColormapEffect: 12 bytes
   vtable for kaleidoscope::plugin::LEDSolidColor: 12 bytes

24 new symbols (527 bytes, see details below):
   _GLOBAL__sub_I__ZN12kaleidoscope6plugin10LEDControl7mode_idE: 6 bytes
   _GLOBAL__sub_I__ZN12kaleidoscope6plugin16LEDBreatheEffect15ExportedLEDMode6updateEv: 22 bytes
   kaleidoscope::internal::led_mode_management::array: 37 bytes
   kaleidoscope::internal::led_mode_management::cur_led_mode: 2 bytes
   kaleidoscope::internal::led_mode_management::cur_mode_id: 1 bytes
   kaleidoscope::internal::led_mode_management::getLEDMode(unsigned char): 106 bytes
   kaleidoscope::internal::led_mode_management::led_mode_buffer: 6 bytes
   kaleidoscope::internal::led_mode_management::retreiveLEDModeFactoryFromPROGMEM(unsigned char, kaleidoscope::internal::led_mode_management::LEDModeFactory&): 26 bytes
   kaleidoscope::internal::typed_plugins::LEDMode::array: 15 bytes
   kaleidoscope::internal::typed_plugins::LEDMode::getLEDMode(unsigned char): 18 bytes
   kaleidoscope::internal::typed_plugins::LEDMode::num_entries: 1 bytes
   kaleidoscope::plugin::ColormapEffect::ExportedLEDMode::onActivate(): 30 bytes
   kaleidoscope::plugin::ColormapEffect::ExportedLEDMode::refreshAt(unsigned char, unsigned char): 32 bytes
   kaleidoscope::plugin::LEDBreatheEffect::ExportedLEDMode::update(): 52 bytes
   kaleidoscope::plugin::LEDControl::mode_id: 1 bytes
   kaleidoscope::plugin::LEDMode* kaleidoscope::internal::led_mode_management::generateLEDMode<kaleidoscope::plugin::ColormapEffect, kaleidoscope::plugin::ColormapEffect::ExportedLEDMode>(void*, void*): 26 bytes
   kaleidoscope::plugin::LEDMode* kaleidoscope::internal::led_mode_management::generateLEDMode<kaleidoscope::plugin::LEDBreatheEffect, kaleidoscope::plugin::LEDBreatheEffect::ExportedLEDMode>(void*, void*): 24 bytes
   kaleidoscope::plugin::LEDMode* kaleidoscope::internal::led_mode_management::generateLEDMode<kaleidoscope::plugin::LEDSolidColor, kaleidoscope::plugin::LEDSolidColor::ExportedLEDMode>(void*, void*): 20 bytes
   kaleidoscope::plugin::LEDSolidColor::ExportedLEDMode::onActivate(): 20 bytes
   kaleidoscope::plugin::LEDSolidColor::ExportedLEDMode::refreshAt(unsigned char, unsigned char): 28 bytes
   memcpy_P: 18 bytes
   vtable for kaleidoscope::plugin::ColormapEffect::ExportedLEDMode: 12 bytes
   vtable for kaleidoscope::plugin::LEDBreatheEffect::ExportedLEDMode: 12 bytes
   vtable for kaleidoscope::plugin::LEDSolidColor::ExportedLEDMode: 12 bytes

########################################################################
Details follow
########################################################################
The following 340 symbols' assembly differs

******************************************************************
.do_clear_bss_start (size unchanged)
******************************************************************
   - aa 39       	cpi	r26, 0x9A	; 154
   ?  ^  ^                   ^^    ^^

   + ae 36       	cpi	r26, 0x6E	; 110
   ?  ^  ^                   ^^    ^^

     b2 07       	cpc	r27, r18
   - e1 f7       	brne	.-8      	; 0x740 <.do_clear_bss_loop>
   ?                                   -

   + e1 f7       	brne	.-8      	; 0x774 <.do_clear_bss_loop>
   ?                                  +


******************************************************************
AbsoluteMouseAPI::moveTo(unsigned int, unsigned int, signed char) (size unchanged)
******************************************************************
     cf 93       	push	r28
     df 93       	push	r29
   - 00 d0       	rcall	.+0      	; 0x3de6 <AbsoluteMouseAPI::moveTo(unsigned int, unsigned int, signed char)+0x6>
   ?                                   - ^

   + 00 d0       	rcall	.+0      	; 0x3e22 <AbsoluteMouseAPI::moveTo(unsigned int, unsigned int, signed char)+0x6>
   ?                                    ^^

   - 00 d0       	rcall	.+0      	; 0x3de8 <AbsoluteMouseAPI::moveTo(unsigned int, unsigned int, signed char)+0x8>
   ?                                   - ^

   + 00 d0       	rcall	.+0      	; 0x3e24 <AbsoluteMouseAPI::moveTo(unsigned int, unsigned int, signed char)+0x8>
   ?                                    ^^

   - 00 d0       	rcall	.+0      	; 0x3dea <AbsoluteMouseAPI::moveTo(unsigned int, unsigned int, signed char)+0xa>
   ?                                   - ^

   + 00 d0       	rcall	.+0      	; 0x3e26 <AbsoluteMouseAPI::moveTo(unsigned int, unsigned int, signed char)+0xa>
   ?                                    ^^

     cd b7       	in	r28, 0x3d	; 61
     de b7       	in	r29, 0x3e	; 62
     dc 01       	movw	r26, r24
     13 96       	adiw	r26, 0x03	; 3
     7c 93       	st	X, r23
     6e 93       	st	-X, r22
     12 97       	sbiw	r26, 0x02	; 2
     15 96       	adiw	r26, 0x05	; 5
     5c 93       	st	X, r21
     4e 93       	st	-X, r20
     14 97       	sbiw	r26, 0x04	; 4
     16 96       	adiw	r26, 0x06	; 6
     3c 91       	ld	r19, X
     16 97       	sbiw	r26, 0x06	; 6
     39 83       	std	Y+1, r19	; 0x01
     7b 83       	std	Y+3, r23	; 0x03
     6a 83       	std	Y+2, r22	; 0x02
     5d 83       	std	Y+5, r21	; 0x05
     4c 83       	std	Y+4, r20	; 0x04
     2e 83       	std	Y+6, r18	; 0x06
     ed 91       	ld	r30, X+
     fc 91       	ld	r31, X
     01 90       	ld	r0, Z+
     f0 81       	ld	r31, Z
     e0 2d       	mov	r30, r0
     46 e0       	ldi	r20, 0x06	; 6
     50 e0       	ldi	r21, 0x00	; 0
     be 01       	movw	r22, r28
     6f 5f       	subi	r22, 0xFF	; 255
     7f 4f       	sbci	r23, 0xFF	; 255
     09 95       	icall
     26 96       	adiw	r28, 0x06	; 6
     0f b6       	in	r0, 0x3f	; 63
     f8 94       	cli
     de bf       	out	0x3e, r29	; 62
     0f be       	out	0x3f, r0	; 63
     cd bf       	out	0x3d, r28	; 61
     df 91       	pop	r29
     cf 91       	pop	r28
     08 95       	ret
6 Likes
2

Ooooh, this is awesome!! Thank you for writing & sharing it!

3

Nice to hear that you find it useful. If you have any ideas for possible improvements, please let me know.

Up to now, this is only tested on Ubuntu where I did not need to install additional Python packages. Can’t tell for other systems, though. Any info appreciated.

4

I just added html output to gain sortable tables for symbol names, sizes, changes, etc.
Any ideas to make this prettier are welcome (I am pretty much a web design rookie).

5

Screenshot_20190321_185808
Screenshot_20190321_185808.png468×571 21.3 KB

2 Likes
6

Just wanted to add that html layout is fixed now. Everything still looks like coming from the good old 90ies, though :wink:

7

This is amazingly cool. I love it. I’d <3 makefile targets that add it to Kaleidoscope to diff the current checkout against the previous commit.

8

It’s Python and it requires some extra packages. I am a Linux developer. Not sure about the package availability under other platforms.

Diffing against the previous commit. Sounds cool but I am not entirely sure how the previous commit would be defined best. Which commit state of which of the bundle’s libraries? Would we skip local commits and consider only the most recent commits that are already on github’s master branches?

Here’s my usual workflow that I adopted recently.
First, I checkout the bundle and pull it up to its most current state using make maintainer-update-submodules. Next, I take a recursive copy of the bundle directory and build the origin as a reference. Then I start developing in the copied repo. Once in a while I check the impact of my changes by diffing the binary of the development build against the reference binary.

Some additional thoughts about how this could be used to aid PR reviews:

I am planning to improve the Kaleidoscope elf_diff wrapper in a way that it compares the generated firmware binaries of all smoke-examples tests between two copies of the bundle and summarize the results it in a short report that comprises all tests and just considers changes in terms of PROGMEM and RAM consumption. That would make it easy to detect problematic resource usage.

Once I have that working, the same workflow as I described above could be used by travis to automatically diff PR smoke-examples tests against reference builds and then generate a pdf change report.

How could travis publish such a report? Can build artefacts like pdf reports be somehow be attached automatically to a github PR? It would be really cool to have automatically generated binary change reports available to the reviewers, updated with every commit that is added.

1 Like
9

That all makes sense. I think I was thinking of the most recent local change. Though I suppose it’s easy enough to just say you have to give it a reference.

For travis: The simplest possible answer is to have it print the report to STDOUT. That’s not as nice as getting the pretty HTML report, but it’s enough to make sure that we -have- it on every travis run.

10

If there’s a fairly simple way to have it produce markdown output, that would be a nice way of sharing the report on GitHub — for example, as a Gist.

11

Yeah, markdown would be cool but it cannot be colored. Can we have gist like pdfs somewhere and then let travis post a link to those?

12

Personally, I find PDFs much less convenient to read than HTML or markdown, especially if a table is large enough to span two pages.

13

The thing I care for most is the colored highlighting of those numbers that are crucial within the report. That’s why markdown is not an option. I read that github strips even the HTML that would be accepted by some Markdown flavors.

HTML is cool because it allows tables to be sorted. I create the pdf documents from HTML pages anyway. But up to now I am not aware of a good way to publish stuff as HTML. If you have any ideas, I am very interested.

14

One reasonably easy way is to push to GitHub Pages.

1 Like
15

Would that enable us to have multiple pages, one for every open PR? If not, any ideas for workarounds?

16

I’d build it as an automation in Travis: see a PR, build the diff, store the HTML in a file based on the PR ID, push that to a separate repo (say, kaleidoscope-binary-diffs), and let GitHub build pages from that. We don’t disturb the history of Kaleidoscope then, and can have per-PR pages.

We might be able to build a tool that’d link the result to the GitHub PR itself (a’la DCO or Travis).

2 Likes