Hey, it appears I forgot about the huge difference between debug and release builds. Rookie mistake, I know... 
The release build is around 370KB which seems to be close to what I expected (as it excludes the BuildSystem part).
This means I'll be trying to pod llbuild next time I have some spare time. 