Deciding to just avoid all possible hot chimney pipe exposure to the walls, I've added a cement board outer wall:
There's still a gap, so I put cement board all the way around as well:
and then added a little extra aluminium at the top for heat buffering. Can't hurt, right?
