Obviously there is a good, bad, and absolutely dreadful way of of doing anything no matter what the final method of chosen application is.
PHP isn't necessarily required to do a templating system, although using a more sophisticated server-side engine does makes things alot easier and in my mind atleast, alot cleaner. You can do the same thing your wanting to do SSI(server side includes).
CODE
<!--#include virtual="/footer.html" -->
Obviously, you'll need to change any files using SSI to .shtml and have a server/web host that supports such.
PHP templating systems are notorious for the previously mentioned hassle of finding what file your actually wanting to update.
When I first started using PHP, I just had to program a bulletin board. Matt Mecham(Invision Power Board) was my hero and I didn't dream of doing anything else. When it came time to write my template engine, the only method of templating I had ever seen at this point in time was Matt Mechams template "functions" using in IPB. I didn't like the idea of OO programming, it intimidated me at the time, so I ended up having a file for everything. Just my main page had over 30 templates! Can you imagine how much of a pain that is?
The perfect solution for you, assuming your actual page content doesn't have dynamic content(because then you'd already be using a dynamic scripting language) would be a header and footer system.
CODE
<?php
require('./header.html');
?>
Actual page content
<?php
require('./footer.html');
?>
Mixing HTML and PHP in the same file is generally flowned upon and professional programmers using such a style are usually crucified and left to die, but I'm speaking mere simplicity and ease of use for the webmaster just looking to run a simple website.
Instead of just answering your question, which I feel I would have simply provided a biased answer being a PHP programmer, I just gave you two options and left the final call to you. Hope thats alright.