My reaction would be 'No'. Your broadband speed is very likely to be considerably less than either WiFi 'G' or 'N', so download/upload speeds from/to the Internet will be unaffected. But, if you transfer a lot of files from one machine to another on your local network, then you might notice a difference.
There are, too, differences in the 'range' of the two standards, should that be a concern to you.
Tim