Hosting can impact site speed.
"80% of the end-user response time is spent on the front end." The preceding statement could be interpreted to mean that back end hosting has little impact on front end site performance.
To test this experiment, an existing site was replicated at a new and separately hosted IP address. DNS was changed from the original host IP to the new host IP. A few days later, DNS was changed back to the original host IP. A few days after that, DNS was again changed to the new host IP from the original host IP.
According to Google's Site Performance tool, pages at the new host IP (dashed line) loaded much faster than pages at the original host IP (solid line). There appears to be an obvious and immediate improvement of more than 50% when DNS was initially pointed from the original host to the new host. Similarly there appears to be a decrease in speed when DNS was pointed back at the original host IP from the new host IP and increase when pointed back at the new host IP again. Since DNS was pointed at the new host IP, site speed and performance have continued to improve according to Google Webmaster Tools.
This experiment seems to indicate a strong correlation between changes in hosting and changes in site performance. This correlation is no real surprise given, the new host is highly rated as fast and reliable. Conventional wisdom is that "hosting" doesn't impact site performance but, I think it's worth testing just in case your site is one of those rare exceptions.