The Problem With Single Page Applications
An SPA is going to be significantly more complex than a non SPA website. Period.
Most developers opt to use a framework to help mitigate the inherent complexity with a SPA. However, a framework isn't actually necessary. But for larger SPAs a framework is recommended, and the choice of that framework could have significant long-term ramifications.
One of the main goals for a framework is to try to mitigate complexity, but unfortunateley they aren't a magic bullet that solves the problem entirely. Frameworks require time and effort to learn, and they introduce a fairly rigid set of rules which restricts how things are implemented. The learning curve varies by framework but are typically steep. And because there are a lot of different frameworks, and developers tend to prefer a specific tool, a SPA website is going to be bound by the limitations of a particular framework and highly dependent on the availability of developers who know it.
Size & Performance
SPA frameworks are notoriously bulky, in some cases adding hundreds of kilobytes of code to the final product. Typically this is a trade-off between features and performance; a more feature-rich framework will be bigger. Regardless, a larger library is going to take longer to load, despite tricks and techniques to optimize it. This ultimately affects page load time, which in turn can drastically impact a user's experience. No matter how you look at it, performance does matter.
With any website, maintainability is extremely important - after all, websites need to be updated and maintained. A framework will complicate the maintainability of a website, especially if the original developer is not going to be responsible for the maintenance.
Another concern is the long-term support for the framework of choice. There are many SPA frameworks available, but as so often happens with technology, projects are abandoned or replaced. Newer frameworks will replace the old ones, but unfortunately this doesn't mean an easy migration - in all liklihood refactoring a website to use a new framework would require a complete rebuild.
SPA's Are Not Search Engine Friendly
A Single Page Application is not indexable by internet search engines. Due to the nature of an SPA site, there is simply no way for a crawler to effectively explore and index the site. The only obvious solution is to create what are known as "doorway pages" which are essentially a statically rendered copy of key elements of the SPA site. Unfortunately, this solution is pretty fragile - not only do search engines frown on (and penalize) sites that use doorway pages, but unless you are willing to pay a company to maintain that service a DIY based approach is almost certainly going to fail badly.