In our fight for more secure web we killed the open web.
When hypertext was young, the reference was more than just a link. The idea was to really link the two documents, such that the related document is never lost. We could not do it this way at the time. So we added with more simple approach: the link is just a link. There may be no relation between the two documents, there are no guarantees.
For a while, this created an Open Web. The data was there. Any page can link to any page, it could use data from anywhere in the Internet. We started to have sites, which branded themselves "Portals", "Meta Search Engines" and "Aggregators". It was great. The world was becoming more and more connected.
Not everybody used this openness in a fair way. Soon sites started to fight hot-linking of their content. The hackers used cross-site scripts to benefit from user's cookies and steal user's data. Or worse. And the struggle for security has begun.
Bit by bit, the Secure Web nimbled at the Open Web principles. Browsers were adding more and more features that would "protect" users. Mostly from themselves. Eventually, we lost the Open Web.
I wanted to create a simple HTTP page recently. There is a site on the Web that presents some sports-related statistics and I wanted to use the data and present it differently. Turns out, it is impossible on the modern Web. Both sites in question are just a collection of static HTML files. Both sites have no concept of user registration. They do not ask or allow user to ever enter her information on either of the sites. And yet the browsers prevent any attempt by JavaScript on site A to get the data from a page on site B.
It is done in the name of security. XSS is a big scary beast and browsers blatantly disallow it. It does not make the Web much more secure. But it does make it way more of a walled garden. In this particular case XSS prevention makes the Web LESS secure. Just consider what a developer needs to do to make it work.
The developer needs to write a simple backend, which proxies a request. When user visits site A, it makes a call to the A's backend, which calls site B from the server and returns the data. Since A's developer does not want to write a handler for each possible page on site B, the proxy is going to be dynamic. There will be a parameter which tells the server which URL to hit. Do you see where it is going? It is actually not trivial to write a secure proxy. A simple mistake may open the A's developer's internal network to the hacker. A vulnerability that had no chance to exist if JS was allowed to call site B directly from the user's machine.
The existence of the dedicated backed also presents a legal hurdle. A typical site has a copyright on its pages. Meaning that non-authorized reproduction of their content on other sites are not allowed. If it were the end-users PC making an HTTP request, there is no legal problem. Site A simply supplies an utility to enhance the content. Site A never served any of the content it did not own. Not so if there is a proxy on the backend. In this case the content appears to come from site A to the end user, which is a direct violation of the copyright.
If you think this is only a theoretical problem, I have another example. Facebook has barely any search functionality, yet it bars any other search engine from improving it. This baring is made possible by the Secure Web premises and would not be possible if Open Web was still a thing. Just ask yourself, when was the last time you heard "I saw this on facebook, but I am not going to waste time searching for it". I hear a variant of this almost every week.
Another problem with the Secure Web (perhaps I should start to put quotes on it, as it is not really secure) is its reliance on badges. In its early incarnations sites would embed an image that reassured users that the site is safe. It was easily spoofed, of course. So we went further the same way and added the badges to the browsers. Now if you visit a site over HTTP you see a red label in your browser.
This very blog is served over both HTTP and HTTPS. Exact same version and there is no redirect from port 80 to 443. There is no reason for TLS on this site really. I do not even have comments here. For the exact reason that I do not want to compromise the site's security. And yet if you visit this perfectly safe site over HTTP your browser is going to mark it insecure. You may point out that a hacker can sniff out which blog articles you are reading, but your browser is actually advertising this to everyone interested anyway. I have no trackers on this site, but your browser have them embedded. In the name of security.
The new Firefox Quantum, released this week, went even further. If you use it to visit this very blog over HTTPS you'll see a "B" rating for the security. And the sole reason for the "B" rating is that my site is missing a badge from some tosdr.org site.
It is quite a sad state of things. With the severed cross-site links and the badge system that benefits big players we are pushing everybody towards the big player's walled gardens. Those have no problem sharing the data inside. And the badges are all green.
And the worst part, the web is no more secure than it was before. We still have security researchers finding huge holes in the most secure browser/OS combinations.