Thank you for taking time to share your perspective. However, I remain skeptical.
It's true that a website using Cloudflare is more independent than a Facebook page, in that in the former case, the company can take their domain to another provider. But my idea of an independent Web is a large number of websites depending on a large number of high-quality hosting providers. The latter number will inevitably be smaller, but shouldn't be single-digit. That would lead to too much potential for abuse of power.
Also, the more sites are using a single provider with its black-box algorithms and heuristics, the more potential there is for bad consequences for innocent users when those things misfire. That's what worries me about the bot-fighting feature you launched on Monday.
To respond specifically to part of what you said:
> The concern was that the challenges of being online would get so hard that individual websites would give up and just move to run Facebook pages.
I don't think I understand how Cloudflare actually helps here. I think the average bar, karaoke DJ (I love karaoke), spa, or other small business that might just use a Facebook page would be served just as well by the kind of hosting provider that gives your website a single IP address pointing to a single machine. Are DDoS attacks and bots really that big of a problem? If so, I haven't run into them in the 16 years that I was the programmer and sysadmin for a small company (admittedly, online services are that company's business). Maybe we just didn't make the right enemies? Now, maybe small web hosting providers could make it even easier to set up a new website, but Cloudflare doesn't do anything about that problem anyway. If the concern is performance, maybe we need better alternatives to WordPress and Drupal, and more local hosting providers, so the website for small businesses can be closer to their mostly-local customers without using a CDN.
> The latter number will inevitably be smaller, but shouldn't be single-digit.
The space Cloudflare is in could afford plenty of players, I think—more than a single-digit amount. There’s nothing about Cloudflare’s business strategy that implies/necessitates that they’d become a monopoly in a market equilibrium state. The only reason you don’t see a pack of Cloudflare clone-companies, AFAIK, is that the talent required to clone Cloudflare is rare.
(Interestingly, an ISP—especially a cellular ISP familiar with routing roaming circuits—could totally pivot into Cloudflare’s business to expand globally. I wonder why we haven’t seen that?)
> I think the average bar, karaoke DJ (I love karaoke), spa, or other small business that might just use a Facebook page would be served just as well by the kind of hosting provider that gives your website a single IP address pointing to a single machine. Are DDoS attacks and bots really that big of a problem?
I feel like the perspective you’re coming at the problem from here is already heavily influenced by the contraction and centralization that the web went through in the early 2000s. Yes, right now, businesses just want essentially an online business card, and Facebook handles that just fine. But their desires are more of an acknowledgement of the practicalities of what’s economical for them to have built and hosted in the current (or recent-historical, since it takes a while for people’s thoughts on this to shift) web landscape.
Look around the internet of the 90s. Companies didn’t used to build business-card websites. The dreams of even the most run-of-the-mill SME used to be far more grandiose. At the very least, every company who knew what the options were, wanted to host a forum for the community composed of their customers. Many of the web’s most prominent standalone forums were started back then. Why so few today? Because ambitious, dynamic, user-generated-content-filled sites like these do get hurt by spamming and DDoSing. They’re hard to run—and not just in a community-management sense, but in an ops sense.
Cloudflare’s tech (which, again, anyone could offer, not just Cloudflare) can and does provide the protection required to allow SME websites to be a little bit more ambitious again, to the point that they’re not just doing something commoditizable by Facebook.
Talent is everywhere, but a lot of people who have it don't want to move to a big city. So IMO, the next Cloudflare's developers should be as widely dispersed as its POPs.
Edit: The more recently added part of your comment is very insightful, and I hadn't thought about it that way. Still, I think we could go a lot further with old-school hosting providers if we traded PHP and Ruby for Rust, Nim, and the like. Note that I didn't mention garbage-collected languages, because lots of applications running efficiently on a shared host is incompatible with a garbage collector that really wants the whole heap to itself.
GC'd langs like golang, crystal, nim, etc. would probably be just as effective in practice, while remaining more accessible to business app developers.
.NET Core benchmarks since Span<T> have been very interesting, especially relevant to this particular discussion because ASP.NET (web stack) was a primary consumer/driver for Span<T> APIs.
I ran a small web service in the video game industry for several years, and CloudFlare was essential to our survival, as the DDoS attacks would repeat every few weeks, and at times last 6 to 12 hours at a time. CloudFlare simply ate that up, and our customers were not impacted. Today, at a different company, different industry, we use CloudFlare for similar needs, but within physical area security networks. It's essential.
They do, and CloudFlare has historically been part of the reason why they have such high DDoS risks. There's a bunch of "booter" sites out there which effectively sell botnet-as-a-service DDoS attacks to gamers, and those sites have relied on CloudFlare to stay online. Without that protection their competitors would DDoS their websites offline most of the time. Also, most reputable hosting and CDN services don't allow booters because they're both highly illegal and disruptive to the entire internet. CloudFlare, on the other hand, openly permits them.
We were hit hard by Chinese IP addresses. After a while we just blocked the entire Chinese IP range. Expecting script kiddies to try to hack our system, we started out with a Federal Reserve quality hardware firewall, and I suspect the presence of that security attracted attention.
Hetzner is 20 years old company with ~300 employees and ~230,000 servers. Of course on scale of AWS, Google and others it's fairly small, but CloudFlare is not all that much larger.
As a guess, people who are invested in games are more likely to consider themselves techy people, the competition makes everything a bit tenser and elicits more excitement, and games are explicitly online only.
It's true that a website using Cloudflare is more independent than a Facebook page, in that in the former case, the company can take their domain to another provider. But my idea of an independent Web is a large number of websites depending on a large number of high-quality hosting providers. The latter number will inevitably be smaller, but shouldn't be single-digit. That would lead to too much potential for abuse of power.
Also, the more sites are using a single provider with its black-box algorithms and heuristics, the more potential there is for bad consequences for innocent users when those things misfire. That's what worries me about the bot-fighting feature you launched on Monday.
To respond specifically to part of what you said:
> The concern was that the challenges of being online would get so hard that individual websites would give up and just move to run Facebook pages.
I don't think I understand how Cloudflare actually helps here. I think the average bar, karaoke DJ (I love karaoke), spa, or other small business that might just use a Facebook page would be served just as well by the kind of hosting provider that gives your website a single IP address pointing to a single machine. Are DDoS attacks and bots really that big of a problem? If so, I haven't run into them in the 16 years that I was the programmer and sysadmin for a small company (admittedly, online services are that company's business). Maybe we just didn't make the right enemies? Now, maybe small web hosting providers could make it even easier to set up a new website, but Cloudflare doesn't do anything about that problem anyway. If the concern is performance, maybe we need better alternatives to WordPress and Drupal, and more local hosting providers, so the website for small businesses can be closer to their mostly-local customers without using a CDN.