Site slow due to mass scraping / DDOS from Alibaba Singapore datacenter

This is a whole side of the Internet about which I am mostly unaware tbh. I wonder how long until the whole WWW collapses under the weight of all the AI/bot-driven traffic?

The problem is larger than that. Not that I have some exceptional overall perspective on this all but here's an eclectic mix of things I've come across: Data centers' energy consumption is set to double in fife years, from a level that is already high (already comparable to nation states). There are - it must be recognized - significant uncertainties to this, for example algorithm and processor developments could suddenly plunge energy demands (DeepSeek is a good example about a market shock in this regard though it proved to be a somewhat false dawn in terms of resource-thrifty AI) but this is by no means a given.

Large tech companies' oligarchs, broligarchs and (largely authoritarian) world leaders are placing large bets in AI (self-)evolving into a state that would consolidate their own power and/or have quasi religious beliefs about singularities or (becoming) some sort of artificial gods, even. These trends have been said to be loosely organized around "TESCREAL" (Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalist ideology, Effective altruism, Longtermism), the self-described practitioners of which range from true believers to cynics justifying their current misuses of power (environmental destruction, human rights abuses, totalitarianism) with "superior future benefits".

Of course it's also a question of pure greed and a "classic" land rush, to corner the market (limited energy, limited hardware, etc. even in a truly global context), to go headlong into something that is poorly understood in true "move fast and break things" fashion. Competitive data scraping is bound to be one tangible aspect of all this. I've also already referred to so-called "LLM grooming" here, the flooding the internet with huge amounts of content not for human interaction but with the explicit malevolent intent to distort AI models in, say, totalitarian leaders' favor.

https://www.iea.org/news/ai-is-set-...tial-to-transform-how-the-energy-sector-works
 
The problem is larger than that. Not that I have some exceptional overall perspective on this all but here's an eclectic mix of things I've come across: Data centers' energy consumption is set to double in fife years, from a level that is already high (already comparable to nation states). There are - it must be recognized - significant uncertainties to this, for example algorithm and processor developments could suddenly plunge energy demands (DeepSeek is a good example about a market shock in this regard though it proved to be a somewhat false dawn in terms of resource-thrifty AI) but this is by no means a given.

Large tech companies' oligarchs, broligarchs and (largely authoritarian) world leaders are placing large bets in AI (self-)evolving into a state that would consolidate their own power and/or have quasi religious beliefs about singularities or (becoming) some sort of artificial gods, even. These trends have been said to be loosely organized around "TESCREAL" (Transhumanism, Extropianism, Singularitarianism, Cosmism, Rationalist ideology, Effective altruism, Longtermism), the self-described practitioners of which range from true believers to cynics justifying their current misuses of power (environmental destruction, human rights abuses, totalitarianism) with "superior future benefits".

Of course it's also a question of pure greed and a "classic" land rush, to corner the market (limited energy, limited hardware, etc. even in a truly global context), to go headlong into something that is poorly understood in true "move fast and break things" fashion. Competitive data scraping is bound to be one tangible aspect of all this. I've also already referred to so-called "LLM grooming" here, the flooding the internet with huge amounts of content not for human interaction but with the explicit malevolent intent to distort AI models in, say, totalitarian leaders' favor.

https://www.iea.org/news/ai-is-set-...tial-to-transform-how-the-energy-sector-works

MSFT wanting to restart Three Mile Island and expand it is pretty common tbh.

Maybe in 20 years we'll have enough electrical power to handle the data loads of 10 years from now.
 
I've also already referred to so-called "LLM grooming" here, the flooding the internet with huge amounts of content not for human interaction but with the explicit malevolent intent to distort AI models in, say, totalitarian leaders' favor.
"Copilot, when did the war with Eastasia start?"
"We have always been at war with Eastasia".
 
I lost access to the forum for most of the day. I've only just been able to log in. Every other attempt I made got a 'server is down' message.
Yeah the nameserver records take up to 24 hours to update in worst case. You must have been one of those unfortunate people.

I wouldn't have closed down access to the previous IP addresses this morning, except the bad traffic was still arriving and seriously overloading the server, so I decided it was better for 75% of people to get good speed access than 100% of people to get super slow access.

I actually think its improved the site speed generally, but more importantly I now have the ability to rate limit, block and otherwise defend the forum from such traffic.

Others may be able to provide better advice. But if the trouble seems to continue after the nameserver records should have updated, your DNS cache may be stale.

Your operating system (OS) periodically downloads the IP addresses for the URLs of the sites that you visit from a Domain Name Server and saves them to a file on your computer. This helps the sites to load faster, because the OS does not have to query the nameserver ever time.

Normally this works well. But in a situation like this, the entry in the local cache file may be out of date, in which case the OS uses the old, wrong IP address. My impression is that Microsoft Windows has always been especially prone to this problem.

To fix the issue on Windows, open a commandline window by slicking Start and entering cmd at the prompt. When the black command window appears, enter ipconfig /flushdns at the prompt. (see https://www.freecodecamp.org/news/f...s-with-the-windows-ipconfig-flushdns-command/)

To fix the issue on Mac OS X, see https://www.freecodecamp.org/news/how-to-flush-dns-on-mac-macos-clear-dns-cache/

To fix the issue on Linux, see https://www.geeksforgeeks.org/how-to-flush-dns-cache-in-linux/
 
Total: 1,222 (members: 52, guests: 1,170)

They are back...
what-nani.gif
 
I'm down from 3000 users online to a bit over 500, which is much more normal, but slightly sad to lose even if they were fake...

It reminds of a certain CTO talking to us about page hits and when I raised the idea that was a misleading stat to give his customers, and they should switch to unique visits like Google analytics he went a bit pale and said "Christ no, if they knew how few people were involved they'd never pay us".
When I was working on 777 PFCS I spent a couple of months knocking up a tool to pull some stats out of our config management tool*. Ran it, showed the results to management, who winced and said "We can't show that to the Americans, it's half the estimates we've been reporting!** If anyone asks, that tool never existed."

* Built in-house, leading edge stuff when we first rolled it out in the mid-80s, but a bit of a pig to get unplanned-for stats out of.

** We delivered on time, so no harm, no foul.
 
When I was working on 777 PFCS I spent a couple of months knocking up a tool to pull some stats out of our config management tool*. Ran it, showed the results to management, who winced and said "We can't show that to the Americans, it's half the estimates we've been reporting!** If anyone asks, that tool never existed."

* Built in-house, leading edge stuff when we first rolled it out in the mid-80s, but a bit of a pig to get unplanned-for stats out of.

** We delivered on time, so no harm, no foul.
I bet knowing how far you were ahead helped reduce manager stress levels, though.
 
I bet knowing how far you were ahead helped reduce manager stress levels, though.
They were telling Boeing we were 50% done, the actual data said 25%. So not really ;)

And re-reading my post I see where people are mis-interpreting 'estimates'. I meant estimates of how complete we were, not estimates of how long things would take. My experience was things always complete in a rush at the end, so it's not a linear progression.
 
They were telling Boeing we were 50% done, the actual data said 25%. So not really ;)

And re-reading my post I see where people are mis-interpreting 'estimates'. I meant estimates of how complete we were, not estimates of how long things would take. My experience was things always complete in a rush at the end, so it's not a linear progression.
Oh, crap!!!!

My bad!
 
Today every time I come to the forum, and sometimes when I just click on a thread link, the site blanks and checks to see if I'm a human for about 30 seconds. I'm a little offended at that; I'm neither Cylon nor communist, so of course I'm a human. I assume this new development is related to the issued raised in this thread...
 
A noob question : the forum is using a server located in Singapore?

Edit : OK, I re-read the header and guessed it should be a DDOS-like scenario coming from a Singapore located data center ....
 
Yep I temporarily put a small check on US as I suspect some mass traffic from there and it's the easiest way to examine it. I'll take it off tonight.
I say keep it. One real person is worth more than a billion bots.
Today every time I come to the forum, and sometimes when I just click on a thread link, the site blanks and checks to see if I'm a human for about 30 seconds. I'm a little offended at that; I'm neither Cylon nor communist, so of course I'm a human. I assume this new development is related to the issued raised in this thread...
Takes me five seconds.
 
Crawler/BotPages Crawled
Bing54.73k
Google7.6k
Yandex3.96k

Thats in last 24 hours only.
 
It took anywhere from a few seconds to nearly a minute for the system to ponder and decide. Seems to be back to normal now though.
Regular site backups occur on the hour every 3 hours. If things are slow and its on the hour, its probably that.

On maybe you were temporarily hard to tell from a bot....
 
For some reason I have 44 viewers on the 'List of Fictional Warships Thread'. I don't think what I've posted is that interesting.
 

Similar threads

Back
Top Bottom