Necessary and Sufficient

I love it when people explain complex topics in simple terms. Not because I'm a simpleton but because it makes the knowledge more accessible to the wider audience who really ought to know more about this stuff.

With the "Rich vs Reach" debate still raging between desktop and web developers (and firms with a vested interest in either. i.e. Microsoft and Google), the Google Chrome team put together this nice little booklet to convey information about web-delivered applications. Sure, they have a very strong preference for web apps but it is a fairly impartial summary.

Put simply web apps are moving forward in leaps and bounds because of many great innovations:

• Developer tools like GWT (Java to JS) and WebSharper (F# to JS) are being created to allow developers to generate pesky JavaScript code from other higher-level languages, to accommodate folks who aren't JavaScript fans. This enables much easier development of web apps for those who are from a desktop development background;
• Modern browsers, like Chrome (V8), FireFox and IE9 (Chakra), have much smarter JavaScript engines that compile rather than interpret JavaScript. This makes JavaScript code run a heck of a lot faster than it use to;
• The slowest parts of web page rendering: image downloads and DNS resolution, are being addressed by new image compression techniques, like WebP, and browser features like DNS pre-fetching;
• The new HTML5 standard includes many new features like the canvas tag, the video tag, and web sockets that make for a much richer in-browser user experience; and
• New techniques help minimize the latency associated with server round-trips that typified web apps for many years. Currently, developers resort to Ajax and Comet techniques to hide or minimize the latency but Comet is hard to get right across all browsers. The HTML 5 WebSocket represents a standardized way to address the problem that has been targeted by Comet and Ajax in that it defines a single-socket full-duplex/bi-directional connection for pushing/pulling information between the browser and server. Thus, it avoids the connection and portability issues of Comet and provides a more efficient solution than Ajax polling.

The next 5 years is going to see some really cool web applications that are installation-free, have "desktop-like" user experience, and are backed by on-demand super computers. i.e the cloud. I can't wait to see where it goes from here!

Today I noticed that Google Chrome and Firefox were both telling me my website was a malware source, because it had been redirected to a URL at webarh.com (the nasty people) - obvious from the URL shown. This is a Red Hat Linux setup using php, mySQL and Apache, and here's what I did to diagnose and fix the problem, in case you get infected to...

First thing I did was figure out if it was a DNS hijack or a website attack. This can be done by resolving the domain name and ensuring it is the public IP of your web server.

[root@red3 /]# nslookup www.necessaryandsufficient.net
Server:         61.9.195.193
Address:        61.9.195.193#53

Non-authoritative answer:
www.necessaryandsufficient.net  canonical name = necessaryandsufficient.net.
Name:   necessaryandsufficient.net
Address: 121.210.117.191

[root@red3 /]# curl -s myip.dk | grep "IP Address" | egrep -o '[0-9.]+'
4
121.210.117.191
6
[root@red3 /]#

Since they both say 121.210.117.191, this indicates that it isn't a DNS attack. Something on the website is screwing things up. We can confirm this with another curl command...

[root@red3 /]# curl -i www.necessaryandsufficient.net | more

In the infected state you will see a 302 (redirect) to the nasty URL on webarh.com in the output to the command above. So something in the website config is redirecting. Given this is Apache with mod-rewrite perhaps someone has managed to setup new redirects. Let's do some simple string matching first up...

[root@red3 /]# grep -r "webarh" /var/www/

This threw up a bunch of .htaccess files that had redirect instructions to the nasty webarh.com URL. Killing those is the first step. Trying the site again, and the problem still wasn't fixed. There must be more nasty redirects lurking elsewhere, so I looked more closely to the output of the above command and saw a bunch of index.php files had some Javascript that was doing a redirect in code. Setting document.location.href = "whatever" tells the browser to go to the URL you specified (albeit with a depricated command), so you have to deftly trim out those nastry script tags with your favourite text editor.

Tried the website again and now the home page comes up but the links, Wordpress use seo-friendly SLUGs, don't work. Clearly this is because we've lost some of the URL rewrite rules from the base directory of the website. Luckily I have backups and that file hasn't changed in ages (until now) so a simple restore fixed that.

Things seem back up and running now, so let's figure out what caused all this in the first place and how to patch it. I did some Googling first then dived into the web server logs to confirm something I'd read about a phpMyAdmin security hole...

[root@red3 logs]# cd /usr/local/apache2/logs
[root@red3 logs]# grep "/scripts/setup.php" access_log

This threw up a massive scan (GET ...) of setup.php files for older versions of phpMyAdmin confirming what I'd seen posted about this attack. Most HTTP GETs had 500 "internal server" errors, but the file mentioned above returned a HTTP 200 code and the attacker posted to this file (see last 2 lines of output below).

125.16.9.12 - - [24/Nov/2010:08:00:52 +1100] "GET /phpMyAdmin-2.8.0-beta1/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:00:53 +1100] "GET /phpMyAdmin-2.8.0-rc1/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:00:53 +1100] "GET /phpMyAdmin-2.8.0-rc2/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:00:53 +1100] "GET /phpMyAdmin-2.8.0.1/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:00:54 +1100] "GET /phpMyAdmin-2.8.0.2/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:00:56 +1100] "GET /phpMyAdmin-2.8.0.4/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:00:56 +1100] "GET /phpMyAdmin-2.8.0/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:00:58 +1100] "GET /phpMyAdmin-2.8.1/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:01:21 +1100] "GET /pma/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:01:23 +1100] "GET /roundcube/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:01:23 +1100] "GET /scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:01:24 +1100] "GET /sl2/data/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:01:27 +1100] "GET /sqlmanager/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:01:27 +1100] "GET /sqlweb/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:01:27 +1100] "GET /typo3/phpmyadmin/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:01:27 +1100] "GET /web/phpMyAdmin/scripts/setup.php HTTP/1.1" 500 546
125.16.9.12 - - [24/Nov/2010:08:01:28 +1100] "GET /web/scripts/setup.php HTTP/1.1" 500 546
72.167.45.134 - - [26/Nov/2010:01:37:02 +1100] "GET /phpMyAdmin/scripts/setup.php HTTP/1.1" 200 13723
72.167.45.134 - - [26/Nov/2010:01:37:03 +1100] "POST /phpMyAdmin/scripts/setup.php HTTP/1.1" 200 24975
92.240.68.95 - - [26/Nov/2010:03:22:27 +1100] "GET /phpMyAdmin/scripts/setup.php HTTP/1.1" 200 13744
92.240.68.95 - - [26/Nov/2010:03:22:28 +1100] "POST /phpMyAdmin/scripts/setup.php HTTP/1.1" 200 13744

Essentially, the setup file for phpMyAdmin allows arbitrary PHP code injection, and the attacker exploited this to inject all these nasty redirects. See this article for more information.

You could upgrade phpMyAdmin but I don't use it much so it's gone. What a PITA.

As Christmas is approaching I was thinking about my year in review and, more importantly, putting some thought into what is likely to happen over the next few years in the Technology space. Why? Because having a view of the future helps you chart your current course to take as much advantage as possible of that vision.

"Talent hits a target no one else can hit; Genius hits a target no one else can see.” ...Arthur Schopenhauer (1788- 1860)

In case you're interested here's my predictions for 2011-2012 (in no particular order):

1. Because of economics, cloud computing will become pervasive, radically disrupting the way software is delivered and sold; and will enable much more computationally-challenging problems to be addressed by innovative companies

2. Developers will eventually morph their application designs to fit the scale-out model that is on offer via cloud computing, which invariable means horizontally-scalable databases (think NoSQL) and divide-and-conquer parallel processing libraries like MapReduce and Hadoop

3. Ubiquitous, high-speed wireless will allow us to become an "always-connected Planet" making issues like data portability, and multi-device support more important

4. Because of their lower price point, and greater household density, mobile phones / tablets will be the dominant "connected device" in the marketplace

5. Open mobile ecosystems based on Linux, like Android, will overtake closed ecosystems like iOS/iPhone

6. Desktop applications will become even more dis-intermediated via advances in web standards, like HTML5, and supporting tools

7. There will be some convergence in the "rich vs reach" argument as desktop apps push their data to the cloud, and web apps add features to mimic common desktop patterns

8. The "real-time web" (communication, search, analytics) will continue its rise in prominence with technologies like XMPP, Reverse-AJAX, node.js, Operational Transformation, etc being joined by lots of new innovations in this area

9. Social-transactions (getting recommendations from your friends rather than strangers) will become more prevalent due to the increase of online social networking

10. Data volumes hoovered up by companies will grow enormously putting a premium on extracting value from the data, moreso than the technical challenge of storing it

11. Helped by Metcalf's Law, the online games industry will continue to grow dwarfing the movie business and representing a large percentage of younger peoples time on computers and phones

12. With multi-core machines become the norm and GPU video cards offering additional processing power, developers will turn to languages with functional elements to simplify concurrency issues. Existing languages like C/C++, Java, C# have capability to create and manage threads but there is a lot of onus on the developer to perform synchronization and lock management that is avoided with Erlang, F#, and Scala.

13. The fight for the loungeroom has only just begun. All the major players, Google /Microsoft /Apple, want to own your set-top box. Watch out for fierce rivalry in that space as more media companies and content-providers partner with distribution providers like YouTube who in turn need to augment their user-generated content repositories with some "hits" to go with the "long tail".