Combined Intelligence – Taking on Google takes more than just a few crufty brain pans!

[rant]

Sometimes I like to do a little reading on the web. In many cases it is a quick and easy way to kill a few minutes, and I don’t have to keep track of a hardbound book. I was inspired to write this article by what I read, hoping perhaps you will enjoy this reading as well. Anyway, I’m reading through the web in one window, and tailing apache logs in another. Suddenly I notice some systems displaying tar-pit like messages:

TCP: Treason uncloaked! Peer 38.108.180.95:31794/80 shrinks window 344798477:344803373. Repaired.
TCP: Treason uncloaked! Peer 38.108.180.95:31794/80 shrinks window 344798477:344803373. Repaired.

This is typically a mis-configured interface, or perhaps a mis-behaving switch. However, in totally clean cluster environments like the one I was looking at, I usually give this some follow through as I should not be getting window scale corruption. A quick look at the ns records refers me to “name = h-180-95.scoutjet.com”. Curious about what they are doing, I opened up the browser and read the little spider page.

http://www.scoutjet.com

I decided to take a closer look, after all its every day that some foreign cracker will spoof IP addresses of idiotic tech companies just to get double desert. A quick look and what do you know!? The offending IP is not spoofed, and does in fact reside within their netblock:

%rwhois V-1.5:0010b0:00 rwhois.cogentco.com
38.108.180.95
network:ID:NET-266CB40018
network:Network-Name:NET-266CB40018
network:IP-Network:38.108.180.0/24
network:Postal-Code:94065
network:State:CA
network:City:Redwood City
network:Street-Address:100 Marine Parkway
network:Org-Name:Blekko
network:Tech-Contact:ZC108-ARIN
network:Updated:2008-09-17 10:42:07
network:Updated-by:John Knowles

First thing that crossed my mind was why does such an innocent looking spider page come up while investigating a potential TCP datagram corruption situation?

So, reading on a little I see these fellows want to take on Google! This altered my thinking rather quickly. If you are going to take on Google saying you are building a search engine that will outperform theirs (and say that Google is “going down”), you should probably at least have a very public debut of your web search, some rallied public support, and perhaps a few hard metrics! Otherwise, it gives the tech savvy reader much the same impression I got: “This sounds like a bunch of geeks who hatched some plan at a pot party, and are now scouring the web for attention”.

Moving forward, I see I was most likely correct: http://www.techcrunch.com/2008/01/02/the-next-google-search-challenger-blekko/

Now, I normally won’t rag on people, especially not publicly, but here’s the part where I start getting annoyed with society today. At what point do people become so naive that they will launch a publicly printed article based on so much freeze dried crap? The part that really stuck in my craw was this -> http://www.crunchbase.com/person/rich-skrenta {}

I’m sorry Rich, been involved with computers since the early 1970’s, and despite the clarity of your ego you did NOT invent the first “self-spreading” virus. The FIRST ever computer virus was named Creeper, and it was created back in the early 1970s by a fellow named Bob Thomas. At the time the Internet didn’t exist as we know it today, it was referred to as ARPA-NET which was one of the many extra-nets used by the medical, military, and scientific communities to rapidly share everything from medical prescriptions to scientific data sets. Though Bob’s virus was done as a test, and only affected VAX type computers, it none-the-less was created while Rich was still just a wee tot.

The first “wild” Trojan was called “Animal” (named for the pervasive way it would copy to any connected equipment, including disk drives) came out shortly thereafter in the mid 70’s.

Now credit where credit is due, Mr. Skrenta did create the Elk Cloner virus as he claimed in 1982, however lets reveal the truth behind the myth of this particular virus (of which I still have a copy on a library of intentionally infected disks, even though I don’t have a living Apple II to run it on). Creating a boot virus for the Apple II is hardly something I’d try to stake a claim to fame on…

A virus is nothing more than a malicious computer program which typically harbors some ability to copy itself to other components of the system. So, creating a simple computer virus? Not a daunting task. Its (once again) QUITE simply a MALICIOUS COMPUTER PROGRAM. Considering people have been programming with computer languages since Bell Labs in the 1950’s, this hardly seems like a feat.

That being said Elk Cloner was not really so much of a feat at all. In fact, other similar mal-ware was created, just never unleashed on the public (which seems a lot like a high school prank). The Apple II was widely known by experts of the time to be very susceptible to intrusion, particularly with respects to the disk drives. Since Apple II was reliant on floppy disks to run it seemed like a no brainer that copying malicious data on the source disks would certainly allow for operating system corruption on subsequent systems that booted from that same floppy.

Secondarily, the Apple II was also widely known for its lack of boundaries with boot strapping. A “Boot strap” simply refers to a small set of functions coded into the disk startup that allow user driven executable code to be launched into initial memory on a system. While the straps typically contain only logical volume data to be used in sizing up the geometry metrics of the installed hard disk, and launching the disk’s start-up programming, the Apple II boot routines were widely known to be easily extensible through a lack of byte counts on the strap. In English, this type of malicious coding will not work on any computer that’s less than 25 years old, and really just knowing you can modify the boot sector or write to unallocated memory of the boot areas is more than half the battle.

To prove my case, my good friend Donald Judd and I wrote a simple virus while we were at Arizona Tech that was intended as “proof of concept” that Windows operating systems of the day lacked any sanity when it came to intrusion of their operating environment. The really cool part, is we wrote it using nothing more than Debug.exe (a system included debugging/memory editing utility), and did it in less than 10 lines! In fact the entire virus was only 8 bytes in size. For those of you who are familiar with the environment, here’s the concept:

Simple put: Read the A register in memory. Move the AX register to value 0, call interrupt 19 (hard reboot), read the difference between the registers (8 bytes), write the file.

Now, this is just proof of concept stuff. Interrupt 19 was a hardware interrupt which signaled the system to restart (regardless of circumstances). While this code could not copy itself to a boot sector, it does demonstrate that writing a malicious computer program is simply that: Writing a malicious program. In fact interrupt 19 went away when the inception of software based interrupts within Windows in the late 90’s, and running it on a modern system will result in nothing more than a “protected memory” error (if anything at all…you’d have to run it from %comspec% to get the error):

In other words lots of people out there started exploiting Microsoft’s inability to control protected memory in a sane way, and MS learned from it. Anyway, the point is this code was eventually discovered and exploited by the people who wrote virii like “Melissa” and “Love”. In fact we experimented creating DOS boot type virii which would do non-destructive things like cascade ASCII smiley faces at you while your computer boots.

This article seems a lot like a rant, and well, I guess it is. After reading self edited wiki entries about Mr. Rich I decided it was high time that someone stepped up, and told the rest of the world that are NOT computer savvy, DON’T BE IMPRESSED with tomfoolery! Computing is a combined intelligence, and not one single stand alone person in the world possesses the knowledge to completely rebuild a modern computer in the event all the technology somehow went away. In other words, if there was a global holocaust, and all the computers and supporting archives were destroyed, there’s not a SINGLE person in the world that could recreate what we think of as a computer by themselves! Computing is a COMBINED INTELLIGENCE, and as such, there will always be someone smarter out there.

Don’t let someone take fame from a high school prank that isn’t really all that original, and use it to rally esteem which would cause one to think such a person can take on Google. If you think I’m kidding, look at what Google represents. The literal definition of the math phrase the search engine took their name from is:

“A googol is the large number 10100, that is, the digit 1 followed by one hundred zeros (in decimal representation). The term was coined in 1938 by Milton Sirotta (1929–1980), nephew of American mathematician Edward Kasner.”

To take them on you’ll need a lot more than a few geeks with a funny paper bag on their head, or pictures of some geek brainstorming session coupled with someone who self-proclaims to the world they are superior because they created a malicious computer program. You’ll need someone smarter than them. Smarter than me, perhaps even smarter than Kevin David Mitnick. You’ll need Steven Hawkings on acid! You’ll need….a googleplex of brains. In short, you’ll need COMBINED INTELLIGENCE!

[/rant]

Enabling WebDav for CDN [Building Coda file system support] using CentOS 5.2 Linux

Many people have asked me about setting up WebDav to directly address a CDN repository as a /local/path directory. This requires Coda file system (Advanced NFS) support. This particular tutorial is directed at people who wish to use WebDav support for a CDN such as BitGravity (as in this example) on a Linux system. The particular flavor of Linux used in this tutorial is CentOS 5.2, so if you are using a different form of Linux the procedure is similar but will vary from distro to distro.

Install the Necessary Packages:

To access the necessary packages in Yum, the RPM repository needs to be up to date. I used the following commands to clean up Yum:

rpm -e yum-plugin-fastestmirror
yum clean all
yum install yum-metadata-parser yum-updatesd

yum install yum-plugin-fastestmirror

Once added, run the following installations:

yum install davfs2
yum install dkms
yum install dkms-fuse
yum install fuse
yum install fuse-davfs2

Verify that the Coda File System is Supported in the kernel:

For the WebDAV share to mount correctly the module for the Coda file system needs to be supported by the kernel. To test if it is supported run the following command command:

dmesg | grep -i coda

If the system returns an output, then everything should be fine, and the configuration of WebDAV can continue. However, if it responds with nothing then the kernel does not support coda. To overcome this obstacle the kernel source tree will need to be built so that the necessary coda files can be extracted.

Build the Kernel Source Tree and Add Coda Support

By building the kernel source tree the necessary files for coda file system support can be manually compiled, and then copied into the current kernel’s library. In this way you don’t have to rebuild your running kernel.

During the most of the build procedure it is critical that it be performed as a non-root user to maintain the necessary permissions.

>> As root user or su -  <<

yum install kernel-devel
yum install kernel-headers
yum install rpm-build
yum install redhat-rpm-config
yum install unifdef

>> Exit su – (or sudo to a non-root user) and continue with the following steps! <<

After installing the packages log into the non-root user and navigate to ~ (the $HOME directory), and create the directory for the source:

mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
echo ‘%_topdir %(echo $HOME)/rpmbuild’ > .rpmmacros

Type “uname -ar” to display the kernel version and then install the respective kernel source. For example:

rpm -i http://mirror.centos.org/centos/5/updates/SRPMS/kernel-2.6.18-92.1.18.el5.src.rpm 2> /dev/null

Now spec the kernel module version for this build with:

rpmbuild -bp –target=`uname -m` kernel-2.6.spec 2> prep-err.log | tee prep-out.log

After installing the kernel source tree navigate to the install location and start the build process:

cd ~/rpmbuild/BUILD/kernel-2.6.18/2.6.18.x86_64/
make oldconfig

Accept the default values [especially if you aren't sure].

make menuconfig

Browse the source tree menus to ‘File Systems’ –> ‘Network File Systems’ –> ‘Coda File System’. Type “m” for the value, which specifies modular support for coda.

Continue the build process with these commands:

make prepare
make modules_prepare
make M=fs/coda

With the build steps completed the fruits of labor can be plucked from the kernel source tree:
>> As the root user, or su – <<

cp ~/rpmbuild/BUILD/kernel-2.6.18/2.6.18.x86_64/fs/coda/coda.ko /lib/modules/2.6.18-53.1.6.el5/updates/

And at last, coda module installation can be initiated:

depmod -a

modprobe coda

Check that the module started correctly with :

lsmod | grep coda

Now we want to be sure the Coda module starts at boot time so add this line to /etc/modules.conf

probe coda

Configure WebDAV

Once all packages are installed navigate to /etc/davfs2 and edit two configuration files. The configuration for BitGravity’s WebDAV will be used as an example.

By default all the options are commented out on /etc/davfs2/davfs2.conf. For WebDAV to work properly for Deca’s setup the following lines need to be un-commented, and edited as such:

# General Options

# —————

dav_user        davfs2            # system wide config file only
dav_group     users             # system wide config file only

kernel_fs     coda

# WebDAV Related Options # ———————-

use_locks 0

# Debugging Options # —————–

debug most           # possible values: config, kernel, cache, http, xml,
#      httpauth, locks, ssl, httpbody, secrets, most

And add the following line to the Credential Line section of the secrets file:
[Adjust as per your own particular CDN. This line was for BitGravity but check with your CDN provider to be sure what your credentials are before proceeding.]

http://webdav.bitgravity.com/   userid@yourdomain.com  password

Next, make a directory for the WebDAV share to mount:

mkdir /davtest

Create dav2fs user:

useradd dav2fs [make sure you choose nologin, and no shell]

Check /etc/passwd to be sure the user dav2fs has no permissions on the system [/sbin/nologin should be the shell interpreter]:

davfs2:x:101:102:davfs2:/var/cache/davfs2:/sbin/nologin

Give the new user ownership of the WebDAV mount

chown -R dav2fs.users /davtest

The final file to edit is /etc/fstab, in which you will add a line like this:

http://webdav.bitgravity.com/       /davtest     davfs   uid=dav2fs,gid=users,dir_mode=755    0 0

To initiate the mount run:

mount -a

Test that you have everything correct with:

touch /davtest/test.txt
ls -l /davtest/test.txt

You can manually mount the share for troubleshooting purposes with:

mount.davfs -o dir_mode=755,uid=dav2fs,gid=users http://webdav.bitgravity.com/ /davtest

If there is a problem check /var/log/messages!

About InnerTechnical.com

InnerTechnical.com was founded by a group of individuals whom, at the time of inception, were working for companies that had some very specific needs and no way to fill them. From Linux Networks and Systems Administration, to High Availability Server Platforms like Reverse-Proxy, to RF Engineering, to Dark Fiber. It became apparent there was a gross need in American corporate infrastructure for greater organization, and consolidation where IT is concerned. Especially with so many companies out there taking the bait on “vaporware”.

Most of these companies had the demand, but there was some “black box”, or “legacy network”, or “software expense” that kept the need from being served.

InnerTechnical.com’s goal is to facilitate filling that need by providing experts in that particular form of technology who can analyze your situation, make safe/sane recommendations, perform network design and engineering based around your needs or existing model, and give businesses with high technology needs the tools they need to eliminate their problems. Machines and technology should work for people, not the other way around.

Our S.M.E.s (or Subject Matter Experts) are comprised of a group of the top minds in the tech fields. Each of them is very experienced on a broad scale with technology, and each of them has a highly trained focus in the school of thought in which they operate.

Best practices should never be ignored, even when they are extremely tedious or seemingly not needed (until perhaps later down the line). Many people will take the “easy way” or the “fast way” over the “right way”. The problem with this strategy should be apparent, particularly in technology realms, because its almost a sure recipe for disaster later on. Many “short cuts” will introduce logic bombs, and system interdependencies which in technology can be a very bad thing because if one system goes down, they ALL go down.

One bad apple really can spoil the whole bunch, so let us spot your bad apple for you, and show you how to make a little cider!