Enabling WebDav for CDN [Building Coda file system support] using CentOS 5.2 Linux

Many people have asked me about setting up WebDav to directly address a CDN repository as a /local/path directory. This requires Coda file system (Advanced NFS) support. This particular tutorial is directed at people who wish to use WebDav support for a CDN such as BitGravity (as in this example) on a Linux system. The particular flavor of Linux used in this tutorial is CentOS 5.2, so if you are using a different form of Linux the procedure is similar but will vary from distro to distro.

Install the Necessary Packages:

To access the necessary packages in Yum, the RPM repository needs to be up to date. I used the following commands to clean up Yum:

rpm -e yum-plugin-fastestmirror
yum clean all
yum install yum-metadata-parser yum-updatesd

yum install yum-plugin-fastestmirror

Once added, run the following installations:

yum install davfs2
yum install dkms
yum install dkms-fuse
yum install fuse
yum install fuse-davfs2

Verify that the Coda File System is Supported in the kernel:

For the WebDAV share to mount correctly the module for the Coda file system needs to be supported by the kernel. To test if it is supported run the following command command:

dmesg | grep -i coda

If the system returns an output, then everything should be fine, and the configuration of WebDAV can continue. However, if it responds with nothing then the kernel does not support coda. To overcome this obstacle the kernel source tree will need to be built so that the necessary coda files can be extracted.

Build the Kernel Source Tree and Add Coda Support

By building the kernel source tree the necessary files for coda file system support can be manually compiled, and then copied into the current kernel’s library. In this way you don’t have to rebuild your running kernel.

During the most of the build procedure it is critical that it be performed as a non-root user to maintain the necessary permissions.

>> As root user or su -  <<

yum install kernel-devel
yum install kernel-headers
yum install rpm-build
yum install redhat-rpm-config
yum install unifdef

>> Exit su – (or sudo to a non-root user) and continue with the following steps! <<

After installing the packages log into the non-root user and navigate to ~ (the $HOME directory), and create the directory for the source:

mkdir -p ~/rpmbuild/{BUILD,RPMS,SOURCES,SPECS,SRPMS}
echo ‘%_topdir %(echo $HOME)/rpmbuild’ > .rpmmacros

Type “uname -ar” to display the kernel version and then install the respective kernel source. For example:

rpm -i http://mirror.centos.org/centos/5/updates/SRPMS/kernel-2.6.18-92.1.18.el5.src.rpm 2> /dev/null

Now spec the kernel module version for this build with:

rpmbuild -bp –target=`uname -m` kernel-2.6.spec 2> prep-err.log | tee prep-out.log

After installing the kernel source tree navigate to the install location and start the build process:

cd ~/rpmbuild/BUILD/kernel-2.6.18/2.6.18.x86_64/
make oldconfig

Accept the default values [especially if you aren't sure].

make menuconfig

Browse the source tree menus to ‘File Systems’ –> ‘Network File Systems’ –> ‘Coda File System’. Type “m” for the value, which specifies modular support for coda.

Continue the build process with these commands:

make prepare
make modules_prepare
make M=fs/coda

With the build steps completed the fruits of labor can be plucked from the kernel source tree:
>> As the root user, or su – <<

cp ~/rpmbuild/BUILD/kernel-2.6.18/2.6.18.x86_64/fs/coda/coda.ko /lib/modules/2.6.18-53.1.6.el5/updates/

And at last, coda module installation can be initiated:

depmod -a

modprobe coda

Check that the module started correctly with :

lsmod | grep coda

Now we want to be sure the Coda module starts at boot time so add this line to /etc/modules.conf

probe coda

Configure WebDAV

Once all packages are installed navigate to /etc/davfs2 and edit two configuration files. The configuration for BitGravity’s WebDAV will be used as an example.

By default all the options are commented out on /etc/davfs2/davfs2.conf. For WebDAV to work properly for Deca’s setup the following lines need to be un-commented, and edited as such:

# General Options

# —————

dav_user        davfs2            # system wide config file only
dav_group     users             # system wide config file only

kernel_fs     coda

# WebDAV Related Options # ———————-

use_locks 0

# Debugging Options # —————–

debug most           # possible values: config, kernel, cache, http, xml,
#      httpauth, locks, ssl, httpbody, secrets, most

And add the following line to the Credential Line section of the secrets file:
[Adjust as per your own particular CDN. This line was for BitGravity but check with your CDN provider to be sure what your credentials are before proceeding.]

http://webdav.bitgravity.com/   userid@yourdomain.com  password

Next, make a directory for the WebDAV share to mount:

mkdir /davtest

Create dav2fs user:

useradd dav2fs [make sure you choose nologin, and no shell]

Check /etc/passwd to be sure the user dav2fs has no permissions on the system [/sbin/nologin should be the shell interpreter]:

davfs2:x:101:102:davfs2:/var/cache/davfs2:/sbin/nologin

Give the new user ownership of the WebDAV mount

chown -R dav2fs.users /davtest

The final file to edit is /etc/fstab, in which you will add a line like this:

http://webdav.bitgravity.com/       /davtest     davfs   uid=dav2fs,gid=users,dir_mode=755    0 0

To initiate the mount run:

mount -a

Test that you have everything correct with:

touch /davtest/test.txt
ls -l /davtest/test.txt

You can manually mount the share for troubleshooting purposes with:

mount.davfs -o dir_mode=755,uid=dav2fs,gid=users http://webdav.bitgravity.com/ /davtest

If there is a problem check /var/log/messages!

Using Perl -e from command line

Often times in systems administration you will have to perform tasks which can seem extremely tedious. An example of this is replacing every instance of a word in text files. At times like this you can get a lot of help from Perl, or “Practical Extraction and Reporting Language”. Perl is well suited to help us in cases like this.

One possible way to handle the problem would be to copy the file from the server, and use some form of text editor to replace every instance of a word, and then save the modified file back to the server. There’s an easier way. Perl -e from command line!

The -e switch tells Perl to execute anything between two ‘ marks. For example:

Using Perl’s substitution operator in combination with this can be a very useful tool! For example, lets say you have a text file that you wish to replace every instance of the word “cat” with the word “dog”. Using Perl -e from command line with the substitution operator is one possible way to do this rapidly.

For example, in Perl to substitute cat with dog in the current line of text, you would use the following code:

s/cat/dog/gi, $_;

This says “Substitute the word dog for the word cat in every line of text we process”. The /delimits the search, and terminates it. The positional will be switched at run time, so three / marks are used. One to begin the match, one to indicate the end of the first match word, and the beginning of the replacement word, and then a third / mark terminates the search string. The “g” and “i” switches used just after the search string terminator tell Perl to “replace this word regardless of position in the line (g=global)”, and to “ignore case so that upper and lower case words (or a combination thereof) will be replaced as well”. Remember that when doing case sensitive replace operations, the previous case of the word will be destroyed in favor of the new word. For example, both “CaT” and “cAt” would be replaced with “dog” in the above example.

Here is a live example for you. Lets say we wish to replace all instances of the word cat with the word dog in a textual file:

Now we’ll use Perl -e from command line to replace cat with dog:

The command I used was:

cat animal.txt | perl -e ‘while(<>) {s/cat/dog/gi, $_; print;}’

Notice everything in the scripted section of the command line is encompassed between two ‘ marks. What the while(<>) { statement does is tells Perl “while processing input from STDIN do….”. In the command the $_ variable is a “default variable” telling Perl to perform the operation on the current line of text being processed. Since we are always using the default, we can eliminate that variable and let the Perl interpreter do the work. The syntax in the next example will demonstrate.

To replace instances of the word in a file this method can by used with “file redirection” to create a new file. For example to replace all instances of the word cat with dog, and create a new “dog file” you would use

cat animal.txt | perl -e ‘while(<>) { s/cat/dog/gi; print;}’ > dog.txt

Be careful using data redirection! You can clobber yourself, so check your syntax carefully!

> will OVERWRITE

>> will APPEND

Perl can be used in many such ways as a powerful tool for systems administrators. Perl from the command line brings all of the capabilities of perl to filtering and processing data from the command line. Check back soon for more tutorials regarding using Perl from command line, including database interaction!

Linux and WiFi

Configuring a wireless networking card under Linux can sometimes be a hassle. Most hardware vendors are not yet advanced enough to offer multi-platform drivers, and the need for wireless networking in Linux has grown substantially as the OS gains favor in the home user environment.

While many distributions of Linux support wireless networking cards, as well as extended USB support, some installations can be a little counter-intuitive when it comes to getting the hardware installed and working. For this reason a good recommendation is Mandriva Linux (2008.1 Spring / One). Mandriva has done even more than before to expand their already robust hardware support, and a Linksys USB N type wireless network adapter installed with little or no trouble at all.

It is important to point out that Mandriva configured the wireless card through the USB core, and NOT through NDISWrapper. This is important because it is a native driver base, and not an emulator. The one caveat to the installation was getting the network connection manager to see the previously installed adapter. There are two easy ways to accomplish this.

Plug in the Linksys N card to the USB port.

Insert your Linksys driver disk in the cdrom, and mount it, or note if it is auto-mounted in /media.

If you are running a desktop, such as KDE or Gnome, then you will go through the “System”…”Applications” menu to “Configure your computer” (Control panel). From there, you will click on “Add new network connection”, and select “Wireless” as the type. When prompted for the driver, point the wizard to the Linksys factory Windows XP driver. In this case, the path was /media/WUSB300N/Drivers/XP/*.inf

At this point if your system as correctly installed initially, it should recognize the USB core driven devices such as the Linksys N card, and should prompt you with the message “The device you are attempting to configure is already configured using the USB core. Do you really want to use NDISWrapper?”

Click NO/Cancel

At this point a “service network restart” from a terminal (command line), or starting the control panel “Configure network connections” applet should allow you to see the working adapter. If it did not immediately refresh, you might consider a reboot if you are a novice, so that the system should init the wireless WLAN0 interface on start-up. If you are an experienced user you should be able to simply refresh the networks list in the wireless control interface in the “Start Bar” icons.

Services

We offer engineering, troubleshooting, design, private instruction, and consultation in the following areas:

* Linux Systems and Networks Administration
* Microsoft Networks Administration
* High Availability – Load Balancing and Cluster Nodes
* Network Engineering
* Network Security – Firewalls, Global ASA, Penetration Testing
* RF Engineering – Performance Reporting Only
* Electronics Engineering – Including prototyping
* Programming – Such as Perl, PhP, C, and many other types…
* eCommerce – Solutions and Integration
* Database Administration
* Server Administration – WWW, httpd, Apache, Proxies, HA, CAN, IIS, SQL
* CMS – Web Content Management Systems – Design and Integration
* Open Source Projects
* Data Mining – Data Object Orientation and Programming, Database Integration
* Business Process
* Cable Management
* Structured Cabling
* Fiber Optic Networks
* Ether Networks
* Microwave IP Networks, and P2P Microwave Links, Link Projections
* Project Management

Please contact us, and someone will get back to you (usually within 48 hours).

Contact

Inner Technical

1. First Name(required)
2. Last Name(required)
3. Company(required)
4. Email Address(valid email required)
5. Website(required)
6. Tell us how we may help:Type your question or request here...(required)
7. Verification:

 

Information Technology Consulting

Our services include:

• Linux/Unix Administration and consulting.
• Windows Administration and consulting.
• Network Administration and consulting.
• Database design and implementation (LAMP, MySQL, Postgresql, MSSql)
• Communications Networks – Engineering, design, and structured cabling/fiber.
• WWW Servers (Apache / IIS) installation and troubleshooting – ongoing administration.
• E-Commerce (open source e commerce platforms and shopping carts.)
• Programming (PhP, Perl, Ajax, Java, C, C++, etc.)
• Web Design.
• CMS – Content Management Systems such as Wordpress MU, Drupal, etc..
• Graphics Design.
• Project Management.

Next Page »