An Intro to robots.txt

Intro

The robots.txt protocol also known as Robot Exclusion Standard or Robots Exclusion Protocol basically defines how web spiders and crawlers (search indexing tools) have access to your site.
The Web Administrator can Allow or Disallow access to certain parts of the site, e.g. temporary, private, cgi directories. It is a plain text file, where the rules for the these spiders, crawlers, where they can go and where not. You can also typically block a folder e.g. /images, because you find it both meaningless and a waste of your site’s bandwidth.

Syntax of robots.txt

# All comments is followed by a the pound sign
User-agent : [crawl agent name or * for all]
There are currently three statements you use in robots.txt

1. Disallow: /path
2. Allow: /path
3. Sitemap: http://example.com/sitemap.xml

Note:

• Crawl-delay: [a number that represent the time in seconds] is not a supported standard, but some bots do obey it.
• Each crawler user agent section must be separated by a blank line.
• Not all search engines respects some pattern matching. E.g. to block any directory name that begins with the name picture: Disallow: /picture*/
• It is recommended if you use multiple User-agents, that the block of code addressed to all spiders (User-agent: *) is the last one. This is to reduce interpretation problems with some old robots.
• Directives are case-sensitive. E.g. junk.html and Junk.html are seen as two different files.

User-agents

At User-Agents.Org is a very good list of these bots.
At the Web Robot’s page you can see a list of common robots – User-agent names.

Blocking

Blocking the entire site:
Disallow: /

Blocking a specific directory, images, and it’s contents:
Disallow: /images/

Blocking a specific file:
Disallow /junk.html

Blocking a specific file type:
Disallow: /*.xls$

Examples:

# Disallowing all robots

User-agent: *
Disallow: /

# Blocking only one robot, e.g. Altavista’s Scooter and allow all others

User-agent: Scooter
Disallow: /

User-agent: *
Disallow:

Now this is something we learned out of trial and error. In this example it looks like if we want to block MSNBot from a certain directory, and also disallow all other bots (including MSNBOT) from some more directories and give them, except for MSNBOT, access to the directory junk.
THIS DOES NOT WORK LIKE THAT!

User-agent: MSNBot
Disallow: /junk/

User-agent: *
Disallow: /cgi-bin/
Disallow: /private/

You need to disallow the directories for all the bots (User-agent: *) also for MSNBot!!! It should be:

User-agent: MSNBot
Disallow: /junk/
Disallow: /cgi-bin/
Disallow: /private/

User-agent: *
Disallow: /cgi-bin/
Disallow: /private/

Validation:

Here is a few sites, where you can validate your robots.txt file:

• http://tool.motoricerca.info/robots-checker.phtml
• http://www.invision-graphics.com/robotstxt_validator.html

Happy coding

Manual Uninstall McAfee VirusScan 7.1 on Windows XP

For some strange reason you do not have the options to uninstall the program from the Add/Remove Control Panel. You cannot find an uninstall option under the menu. You can use the command prompt to do it.

Note: You need Administrative rights to perform these steps.

From the Start Menu select:

• Run
• Type cmd
• Click OK or press Enter
• Type: net stop McShield
• Type: net stop McTaskManager
• Type: net stop McAfeeFramework
• Type: msiexec /x {59224777-298D-4E9C-9AEB-4A91BDA01B27} REMOVE=ALL REBOOT= R /q
• Type: exit

Esta Bom! That’s It!

My name is Rose

The first day of school our professor introduced himself and challenged us to get to know someone we didn’t already know. I stood up to look around when a gentle hand touched my shoulder.

I turned around to find a wrinkled, little old lady beaming up at me with a smile that lit up her entire being.

She said, ‘Hi handsome. My name is Rose. I’m eighty-seven years old. Can I give you a hug?’

I laughed and enthusiastically responded, ‘Of course you may!’ and she gave me a giant squeeze.

‘Why are you in college at such a young, innocent age?’ I asked.

She jokingly replied, ‘I’m here to meet a rich husband, get married, and have a couple of kids…’

‘No seriously,’ I asked. I was curious what may have motivated her to be taking on this challenge at her age.

‘I always dreamed of having a college education and now I’m getting one!’ she told me.

After class we walked to the student union building and shared a chocolate milkshake.

We became instant friends. Every day for the next three months we would leave class together and talk nonstop. I was always mesmerized listening to this ‘time machine’ as she shared her wisdom and experience with me..

Over the course of the year, Rose became a campus icon and she easily made friends wherever she went. She loved to dress up and she reveled in the attention bestowed upon her from the other students. She was living it up.

At the end of the semester we invited Rose to speak at our football banquet. I’ll never forget what she taught us. She was introduced and stepped up to the podium. As she began to deliver her prepared speech, she dropped her three by five cards on the floor.
Frustrated and a little embarrassed she leaned into the microphone and simply said, ‘I’m sorry I’m so jittery. I gave up beer for Lent and this whiskey is killing me! I’ll never get my speech back in order so let me just tell you what I know.’

As we laughed she cleared her throat and began, ‘ We do not stop playing because we are old; we grow old because we stop playing.

There are only four secrets to staying young, being happy, and achieving success. You have to laugh and find humor every day. You’ve got to have a dream. When you lose your dreams, you die.

We have so many people walking around who are dead and don’t even know it!

There is a huge difference between growing older and growing up.

If you are nineteen years old and lie in bed for one full year and don’t do one productive thing, you will turn twenty years old.. If I am eighty-seven years old and stay in bed for a year and never do anything I will turn eighty-eight.

Anybody! Can grow older. That doesn’t take any talent or ability. The idea is to grow up by always finding opportunity in change. Have no regrets.

The elderly usually don’t have regrets for what we did, but rather for things we did not do. The only people who fear death are those with regrets..’

She concluded her speech by courageously singing ‘The Rose.’

She challenged each of us to study the lyrics and live them out in our daily lives. At the year’s end Rose finished the college degree she had begun all those years ago.

One week after graduation Rose died peacefully in her sleep.

Over two thousand college students attended her funeral in tribute to the wonderful woman who taught by example that it’s never too late to be all you can possibly be.

When you finish reading this, please send this peaceful word of advice to your friends and family, they’ll really enjoy it!

These words have been passed along in loving memory of ROSE.

REMEMBER, GROWING OLDER IS MANDATORY. GROWING UP IS OPTIONAL. We make a Living by what we get. We make a Life by what we give.

Firefox Add-On: DOM Inspector

Well, it is no state secret that I am a Firefox fan. It is definite the best Web Browser.

This is such a handy tool for Web developers, especially if you make Themes and Extensions (Add-Ons). You can install it from here. One can inspect in a two page pane the DOM (Document Object Model) of any Web page or XUL (XML User Interface Language – pronounced zool ) window. In other words the structure and the properties of a page or window.

Screenshot:

DOM – Document Object Model
This is the API (application programming interface) for valid HTML and well-formed XML documents. In other all the tags,e.g.<html> ,<div>, <script>, etc. So the DOM specifies how XML and HTML documents are represented as objects, which enables us to use them in object oriented programs.

XUL – XML User Interface Language
This is is Mozilla’s XML-based language that lets you build applications. Although it is not a an official standard, it relies on the existing Web Standards like CSS, Javascript and DOM. This make it quite easy for Web developers to learn. Read more here at Mozilla’s XUL site.

Another Add-On you can also consider is XPather.

Happy Coding

From my pen today.

The rain outside is just lovely. Just what our fruit trees need. It will be the first year that the Apricot tree will bare fruit. The peaches are coming on lovely and the Lemon Tree got a new breeze of life in it. It’s definite going to survive.

I got my new blog, using WordPress of course, installed. Still thinking about a Title. Any ideas?
And the choice of widgets. Oh, there are so many widgets, and made a shortlist which ones to install.
Busy designing a theme for it. Making a theme is not really so difficult as I thought. It’s really going to look cool, but Internet Explorer is driving me crazy. Have to make a special style sheet for it.

Also the issue of CMS. Which one to use? Drupal, Geeklog, Joomla 1.5, Joomla, Mambo, PHP-Nuke, phpWCMS, phpWebSite, Siteframe, TYPO3, Xoops, Zikula? All these choices. Aye.

Well in between there is a post on robots coming up soon. Decided to test out Windows Live Writer. This post was published it. Quite a cool program for bloggers.

Have fun!

Technorati Tags: CMS,Fruit trees,Robots,Windows Live Writer,WordPress Themes

Install XAMPP Lite on Windows

XAMPP lite is a server software and is perfect for testing a website offline, on your computer.
Download XAmpp Lite.
Note: XAMPP is for developers in a development environment.

• Open the self-extracting archive.
  
• Click Install
  
• Wait, until the command window opens.
  Desktop Shortcut: Type y and Press Enter.
  
• Proceed: Type y and Press Enter.
  
• Use Drive Letters: Since this is on a local computer, type y and Press Enter.
  
• Wait… Press Enter.
  
• Type the correct time zone and Press Enter. You can view PHP’s List of supported time zones here.
  
• Launch the Control Panel and select Start next to Apache and MySQL.
  
• You should see that the services is running. You can select Admin to launch it in your web browser or enter the URL http://localhost/ in your browser
  
• Once launched in your browser, select your preferred language.
  
• XAMPP is successfully installed on your computer.
  

There are additional settings you can tweak.
The document directory is "C:\xampplite\htdocs"
Read more about XAMPP for Windows here.
Have Fun!

How To Hide Ads On Go-Daddy Free Site

If you have a Go-Daddy free-site this is how you can hide the advertisements to display in the browsers.
It is this simple!

Update your existing CSS files by adding the following code:
/*Hide Go-Daddy Ads on free-site*/
#conash3D0
{
display:none;
}

If you do not have an existing CSS file, it is quite easy to create.
Just follow these steps:

1. How to create a CSS (Cascading Style Sheet) File:

• Open your favourite text editor.
• Copy a paste the above code in it.
• Save the file as, e.g. style.css
• Upload the file to the root directory or a directory(folder) of your choice to your Web host.

2. How to link a CSS file in your existing Web Pages
Add the following code before the </head> tag in your existing pages.

• If the CSS file is in the root directory:
  <link rel="stylesheet" href="/style.css" type="text/css" / >
• If the CSS file is in the same directory as your Web Page:
  <link rel="stylesheet" href="style.css" type="text/css" / >
• If the CSS file is in another directory, replace the words directoryName with the actual directory name:
  <link rel="stylesheet" href="/directoryName/style.css" type="text/css" / >

Remember to save or upload the files to your Web Host.

Have a professional looking parked domain site

Intro

Well I was actually looking around for WordPress Themes for my new website, then Ian Stewart' Launchpad gave me a great idea for making a html template for websites that is not quite ready yet.

When you’re getting ready to launch your website, most of the hosting companies and/or registars offers a parked domain, that can turn your site into an advertising campaign for them. I am also not a fan of the "Under Construction" pages.

Now you can park your domain in style!

You can download this template here. The MD5 Sum for this file is e859db95e43656b1ffc8bd909d87dec8

The fileparkeddomain.zip contains:

• A Style Sheet: style.css
• Index Page: index.html
• RSS Feed File: mysite.rss
• Readme File: readme.txt
• Images (Sub-Folder)
  • RSS graphic: rss-icon.png
  • Footer graphic: bar.gif
• Screenshot: parkeddomain.jpg

I’ve implemented it on my new Web site. Preview it here.

DomainParking Features

• Professional design and typesetting.
• Integration with Feedburner
• GPL Licensed – you can do with it what you want to.

DomainParking Editing Instructions

RSS Feed File
For more info, read the RSS 2.0 Specifications.
You can update the file with your information in any text editor. Remember to save.
Upload the file to your host.
Validate it at http://validator.w3.org/feed/.
Enter the full URL of your RSS file into the form. Click the button. If there are any errors, the validator will tell you about them. Otherwise, it will validate your file.
Subscribe at http://feedburner/google.com, follow the instruction to get your feedburner links.

The Index File
You can update the file in your favourite html editor or text editor.
Update the page's title with your own.
If you want to use a favicon, make sure to upload your icon to the host and remove <-- Favicon disabled, as well as -->.
If you want to use a logo, make sure to upload your logo imagefile to the host and remove <-- Logo disabled, as well as -->.
In <h1><span>You’ve found </span> zellism.</h1> replace zellism with your name.
In the subscribe-options division, replace the feedburner URI with yours.
Save your file and upload your file to your host.
Head over to http://validator.w3.org/ to validate your file.
Enter the URL of your HTML file into the form. Click the button. If there are any errors, the validator will tell you about them. Otherwise, it will validate your file.

Conclusion

Now you can have a professional looking parked domain theme.
If you need any more help, feel free to leave a comment on this post or contact me by e-mail.
Hope this will help you to have a professional looking site while it is "UNDER CONSTRUCTION".
Best of luck with your website!

DIY – How to Register your Domain name yourself.

Intro

After a friend call me today and ask me to assist them to register their domain name, I’ve decided to write a post and tell you exactly how you can register your own domain name in a few minutes.
Well in South Africa you can easily pay between R150 to R200 to register your .co.za name, by using other companies. If you do it yourself, you pay only R50.00.

Take Note: If you are an eligble Non-Profit organisation in South Africa, you do not need to pay for domain name. It’s for FREE!

Step 1- Check if your domain name is available.

For Non-Profit Organisations – .za.net or .za.org

• Check if your domain name is available here.
• Type YourDomainName.za.net or YourDomainName.za.org in the provided text box.
• Press Enter.

For .co.za domain names.

• Check if your domain name is available here.
• Type in the domain name in the text box.
• Click Enter.

Step 2 – Obtain DNS Nameserver’s name and IP Addresses

You need a minimum of two independant DNS Nameserver names and IP Addresses.

Use your hosting company

• Obtain information from your Web hosting company.
• Note: You need to add your domain name at your host company as a domain or add-on domain before you can proceed.

Use EveryDNS

• Login or Signup at EveryDNS.
• For New users: Just choose an username, enter your e-mail address, and passwords for EveryDNS, and click Signup.
• Enter domain name under Add new domian:(basic) e.g. yourdomainname.co.za or yourdomainname.za.org
• Click the >>(basic) button.
• Your domainname.co.za should appear now under the Primary Domains
• EveryDNS Domain Name Server Details:
  Domain Name Server 1: ns1.everydns.net IP Address: 208.76.62.100
  Domain Name Server 2: ns2.everydns.net IP Address: 78.129.207.168
  Domain Name Server 3: ns3.everydns.net IP Address: 208.76.63.100
  Domain Name Server 4: ns4.everydns.net IP Address: 208.76.60.100
• See note how to update EveryDNS with your Web Hosting details.

Step 3 – Register your domain name

For Non-Profit organisations – .za.net or .za.org

• Read the ZA Nic Policy for Domain Registrations carefully.
• If you do agree with the terms, follow these steps:
  Fill in the Registration Form.
  It is pretty straight forward.

  At the Technical Information section you will fill in your Domain Name Servers
  You must fill in at least 2 names.
  Name1: Domain Name Server 1
  IP1: IP Address for Domain Name Server 1
  Name2: Domain Name Server 2
  IP2: IP Address for Domain Name Server 2

• Double check that the information is filled in accurately.
• If you do agree with ZA Nic Policy for Domain Registrations click Submit.
• If there are any errors resulting from the submission of your request, ZA Nic will reply to you by e-mail. Follow the steps again until your request is accepted.
• It can take a few days before you will get a reply from ZA Nic. You can always check on the Status Page for where your request lies in the queue.

For .co.za Registrations

• Read the terms and conditions carefully. If you agree continue to the next step.

• Download the application form here.
• Open this document in your favourite text editor and complete the details.
• Replace YourDomainName.co.za with your actual domain name
• Only fill in, do not delete any text!!!!
  Section 1:
  1a. Complete domain name: YourDomainName.co.za
  1b. Action – [N]ew, [U]pdate, or [D]elete : N

  Section 2:
  Enter your details accurately.
  2a. Domain Registrant: Your Name and Surname or Organisation name
  2b. Registrant Postal address: P.O. Box 111, Town, Postal Code
  2c. Registrant Street address: Street Address, Town, Postal Code

  2e. [I]nvoice or [A]ccount (Default I):I
  2f. Organisation to be billed or account no. :Your Name and Surname or Organisation name
  2g. Email to send invoices to: Your e-mail address

  Item 2i must contain the Postal Address for Tax Invoicing purposes.

  2i. Invoice Addr: Postal Address, Town, Postal Code

  2j. Registrant Phone No.: Telephone No.
  2k. Registrant Fax No. : Fax No. – can leave blank if you don’t have one.
  2l. Registrant e-mail : Your e-mail address

  2n. VAT No. for 2f : Fill in VAT No. or leave blank

  Section 3: You can leave as is.

  Section 4: Fill in the details of the Administrative person. If there is not a person appointed, fill in your details.
  Section 5: Fill in the details of teh Technical person. If there is not a person appointed, fill in your details.

  Section 6: You need to fill in a minimum of 2 names. e.g.:
  Section 6a: ns1.everydns.net
  Section 6b: ns2.everydns.net
  Replace the names with your Domain Name Server names

  Section 9 – Here you can fill in some descriptions in about what the website will be about
  E.g.
  9a. : Commercial

  Section 10 – Fill in the address as stated in Section 2c.

• Double check that the details are filled in correctly and Save.
• OK, now you are ready to submit your request.
• Select all the text in this document, and copy it.
• Fire up your e-mail program.
• Compose a new message. Make sure it is in Plain Text Format.
• To : coza-admin@co.za
• Subject: New domain YourDomainName.co.za
• In the body of message delete everything if necessary.
• Paste
• Click Send.
• It will take approxiametly 5 minutes before you will get a reply from Uniform SA.
• If all is fine you will receive a message with the subject COZA: OKAY: New domain "YourDomainName.co.za" and an invoice in another message.
• If there were any errors you will receive an e-mail message stating the error. Correct the error and follow the steps until you succeeded.
• So all what is left for you to do, is to pay your R50.00 to UNIFORM SA.
• Their bank details is on their site as well as in the Invoice (sent by e-mail to you).

Note for EveryDNS users:

• Once you setup a hosting plan with a provider (Host4Africa, Hetzner, Gridhost, etc – there are plenty to choose from), they will provide you with a Shared IP Address.
• From EveryDNS, login and select your website.
• Update Your Records: Replace YourDomainName.co.za with your actual Domain Name.
1. Fully Qualified Domain Name: YourDomainName.co.za
2. Record Type: A
3. Record Value: Shared I.P. Address provided by your Web Host company.
4. Click Add.
5. Fully Qualified Domain Name: www
6. Record Type: CName
7. Record Value: YourDomainName.co.za
8. Click Add.

Conclusion

Well, in a few easy steps you can register your domain name yourself.
Since I prefer to have total control over the DNS Settings, I prefer to use EveryDNS for local (South African) domains.
For .com domains I prefer to use GoDaddy. It costs $10.69USD or lower.

Backup Entire Registry in Windows XP

Note: You can create a System Restore Point instead.

Backup registry

From the Start menu

1. Select run
2. Type: regedit
3. Click OK or press Enter
4. From the Registry Editor select My Computer
5. From the Registry Editor’s menu select File | Export
6. In the Export Registry File window, choose a location to save the file in.
   Note: I recommend that you save the file to your Desktop or C:\ drive, so that it easy accesible if you do run into problems later.
7. Enter a name for the backup file in the File name textbox.
   Note: This file name is for you to remember what the exported registry file is for. Since you are backup up the entire Registry I recommend that you use the following filename: Full Registry Backup.
8. Click the Save button.

A Registration File with a REG file extension will be created in the location you chose in Step 6 and with the file name you chose in Step 7.

Watch the video clip here.