This may be useful but if you need a little more control, here’s how to dynamically add pages to your WordPress sitemap:

1. Edit /wp-content/plugins/google-sitemap-generator/sitemap.php

2. Find the line that reads:

$this->AddElement(new GoogleSitemapGeneratorXmlEntry(“</urlset>”));

3. Just before this line, insert:

include(“mysitemap.php”);

4. Create a new empty file into /wp-content/plugins/google-sitemap-generator/ and save it as mysitemap.php.

5. Insert the following content into your newly created file:

<?php

$this->AddElement(new GoogleSitemapGeneratorXmlEntry(” <url>”));

$this->AddElement(new GoogleSitemapGeneratorXmlEntry(“\n <loc>http://www.yourdomain.com/document.html</loc>”));

$this->AddElement(new GoogleSitemapGeneratorXmlEntry(“\n <changefreq>weekly</changefreq>”));

$this->AddElement(new GoogleSitemapGeneratorXmlEntry(“\n <priority>0.5</priority>”));

$this->AddElement(new GoogleSitemapGeneratorXmlEntry(“\n </url>\n”));

?>

6. Once your done, upload sitemap.php and mysitemap.php to your web server.

Of course the iteration process has to occur between the <url> and </url> entries (the part identified in red). You can dynamically add pages using an SQL query for example.

The sitemap.xml and sitemap.xml.gz files will automatically be updated with your dynamic entries next time your rebuild your sitemap.

One way to speed up thing is by finding out the keywords that lead visitors to your competitors’ website (assuming you know your competition of course). To find out what keywords is sending them traffic, I use SeoDigger.com. It’s a free online tool that detects which keywords leads to a particular URL.

SeoDigger.com is pretty easy to use: simply enter the URL of the website you want to do keyword research on.

Once you’ve got the results, you can find out which keywords are searched for often. Just sort the results by WT (Wordtracker) or OT (Overture Tool). Keep in mind that Wordtracker’s keyword popularity estimation is based on a daily basis whereas Overture base its results on a monthly basis.

Looking at the screenshot above, you can see that GoogleLady.com gets a lot of traffic for the search term “adwords tip”. According to Wordtracker, “adwords tip” is searched 248 times a day and GoogleLady.com is in 8th position for this keyword.

Registering on SeoDigger.com gives you the possibility to perform a full search so if you aren’t getting any results for a particular, try logging in and check the “Full search for domain name” checkbox before attempting to do another search.

What other keyword tool do you like?

One important thing to consider when it comes to SEO is your webpage’s first sentences. Your most important keywords should be at the beginning of your webpage. Unfortunately for most websites, the first thing that’s appearing on their website is… the header and the top menu! And usually this menu consists of keywords like: home, contact, about us, privacy, etc.

How Search Engines Are Seeing Your Website

Even though you’ve got this great ergonomic layout that makes your site intuitive, keep in mind that your website isn’t seen the same way by search engines. To get an idea of how my website is seen by search engine, I use this tool from dlperry.com.

This shows your website content as it’s seen by search engines.

Transforming Unimportant Keywords Into Images

Suppose you’ve got your website name written in plain text at the top of your website and that it doesn’t contain important keywords, you could simply replace it with an image file.

This trick has to be used with precaution. Don’t abuse of this technique. I use this to remove unimportant keywords from the top of my webpages, not everywhere, just at the top. Using too many images could slow down your website.

Using CSS To Move Unimportant Keywords At The End Of The HTML Code

Now this is where it gets interesting. For example, we’ll use the following webpage layout (click to enlarge):

If I had used a table to build this webpage, the source code would look something like this (I’ve shorten the “lorem ipsum” content):

<table border=”0″ width=”760″>
<tr><td><img src=”images/logo.jpg”></td></tr>
<tr><td>home | about | contact | privacy</td></tr>
<tr><td><h1>My Document Title</h1>
<p>Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Sed in dolor ut sapien suscipit faucibus. Maecenas a sem. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos hymenaeos. Integer auctor interdum lorem…</p>
</td></tr>
<tr><td>Copyright MyWebsite</td></tr>
</table>

And this is how this webpage would be seen by search engines (again, the lorem ipsum part have been shorten):

home | about | contact | privacy My Document Title Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Sed in dolor ut sapien suscipit faucibus. Maecenas a sem. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos hymenaeos. Integer auctor interdum lorem. Copyright MyWebsite

Now what you want to do is to move the top menu’s content at the end of the HTML code. Visually, you still want it to be at the top of the webpage though. We’re going to use layers (DIV) and CSS to do that.

The first step is to transform the table into layers. We’ll be able to position layers wherever we want them later using CSS. This is the source code used to generate the same layout but with layers instead of a table:

<div id=”divHeader”>
<img src=”images/logo.jpg” width=”400″ height=”90″ />
</div>
<div id=”divBody”>
<div id=”divContent”>
<h1>My Document Title </h1>
<p>Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Sed in dolor ut sapien suscipit faucibus. Maecenas a sem. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos hymenaeos.</p>
</div>
<div id=”divFooter”>Copyright MyWebsite</div>
</div>
<div id=”divTopMenu”>home | about | contact | privacy</div>

And this is the CSS styles applied to the webpage:

#divHeader { position: absolute; width: 100%; height: 90px; width: 760px;}

#divBody { position: absolute; margin-top: 112px; width: 760px; }

#divContent { position: relative; padding: 15px; }

#divTopMenu { position: absolute; margin-top: 90px; padding-left: 15px; width: 745px; }

#divFooter { position: relative; text-align: center; }

So now here’s how my website is seen by search engines:

My Document Title Lorem ipsum dolor sit amet, consectetuer adipiscing elit. Sed in dolor ut sapien suscipit faucibus. Maecenas a sem. Class aptent taciti sociosqu ad litora torquent per conubia nostra, per inceptos hymenaeos. Integer auctor interdum lorem. home | about | contact | privacy Copyright MyWebsite

Now all that’s left to do is to optimize the first keywords found at the beginning of your webpages!

If you’d like to see the full source-code for this tutorial, you can download it here: css_and_images_seo.zip

Let me know what you think.

While there are plenty of tools available, I’ll focus mainly on AwStats today. AwStats is a free and powerful tool that generates advanced statistics for many protocols including web, streaming, ftp and mail transfer. This log analyzer works as a CGI or from command line and generates statistics in HTML format.

Installing AwStats On Linux

1. Get your copy of AwStats from SourceForge.net at
   http://sourceforge.net/project/showfiles.php?group_id=13764
2. Extract all files from the archive and upload it to a temporary folder on your web server. This folder needs not to be accessible through HTTP.# tar xvzf awstats-[version].tar.gz
3. Move all the content from the archive to /usr/local/awstats. When you’re done, this directory should contain:
   • ./docs
   • ./README.TXT
   • ./tools
   • ./wwwroot
4. Change directory to /usr/local/awstats/tools and execute ./awstats_configure.pl# cd /usr/local/awstats/tools
   # ./awstats_configure.pl
   
5. Go through the configuration script.
6. Add a cron task to update statistics on a daily basis. To do this, simply create a new file named awstats_updateall.sh into /etc/cron.daily. This is the script’s content:#!/bin/sh
   /usr/local/awstats/tools/awstats_updateall.pl now
   
7. Add execute permission to awstats_updateall.sh
   
   # chmod 0755 /etc/cron.daily/awstats_updateall.sh
   
8. Create a directory named “awstats” into /var/lib.
9. Verify that the following directives have been added to your httpd.conf file. If not, you will have to add them manually:#
   Alias /awstatsclasses “/usr/local/awstats/wwwroot/classes/”
   Alias /awstatscss “/usr/local/awstats/wwwroot/css/”
   Alias /awstatsicons “/usr/local/awstats/wwwroot/icon/”
   ScriptAlias /awstats/ “/usr/local/awstats/wwwroot/cgi-bin/”
   #
   # This is to permit URL access to scripts/files in AWStats directory.
   #
   <Directory “/usr/local/awstats/wwwroot”>
   Options None
   AllowOverride None
   Order allow,deny
   Allow from all
   </Directory>
   
10. Edit your AwStats configuration to make sure all settings are correct. This file is located in /etc/awstats/awstats.[your domain name].conf
11. Make sure your httpd access log file format is “combined”. To verify this, open /etc/httpd/conf/httpd.conf and look for:
    
    CustomLog [your log path]/access_log commonandchange it to:
    
    CustomLog [your log path]/access_log combined
    
12. Restart the httpd service:
    
    # service httpd restart
    
13. Generate some statistics:
    
    # /etc/cron.daily/awstats_updateall.sh
    
14. Try accessing AwStats through your web browser at http://[your domain name]/awstats/awstats.pl

That’s pretty much it!

You can read the AwStats documentation here : http://awstats.sourceforge.net/docs/index.html

http://www.mydomain.com/
articles.php?category=39&article=27

This is a problem with large websites as they are database-driven and parameters have to be sent to a script for processing. Search engines are much better at indexing static pages. Even though Google doesn’t seem to be afraid of dynamic pages, other search engines such as Yahoo! and Live will probably index just a few pages as they don’t want to be caught in a loop.

Fortunately, there are at least three solutions to this: using PATH INFO, ForceType and mod_rewrite.

There’s another method which involves 404 pages but I wouldn’t recommend it since you won’t be able to submit your sitemap to Google. When you add your website to Google’s Webmaster Tools, 404 pages MUST return a 404 error code but when you use that method to write friendly URLs, you change that 404 code to 200 (which means the page exists even if it doesn’t).

Friendly URLs Using PATH_INFO

This is probably the easiest method although it’s not my favorite (I’ll explain why later). Suppose you have this URL :

http://www.mydomain.com/articles.php?category=39&article=27

Using PATH INFO will give you a URL similar to this:

http://www.mydomain.com/articles.php/39/27

When Apache receives a request for this URL, it will first look for a file or folder named “27″. As it doesn’t exist, it will look for a file or folder named “39″. As it doesn’t exist neither, it will look for a file named “article.php”, which exists, and will call that script. Upon execution of that script, Apache will store the path articles.php/39/27 into a global variable called $PATH_INFO.

At this point, what you want to do is split that variable into an array like this :

<? $myvars = explode(‘/’, $PATH_INFO); ?>

This way, you will be able to access each variables like this :

<?

echo myvars[0]; // returns “articles.php”
echo myvars[1]; // returns 39
echo myvars[2]; // returns 27

?>

Now simply rename your variables to what your PHP script expects to process and you’re done.

If the $PATH_INFO variable is empty, this may be because your need to turn on this feature in your .htaccess file:

AcceptPathInfo On

To do this, you must also ensure that your web hosting provider allows you to override Apache’s main configuration.

The reason why this is not my favorite method is because the URL still needs to call the script and it doesn’t look as nice as a static page URL:

http://www.mydomain.com/articles.php/39/27

Friendly URLs using ForceType

This method allows you to hide your script’s extension (.php). To make a long story short, Apache is configured to parse only .php files as PHP (and maybe .php3, .php4 also). Basically, what you’re going to do is tell Apache to parse files with no extensions as PHP through your .htaccess file :

<Files articles>
ForceType application/x-httpd-php
</Files>

This way you’re going to be able to write URLs like this:

http://www.mydomain.com/articles/39/27

All that’s left to do is to get the rest of the path through the $PATH_INFO like you’ve seen in the previous method.

Friendly URLs with mod_rewrite

This is my favorite method although it’s probably the most complex one. Mod_rewrite is an Apache module which lets you rewrite URLs through regular expressions in an .htaccess file. In my opinion, this is the most powerful and flexible way to use friendly URLs but novices might not be comfortable with this one.

In order to use mod_rewrite, you must make sure that your web hosting provider has enabled this feature in Apache’s configuration file.

So let’s pretend we want to transform this dynamic URL into a friendly URL:

http://www.mydomain.com/articles.php?category=39&article=27

The first thing to do would be to add a field to both our ”article” and “category” table in our database. That new field will contain a “friendly” identifier for each record and must be unique. So instead of identifying our article by ID #27 and the category by ID #39, we would identify them by more friendly terms so that our URL could look something like:

http://www.mydomain.com/articles/tutorials/dreamweaver.html

So here our category is identified by “tutorials” and “dreamweaver” would be the article identifier. Got it? Great.

So now we need to rewrite the above URL so that we can pass our identifiers to our PHP script.

Here’s what our .htaccess file may look like:

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteRule ^articles/(.*)/(.*).html$   articles.php?categ=$1&article=$2 [L]
</IfModule>

Let’s break down these instructions:

• IfModule mod_rewrite.c : this makes sure that the mod_rewrite module is enabled.
• RewriteEngine On : turns on the rewrite engine (duh!).
• RewriteBase : explicitly sets the base URL for per-directory rewrites.

The RewriteRule line is the most crucial one so here’s a detailed break down:

• ^ : this indicates the start of line.
• articles : means that the URL we want to rewrite MUST START by “articles/” (ie.: http://www.mydomain.com/articles/tutorials/dreamweaver.html)
• (.*)/ : this our first variable. Everything that comes before the first slash is put into a variable named $1. That means our category identifier will be stored into $1.
• (.*).html : this is our second variable. Everything until “.html” will be stored into another variable named $2. Our article identifier will be stored into $2.
• $ : this means that nothing must come after “.html” or else the URL will not be processed.
• articles.php?categ=$1&article=$2 : this is where we call our PHP script. Both $1 and $2 variable are going to be replaced by their respective values (see above).
• [L] : the [L] instruction makes sure that the rewrite engine will stop processing the URL after this statement. [L] means “Last”.

Now all that’s left to do is to query your database according to values you passed to it through URL rewriting.

There is a lot more to mod_rewrite and regular expressions so have a look at the following links:

• Apache’s mod_rewrite documentation
• Regular expressions (regex) tutorials, examples and reference

And of course, you can also ask me your questions.

Introducing Google Sitemaps

In June 2005, Google introduced Sitemaps 0.84 so web developers could publish lists of links from across their sites. About a year and a half later, Google, MSN and Yahoo announced joint support for the Sitemaps Protocol and the schema was changed to 0.90. In April 2007, Ask.com and IBM also announced support for Sitemaps and Google, Yahoo and MSN announced auto-discovery for sitemaps through robots.txt.

The Sitemaps Protocol allows a webmaster to inform search engines about URLs on a website that are available for crawling. Basically, a Sitemap is an XML file that lists the URLs for a site. In addition to listing URLs for a site, the webmaster can include additional information about each URL such as: when it was last updated, how often it changes, and how important it is in relation to other URLs in the site.

Here’s a sample Sitemap file :

<?xml version=”1.0″ encoding=”UTF-8″?>
<urlset xmlns=”http://www.google.com/schemas/sitemap/0.84“>
<url>
<loc>http://www.thewebmasterscafe.net/</loc>
<lastmod>2007-09-03T13:15:53+00:00</lastmod>
<changefreq>daily</changefreq>
<priority>1</priority>
</url>
<url>
<loc>http://www.thewebmasterscafe.net/news/submit-link.html</loc>
<lastmod>2007-09-03T09:15:53+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.1</priority>
</url>
<url>
<loc>http://www.thewebmasterscafe.net/publicize
/google-adwords-for-dummies-pt1.html</loc>
<lastmod>2007-09-02T21:46:05+00:00</lastmod>
<changefreq>weekly</changefreq>
<priority>0.1</priority>
</url>
</urlset>

This file must be saved at the root of your website in .xml format or it can be compressed in a gzip archive. Your Sitemap should then be accessible at http://www.yourdomain.com/sitemap.xml or http://www.yourdomain.com/sitemap.xml.gz .

Generating Your Sitemap

Even though some web developers will make their own Sitemap generator, there are some utilities that are available for this task. You can get Google’s Sitemap Generator from the Google Webmaster Tools. The script is written in Pythonso you should be able to run it on most plateforms. I liked using this script because it allows you to submit the Sitemap automatically upon generation. Be careful when using this feature because submitting your Sitemap too often can be considered as abuse. The “-testing” switch prevents the script from submitting the Sitemap automatically.

If you are generating content from a WordPress blog, you can use Google Sitemap Generator for WordPress (it is not a Google product). Just make sure you read the readme.txt file carefully. I’ve installed version 3.0b9 on a WordPress 2.2 blog and it worked fine.

There are plenty of other tools available, just make a search on Google.

Submitting Your Sitemap

Before you go ahead and submit your sitemap, you might want to validate it’s content. Many validation tools such as Free Google Sitemap Validator and Submitter. Once you validated your Sitemap, you can submit it directly to search engines :

• Google Webmaster Tools
• Yahoo! Site Explorer
• Ask.com : Webmasters

Alternatively, you can add a “Sitemap” directive to your robots.txt file :

Sitemap : /sitemap.xml

Where can I find more information on the Sitemaps Protocol?

More technical information can be found over at http://www.sitemaps.org/

Lately I stumbled onto a useful tool that can predict your website’s future PR. As this website (TheWebmastersCafe.net) is pretty new, it’s currently stuck into Google’s “sandbox effect”. This is the probation period where your website still hasn’t got a PR from Google so I wanted to know what my blog’s PR would be in the future. Unfortunately, I haven’t got enough backlinks yet so the prediction isn’t good. On the other hand, my other website, HomeMusician.net, has over 500 backlinks but it’s PR probably won’t move but still, it has a PR 4 which isn’t bad.

Here is the prediction tool as well as some other useful PR-related links :

PageRank Prediction
http://www.iwebtool.com/pagerank_prediction

Live PageRank
http://livepr.raketforskning.com/

Google PageRank Checker
http://www.prchecker.info/check_page_rank.php

iWebTool Page Rank Checker
http://www.iwebtool.com/pagerank_checker