isnoop.net blog

If you are an adsense user, you may have seen this email:

Google AdSense Policy Enforcement
Hello,

While reviewing your account, we noticed that you are currently displaying Google ads in a manner that is not compliant with our policies. For instance, we found violations of AdSense policies on pages such as http://isnoop.net/gmail/

Publishers are not permitted to encourage users to click on Google ads or bring excessive attention to ad units. For example, your site cannot contain phrases such as “click the ads,” “support our sponsors,” “visit these recommended links,” or other similar language that could apply to the Google ads on your site. Publishers may not use arrows or other symbols to direct attention to the ads on their sites, and publishers may not label the Google ads with text other than “sponsored links” or “advertisements.”

Please make any necessary changes to your web pages in the next 72 hours. (truncated…)

It’s nice of them not to bring the hammer down on me for having text that said “Please patronize our fine sponsors,” but it’s even more interesting to see where different ad companies draw the line.

The previous text was officially approved for use on my site when I was serving up AdBrite ads. In fact, AdBrite called me on the phone one morning to ask me to change it from the original text which read something like “Please support this service by visiting the sites below.” I assumed that sort of direct phrasing was frowned upon, but I wasn’t sure and ignorance is bliss.

I assume that Google would disapprove of me posting “Whatever you do, don’t click my ads!” above my AdSense, so that’s why I’m not going to do it. Instead I’ll just publish this blurb about making nice for the kind folks who might just pay me a few dollars towards the costs of running this dedicated server.

Whatever you do, don’t click my ads.

I learned some valuable lessons in high traffic geocoding this week. All this because Google doesn’t offer geocoding services for Google Maps, so you must send them latitude/longitude numbers for any point you want to plot.

This begs the question: How do I quickly come up with the lat/lon coordinates for Shanghai, Anchorage, Indianapolis, Portland, and Seattle? Google does provide a handy link to a Google search for “free geocoder” in their maps API documentation, but none I’ve found have a decent API, work for free, or can sustain the amount of traffic I might request. I’d greatly prefer owning a database and performing the lookups under my own processing power.

The answer I came up with for the low-demand Seattle Emergency Events Map was to screen scrape Google’s own mapping service to see what coordinates they come up with for a given location. It wasn’t pretty, and it wasn’t mine, but I was already using Google so what the heck.

That solution worked beautifully until I got 20,000 visitors to my Maps + RSS package tracking page on Monday of this week. Apparently, Google doesn’t appreciate being hit that much. They temporarily shut down access from my server’s IP to the page I was scraping with a message indicating they’d detected excessive automated behavior. They said something about my tools maybe being a virus. They also kicked my mom in the shin.

When I was notified Google-scraping geocoding wasn’t working anymore (never screen scrape without setting a failure mechansim), I pulled the code and provided a nice message for my site’s visitors. Google dropped the block shortly thereafter, and I hear they gave my mom flowers and apologized for that regrettable shin thing.

I checked out various solutions, trying to find a geocoding database that suited my needs. The US Census TIGER database was far too in-depth and only dealt with US locations. I ended up deploying a commercial IP-to-location database that contains the coordinates for any city that has an IP range associated with it.

Google employees, please skip the following paragraph.

My current geocoding solution involves a lookup in the ip2location tables. If I cannot find a position from there, I check a database cache of locations obtained from Google. If that fails, I scrape the location from Google Maps and cache it for future reference. If that fails for any reason, I go back to the ip2location database and make a darned good guess as to where to point. This typically means centering on a state or even entire country, but it’s better than nothing. This method results in very low traffic to Google, but my goal is zero external reliance.

This geocoding method shouldn’t be long-lived. I plan on converting a copy of the TIGER database for US addresses and purchasing a listing of a few million world locations. I’m always in favor of saving money, so if anyone knows of a free world cities geocoding database, or already has the TIGER database converted to a query-able format, please let me know.

Once I’ve got a satisfactory geocoding system built up, I’d like to open the access and make a public API. That’s down the road a little way, but keep your eyes open for that.

I’ve just published an update to my universal package tracking tool that now enables you to view a map of your package’s progress as it travels across the country.

This new mapping addition builds on the original features of being able to track UPS, FedEx, USPS, and Airborne/DHL packages all in one place and having that tracking information published into a personalized RSS feed. The system automatically detects which company your tracking number belongs to and loads the package data for you.

A nice side benefit of this new addition is that I’m developing a pretty robust Google mapping class, helping my other map projects to evolve.

Despite the advances of the Internet, apparently some news still travels slowly. I closed down my Gmail Invite Spooler page months ago, but I’m still getting hundreds of unique visitors to that page along with about a dozen email requests for Gmail each day. Almost all of the emails and traffic are from foreign countries, so I devised a simple javascript that will allow folks to more easily translate the page into their own language.

You can use this very same javascript. It should work on any site just by including the following:

Here’s what it will look like:

After watching a building burn across the water from my office today, I had the idea of applying the Seattle emergency response data to a map.

You can click on each of the datapoints for some interesting info on what’s going on at this very moment in the Emerald City. In later revisions, I’ll add auto-updating AJAX calls and fancy icons for different event types.

If you live in a major city with data like this, let me know the URL. I might be able to whip up something similar for you.