How to audit every business in your town in an afternoon

I pulled every independent business within 2500 metres of the Quad in Tuscaloosa, checked whether each one had a website that answers, and had the numbers in about forty minutes. The result was 121 businesses and 27 working sites. Here is the whole method, so you can run it on your own town.

It costs nothing. No API key, no account, no scraping anybody's search results.

The data source

OpenStreetMap, queried through Overpass. It is volunteer-maintained, openly licensed, permits this use, and it carries exactly the tag that matters: website.

Overpass takes a query language that looks alarming and is mostly mechanical. The shape you need is: for each kind of business, find every node and way within a radius of a point.

Those coordinates are the centre of your circle. Get them by right-clicking any point in OpenStreetMap. Repeat the node and way pair for every category you care about: shop=barber, shop=clothes, shop=books, amenity=cafe, and so on.

POST that to overpass-api.de/api/interpreter with the query in a form field called data. Send a real User-Agent with a contact address in it. It is a free volunteer service and being identifiable is the rent.

Filtering to independents

You do not want chains. Do not try to detect them by name, because you will build a list of brand strings and it will be wrong in both directions.

OpenStreetMap tags them for you. Drop anything with a brand or brand:wikidata tag. That is one line and it is more accurate than anything you would write by hand.

The check that matters

For each business with a website tag, make a real request and see what happens. This is where both of my mistakes live.

Mistake one: HEAD is not enough

I sent HEAD requests, because they are cheap and you only need the status code.

Plenty of live servers do not answer HEAD. Two restaurants whose sites are perfectly fine came back as broken, and that number reached a published page before I caught it. I had to correct the post in public.

Fall back to GET whenever HEAD fails:

Note the 403 and 405 handling too. A server that refuses to be probed is not a server that is down, and treating it as down is a false statement about a real business.

Mistake two: the claim you are entitled to make

The headline I wanted was "93 businesses near campus have no website." I could not have that one, because it is not what the data says.

OpenStreetMap is volunteer-maintained. A missing website tag does not prove no website exists. It proves that a machine looking for one does not find it.

That turns out to be the better claim anyway, because it is the same question a customer with a phone is asking. Nobody types a business into a search bar hoping it exists somewhere. They type it hoping something comes back.

Write down the claim your method actually supports before you write the headline. If the honest version is weaker, you either need a different method or a different headline, and reaching for the strong version anyway is how a piece of research becomes a thing somebody can disprove in one reply.

What to do with the result

Sort by how interesting the row is rather than alphabetically. The most interesting are the businesses that list a website which no longer answers, because they already decided a site was worth paying for and theirs broke. That is a much shorter conversation than convincing somebody they need one.

Then: name, category, street, and whether the site answers. Nothing else. No phone numbers, no email addresses. The moment you collect contact details you have built a marketing list, and if the reason you told people you were doing this was research, you have made yourself a liar in a way that is very easy to notice.

The whole thing

One Overpass query, a brand-tag filter, an HTTP request per site with a GET fallback, and a sort. Under two hundred lines including the comments explaining why the GET fallback is not optional.

I am using mine to write about independent businesses near campus, one at a time. If you run it on your town and find something surprising, I would like to hear it.