Did Google Scrape My Website? You Be The Judge. Rules Threaten Realtors & IDX Providers. (2024)

[Update] “NAR has supported this ruling! That is the scary thing. My local board threw it on NAR for intrepretation and they indeed said Google is a scraper site and no we must “protect the data” from scaper sites. I think the fear is if it starts with our board, it could easily be implemented as a nationwide policy like Benn said.” See Mike Taylors comment below.

Warning – Long, but worth it if you want to have or keep a dynamic IDX solution for your real estate website or blog.

The Back Story

Recently, my website was reported for allowing the MLS data on my website to be indexed,which led the Indianapolis Metropolitan Board of REALTORS®, (MIBOR), to issue a cease and desist letter to my broker for both our sites, or they would cut our feed.

Since April 10th,Googlecan notindex the detail pages of the IDX on our websites. Our Title tags can not contain any details about the property (Example #1). The decision affected about 15 other brokers and agents in Indianapolis. While I firmly believe I have not violated any MLS rules, we complied! It’s hard to fight an 800-pound gorilla

On April 30th, another demand! The description can not contain ANY MLS data, no address, the MLS number, neighborhood, subdivision, price, bedrooms, baths, property type or public remarks. (Example listing 2)

But Realtor.com can:

MIBORclassifies Googleas a scraper site and therefore, I am allowing Google to scrape or reproduce the MLS data from my site.This interpretation is supported by the National Association of REALTORS®, based on the MIBOR’s listing service rules and regulations.

Section 15.2.2 – participants must protect IDX information from misappropriation by employing reasonable efforts to monitor and prevent “scraping” or other unauthorized accessing, reproduction, or use of the BLC database

The entangled web of rules weabide by to be a participating member of our local board and MLS are subject to the interpretation of the local authorities and agents who police the web for offenders. They say scraping and indexing is the exact same thing.

I guess then we would have to define scraping vs. indexing. While there are detractors who say Google or any search engine are indeed scraper sites, thegenerally accepted charachteristicsof a scraper site should be explored:

• Scraper sites copy and blend information to make it look less like plagiarism
• Scraper sites directly copy another’s site without attribution or copy for the purpose of generating revenue through paid advertising.
• Scraper sites use automated means to harvest data.
• Scraper sitessteal content without adding utility.
• The biggest difference is people go to Google looking for information – and invite Google to visit their site.

You can’t have it both ways; call Google a scraper, then invite them to crawl your site.

Start your newspaper ad with your clientshome featured in black and white and gather up your open house signs, we’re heading backwards. Let’s hide the information the client wants, put our MLS listings in a book and wait a week for them to be published.Let’s force the hand of the client to only have access to the agent who listed the home.

Why stop there – we should just go back to the days of sub-agency, when everyone represented the seller.

Talk about going backwards:

Look what happened to the information for 2 separate listings before and after April 10th. This is the basic search for the MLS number and city MIBOR used to issue their last demand.

Home listed before April 10th and still indexed

Home Listed after April 10th

I’ll bet the second home owners are really happy with their online marketing!

That one little listing with a Diverse Solutions RSS feed will soon be gone too, if the interpretation of the rules is enforced. Until then, at least this home has some marketing exposure.

I have been told this will be a subject discussed at the mid-year NAR meeting next week. Can’t confirm it, though!

It doesn’t bother meif my clients listing ends up on other agents websites or indexed by another REALTOR® and I’m willing to bet my clients don’t care either.I’m thrilled when there aremultiple returns for my clients address. I know many of the entries will be by my efforts and more will be from the collaborative effort of REALTORS® who understand the dynamics of the Internet Data Exchange and those who have harnessed the power of online marketing.

So far, the rules have notimpactedmy website. I’ve actuallly moved from #13 to #6 since April 10th for Indianapolis Real Estate and traffic to my site has been unaffected. This is bigger than my one site; it affects REALTORS®, our CLIENTS, IDX providers, and the future of our ability to market for our clients and our client’s access to data.

My predictions: Control of indexed IDX will be held by our local boards through their public facing site, along with third party aggregators who will gladly take all the listings we give them, and charge us to submit them or have a premium account.They will continue to charge us for leads generated by our listings.

Oh and let’s not forget REALTOR.com, who can use the data because they have a special deal with our board. We know what they charge to have an enhanced listing package. [Insert Russell’s Graphic]

I’m certain there are many unsuspecting REALTORS® and IDXCompanies who have no idea their marketing and websites are about to be axed. No longer will we be able to have RSS feeds, dynamicIDX listingsor indexed MLS data.

I know othersin the industry are working on new ways to deliver the data through advanced feeds and integrated IDX. Unless REALTORS® are allowed to provide this information to the public – why should someone else be allowed to?

It’s a new world of change – where nothing really changes – two steps forward, three steps back. March on…………..

Note from AG staff: If you would like to voice your opinion regarding this ruling to NAR, please email Cliff Niersbach. You can also email Steve Sullivan, MIBOR CEO or call him directly (317) 956.5000 ext. 237. Please keep your words professional if you choose to take action and be sure to reference this article.