I don’t have an axe to grind. I’m not angry. And I’m not worried about Zillow replacing my job. I am concerned about the public trusting Zillow way too much though, so I wanted to share an eye-opening “Zestimate” example to help create conversation. Please give it a read and let’s talk in the comments.
Here’s the Zestimate history of a single family home in Carmichael, CA. I’ve been following this property for the past two months.
1) OVERPRICED: This home was originally overpriced in MLS at $380,000 and the Zestimate happened to be $380,414 when the home listed for sale.
2) PRICE & ZESTIMATE REDUCTION: The price was reduced to $335,000 in MLS and very soon after the Zestimate was changed to $346,364. Seeing this definitely gave me pause and made me wonder how much weight Zillow’s algorithm gives to the list price. Does it usually match the list price when a property is first listed? How much do price changes affect the Zestimate? These are reasonable questions. By the way, I shared about this house on Twitter and Inman News included it in a story.
3) ZILLOW DIDN’T GIVE MUCH WEIGHT TO THE SALES PRICE: The property sold at $350,000 eventually, but then Zillow’s estimate said the property was worth $327,960 after the property recorded at $350,000. So it looks like Zillow’s algorithm did not automatically give strong weight to the actual sales price.
4) ZESTIMATE DECLINES 8% IN 30 DAYS: Despite selling at $350,000 one month ago, Zillow now says this property is worth $301,972, which is a whopping 14% less than it actually sold for last month. Moreover, in just 30 days Zillow states the value of this home has gone from $327,960 to $301,972, which is an 8% loss in value. Did the market really decline by 8% this past month?
ZILLOW’S TRENDS ARE OUT OF SYNC: I can’t speak for every property listed on Zillow, but the Zestimate history in this case is definitely out of sync with the market in Carmichael. Values are certainly softening for the season like I talked about last week, but a decline of 8% in 30 days is simply not accurate. Look at the trend line in my graph that shows the market balancing out, and then look at the Zestimate connected by dots. That’s a huge difference in direction, right? Obviously Zillow hasn’t been inside this house, so we can give grace to an algorithm not knowing the full picture of value for an individual property, but to say the market has dropped by 8% is just inaccurate (and you don’t need to go inside a house to know that).
QUICK STUFF ON MY MIND:
1) It’s not just a “ballpark” for consumers: Despite Zillow touting itself as a “ballpark” valuation or starting point for consumers, many people treat it like a definitive value. Bottom line.
2) Blind trust: I find many consumers trust Zillow without thinking too much about accuracy. I don’t say this to be insulting, but there is a deep trust with this brand right now without many questions being asked. My advice? Think critically about examples like I showed above before you trust with all your heart.
3) Pricing according to Zillow: Many sellers are growing disconnected from the real market lately and they’re prone to overprice. Having sensational real estate headlines for six years is definitely one of the reasons why this is happening, but I think some of it comes down to consumers having more information than ever about prices and value. The struggle is when sellers feel so strongly that a Zestimate is legitimate that they aren’t willing to price below it or hear pricing suggestions from others. My advice? Listen to the market around you as well as real estate professionals. Don’t get so caught up in a Zestimate that you cannot see anything else.
4) Feeling stressed: I had a friend who used to get stressed out when his Zestimate would dip. He has since passed away, but I can only imagine how he’d feel about an 8% price decline in 30 days if that’s what Zillow said about his home. He’d be freaking out, and the truth is he wouldn’t be alone because there are lots of people who think of Zillow as way more than just a “ballpark”. In short, I recommend being careful about letting a website’s value claim affect your mood or even your real estate decisions. By the way, the New York Time’s wrote a piece last week called Why Zillow Addicts Can’t Look Away.
I hope that was interesting or helpful.
SPEAKING GIGS: By the way, if you’re around, I’ll be speaking at the Appraisal Institute’s 2018 Fall Conference in San Francisco on October 19th and AppraiserFest in San Antonio on Nov 1-3. I’m also doing a real estate blogging class locally on October 11th at SAR. Just a heads-up.
Questions: What do you think of Zillow? Is the public trusting this website and brand too much? Any stories to share?
If you liked this post, subscribe by email (or RSS). Thanks for being here.
Absolute genius with the tracking. When someone indicates the zestimate, I tell them they should go with that….
Thanks Mark.
Zillow, born in 2006, is a pre teen. I have one at home too.
Yep, pre-teens who love Fortnite. That’s what we both likely have.
Jeff, awesome comment. Zillow has NEVER seen a normal market. It’s pretty likely their algorithms are going to need some serious tweaking going forward.
Yeah, the market hasn’t been too normal. That’s for sure. We do see the market in the listings and pendings first before the sales, so we’ll have to keep our eyes open to see changes before we see actual sales.
Hi Ryan, I love Zillow it is my best marketing tool in my appraisal bag. Now I know that sounds weird but take a look at the following which is of course directly off the Zillow site.
As an executive summary of the below data, says that Zillow is accurate within 5% of the sales price about 55% of the time. If I as an appraiser could be that accurate all the time I wouldn’t need E & O insurance.
I am telling the RE offices that I teach to go get this data, print if off, laminate it and take out on their listings. This should help the agents get listings and not fight with the seller and help the appraisers not have to fight Zillow Zestimates.
If you are a nut for numbers read below.
Great job as always Ryan. Hope all is well in Old Sacramento. Regards, Brad
Within 5% of Sale Price:
This is the percentage of transactions in a location for which the Zestimate was within 5% of the transaction price. Nationwide, Zestimates are currently within 5% of the final sale price 55.5% of the time.
Within 10% of Sale Price:
This is the percentage of transactions in a location for which the Zestimate was within 10% of the transaction price. In the U.S. as a whole, Zestimates are currently within 10% of the final sale price 75.7% of the time.
Within 20% of Sale Price:
This is the percentage of transactions in a location for which the Zestimate was within 20% of the transaction price. Nationally, Zestimates are currently within 20% of the final sale price 87.6% of the time.
Median Error:
Half of the Zestimates in an area were closer than the error percentage and half were farther off. The median error rate for the country is currently 4.3%, meaning half of Zestimates nationwide were within 4.3% of the final selling price, and half are off by more than 4.3%.
Thanks Brad. I appreciate you sharing the specifics. This goes to show we have to understand how stats work. Zillow states their median error rate is 4.3%, but what does that mean? It sounds really impressive, but then you read the Zestimate is within 5% of the purchase price only 55% of the time, and that doesn’t sound so impressive.
I’d be curious to hear how their numbers are calculated actually. In the example above when the listing went live the Zestimate was $380K, which was 7.9% too high compared to the final price. But when it sold the Zestimate was
6.5% different. Which Zestimate is used for the calculation? Is it the original one when the listing went live or the most recent one? If it’s the most recent one, I can imagine if the algorithm gave more weight to the most current list price that it would be easier to get more accurate stats.
Those statistics come from Zillow, I’d prefer an unbiased study on accuracy. Zillow is a huge corporate business, so I’m a bit skeptical.
My favorite Zillow citation is the fact that their own CEO sold his home at a 40% difference from the zestimate. Your home sample is a great example of what an algorithm fails at…..buyer desirability.
Thanks Barb. Yeah, that story has certainly made the rounds. Zillow is a neat tool, and I don’t doubt its usefulness in today’s world. But it’s wise to understand its limitations.
Great observations Ryan and Zillow is always a fun conversation for appraisers. What I want to know is how often appraisers are within 5% of the sales price. I would love to hear a blog post where you contact some relocation companies and get this data because relocation appraisals are one of those rare times when appraisers can be tested.
Yeah, that’s a good question and it would be interesting to see that type of data. I think your comment underscores the reality that value is not easy – especially for complicated properties that don’t really fit the mold. Cookie cutter tract homes with twenty model matches are the dream, but the outlier properties are the challenging ones.
I will say for the sake of conversation, Zillow is within 5% of the sales price 55% of the time per their published accuracy chart (https://www.zillow.com/zestimate/#acc). I just wanted to clarify so we aren’t saying they are within 5% one hundred percent of the time.
Wow! Nice work Ryan. I’ve never tracked Zillow’s reporting like you did in this article. But your findings are very telling! It really confirms many of our concerns about Zillow. A person might get more reliable results blind folding themselves and throwing a dart at the neighborhood sales price range on a wall. I hope this article really opens the eyes for the public to see that Zillow is a toy, not a tool. Thanks for your eye opening reporting!
Thank you Jamie. I appreciate it. Trusting in Zillow is a big deal. A colleague actually just told me he testified in court and was asked about Zillow on the stand by the opposing party. What a glaring example to show how much a Zestimate means to some people that it would be brought up in a court of law.
Yet another example of people putting too much weight on Zillow’s results. That is pretty wild!
Another fantastic post! It’s so true people believe everything they see on Zillow. Another example is their listing of potential new foreclosures. It could take a year or more to actually be listed, but clients look at you like you have 2 heads when you say it’s not listed yet.
Thank you Jennifer. It’s interesting to hear about pre-foreclosures being advertised (or Notice of Defaults if that’s what these are). What a strategic way to be seen as the hub for what is truly on the market (and will be coming to the market presumably). I imagine that keeps people coming back too. Yet these properties might not hit the market for a year or ever for that matter.
Yes, and they make us look stupid because they believe Zillow.
Sigh. The advantage of being online only and not having a real estate license is there is seemingly less liability. It’s like you can say whatever you want without consequences. And because it’s online people tend to believe it. Right or wrong, that’s how it tends to work.
I don’t think of it as the “Public trusting Zillow to much”. I have had the zestimate discussion with property owners dozens, if not hundreds of times. For the most part they have a vague feeling that it’s not accurate but they really don’t know much about it. I like that Zillow puts the data out there and that the general perception is it’s not entirely accurate. It is a great conversation starter as it gives me a golden opportunity to “show my stuff” as an expert in the field. The same is true of the “price per square foot” conversation. Free marketing for me, yeah! Keep in mind I do mostly private appraisals and almost NO lender work so having this type of conversation is not going to rub some anal AMC the wrong way. If your a fellow appraiser stuck on the AMC treadmill then YMMV.
Thanks Mike. Killer point about Zillow being a conversation starter. Case-in-point. Here we are talking…. 🙂
I do private work too and I find most private clients are absolutely paying attention to Zillow. Well, usually during a divorce they are, but not so much during estate type work unless the property is going to hit the market. To be fair I think most people get the idea that it may not be accurate, but some people really do get stuck on the Zestimate. I find this to be the case when there are multiple parties inheriting a property. It seems like there is often one person who is giving too much weight to the wrong comps (or a Zestimate). And it seems like it’s often the person who is out-of-state too…. Once in a while a lawyer will ask me to explain why my value is different than Zillow because the client is asking, so I’d say even in private work we see people get hung up on Zestimates. I like the idea of “showing your stuff (expertise)”. Great thoughts.
I find that a well written appraisal report will often diffuse lawyers and out-of-state relative concerns. I will end the conversation with “The Zestimate (or price per sf) is a good negotiating tool but its not usually a good indicator of market value”. Folks seem to like that idea.
Great post, Ryan. My experience is that sellers tend to be more married to Zestimates than do buyers. Go figure!
Thanks Alice. I tend to agree with you. That’s been my experience also. My sense is overall buyers know the market much better than sellers because they’re scouring listings and looking intently at Zillow, Redfin, and MLS all day long for months. After a while they know everything on the market, they know what’s wrong with properties, they know when something is priced too high, and they know when something is priced too low. Even my own wife (who doesn’t work in real estate) saw a new listing hit the market yesterday and her thought was, “The price seems low.” I logged into MLS and right away we saw why the price was low (or rather why it made sense at that level). The roof was shot, the house was original, and most of all it has a very challenging location near commercial properties. Are sellers really giving the same attention to detail as buyers? Nope. But they are watching Zillow, and they can very easily get distracted by a Zestimate if they’re not careful. My advice to sellers? Realize more than ever that buyers know much more about the market than they ever used to before. They’re scouring everything out there while you’re just dabbling or looking at online value estimates. 🙂
Agreed Mike. There is still the conversation to be had though. Some people struggle to look past the Zestimate. I especially think having trend graphs makes a big difference. It becomes very hard to argue value is at $525,000 when zero competitive sales are anywhere close to that range.
When we Appraisers don’t know the ANSWER, or the MARKET VALUE of a property, we leave it open the other parties to do the Work for us…or have a CONTEST…
THE FOLLOWING IS REAL: (Google it)
“Announcing Zillow Prize, a contest designed to inspire the brightest scientific minds to compete to improve the Zestimate® home valuation algorithm.
Data scientists everywhere have a unique opportunity to work on the algorithm that changed the world of real estate – and win $1 million for improving it.”
I might add that NO ONE signs off on their Z-Estimate work– I keep looking for a name attached to each Z Value…have not found one… still looking…. We , as Licensed and Certified Appraisers, identify ourselves and SIGN our work every day… and stand by it !
(Keep up the good work, Ryan…)
Bruce J. Ford
26 YRS CA. Certified Appraiser
Thank you Bruce. I appreciate your perspective here. Yes, there is a contest going on. I imagine the algorithm will get better over time since computers are taking over the world… 🙂 But let’s not forget the human element and how difficult value is in the first place. We like to think real estate can be boiled down to a quick equation, but there are many moving parts.
I always love your articles about Zillow.
I am curious about one thing. Why did you choose this one house to follow? Was there something unique about it? Or does it seem to be a representative home in a popular price segment of your market?
I had my reasons Corey. I knew the agent involved and we talked about the property (great agent too). This is a very standard home in Carmichael. I will say Carmichael is a bit funky from street to street, so it’s not a tract area, but this is not a complicated valuation by any stretch. There is ample data and a ton of comparable sales out there. Moreover, stats clearly show the market didn’t drop by 8% in 30 days either. In short, this house is definitely not unique.
I actually have a few examples I’m following right now. I’m taking screenshots of other properties and just watching what Zillow does over time. I’m actually interested to have agents do this too before a property hits the market and throughout the process of price reductions and a sale. I’m very curious to hear if the Zestimate ends up changing based on the input original list price. I’d like to see if there is any pattern we can discern.
Thanks for the rationale. I do check Zillow and Redfin once I make a listing live, and both of their valuations seem to adjust to a nudge above list price, which I find interesting. Thankfully, I haven’t had to do price reductions in a very long time, so no data from me on that one. And, I should track later after closing, but I never think to. I’ll try to remember.
Thanks Corey. The problem of adjusting to the list price is it can end up impacting stats for accuracy. If a list price is realistic (which it probably should be), and the Zestimate adjusts to the list price, then that’s going to make the overall accuracy rates much better. In my mind the biggest test for accuracy would be the Zestimate figure the day before the property listed vs the selling price.
Great idea. I’m going to do that!
You pulled the curtain back to reveal the wizard, Ryan. Everything is not as you would believe. Just kidding, but you would think that consumers would have gotten wise with all of the articles online exposing them. It goes to show you that people believe what they want to believe if it makes them feel better. This is exactly why homeowners should not let their feelings get in the way when pricing and listing their home. You did a great job of documenting this one property to show what is going on at Zillow. Good luck on your speaking gigs!
Haha. Thank you Tom. This is fascinating stuff. I’d love to hear what people think when reading it too. I find many real estate agents don’t want to talk publicly about Zillow at times because The Big Z is such a powerful brand (or maybe the agent has ratings on Zillow or is paying Zillow for leads). Let’s open it up though. Anyone?
As an agent, no fear of the Big Z here whatsoever. It does provide a different perspective and it is part of my job to educate buyer and sellers to its relevance and place in the industry. It does provide useful in certain respects. Any company worth its weight should not fear critique but rather welcome it and use it toward improvement. We tend to grow more from our mistakes than our successes – that’s not say that we should make a habit of having more of the former than the latter.
Glad to hear that Alice. There is definitely space for critique for any professional or any valuation system. I do find many in real estate have strong feelings about Zillow, but they don’t tend to share them publicly as much online. I completely agree with you that there is a place for companies like Zillow in today’s world. I emphatically agree about learning from mistakes too. Our errors really can be our best teachers (well, if we learn from them).
Redfin does the same thing. Instead of actually having the algorithm they say they have that calculates each home’s value, they just have a basic formula where they add or subtract a couple of bucks to the list price. Look at this house that listed for $549,000: http://www.bubbleinfo.com/2018/04/13/redfin-estimates-based-on-list-price/
Thanks Jim. What an example. I clicked on your post. So Redfin actually revises their past history of estimates too? Must be nice when a property actually sells to claim that much accuracy….
Yep, and they flat out lie about it too. Here is what Glenn said:
Redfin CEO Glenn Kelman said in a conference call with reporters Wednesday that the company’s new tool is superior to other estimators because it is more accurate.
“Coming out of the gate, we have a lower error rate,” Kelman said. “And the reason we do is, unlike non-brokerage websites, we have 100 percent access to (multiple listing service) data.”
Wild stuff. Thanks Jim. The irony is even if they did have 100% access to data does not mean their interpretation is correct. That would be like me saying I have 100% access to every word in the Spanish language. That doesn’t mean I’m fluent though… Okay, maybe that’s not a perfect analogy, but it’s what I typed….
Redfin (and everyone else) has the ability to be 100% accurate. All they have to do is publish the exact sales price as the home’s value in the week or month of the sale.
But they would rather make the consumers think they have some magic algorithm that just gets close?
Why not just tell the truth – we can handle the truth!
Exactly. And it just goes to show we can make numbers and stats say whatever we want them to if we’re not careful.
Ryan, I’ve never asked you before but what is your stance on allowing other appraisers to cite and reference your blogs / articles on their own websites? My intention is not to take credit for any of this, but simply to provide the link. I think this is a wonderful article and something that everyone should be aware of (the more people that see this the better sort of thing). Thanks for everything you do!
Hey Jacob, thank you so much for asking. I appreciate the respect. I’ll always be sure to do that for anything you create too. I am always cool with people sharing anything on social media or linking to my stuff from anywhere. That’s always okay and it’s an honor. If someone wants to reprint something or actually republish a post in its entirety though, I definitely want to be asked. If someone wants to re-blog, I tend to prefer someone sharing a snippet of what I said, linking to what I say, and then that person sharing some of their original thoughts too. That way sharing is good for both parties involved (and it’s not just a cut and paste job). I actually have a sharing policy to make things easy and because people often ask. Thanks again. https://sacramentoappraisalblog.com/share/
I would like to speak with you on the telephone about crucial reporting issues I have with Zillow, Realtor.com etc. That can also mislead the general public and result in financial loss to sellers. I have been appraising for over 40 years. my telephone number is 814-254-3695. My name is Lawrence Fenimore in PA
Thanks Lawrence. Glad we can be in touch.
I definitely believe the consumer relies on Zillow too much. I don’t understand how Zillow got to be such a trusted brand. The site has never been good. The format is terrible and the information is inaccurate.
My standard answer to anyone who says, “Zillow said my house is worth …” is “That’s why it’s called a Zestimate.” Then I refer them to the Zillow disclosure page that tells how bad they are.
I can’t convey how much it annoys me that agents are paying thousands and thousands of dollars each month buying leads that are procured from our listings – which cost a fortune to obtain and maintain – using images we pay for and data we enter. We’re so lazy that we can’t or won’t generate our own leads. We’ve just rolled over and are actually paying them to disintermediate us.
Thanks Vicki. I really appreciate hearing your take. If anything consumers certainly aren’t relying too little on Zillow. I think that’s for sure. I think the public doesn’t often realize what Zillow is doing behind the scenes by collecting marketing dollars from agents. So when Zillow connects the public with a trusted agent, it’s really an agent who has paid for those leads. That’s probably obvious to many, but the public may not always be in tune with that.
It’s amazing how we tend to trust what we find online, so I”m not surprised to see this brand rise to prominence. Years ago during the housing crisis I recall President Obama having the CEO of Zillow moderate a discussion on housing instead of someone from NAR. That was an interesting move and very telling for the way the market was moving. And here we are today.
does Zillow make their historic ‘zestimate’ data available for export in bulk – or even for individual homes – so that we might compare neighborhoods in aggregate to actual market data?
Hi Alex. Thanks for the question. Unfortunately I do not know the answer, though I’ve never seen a download link on Zillow’s website, so my guess is NO. I’d like to see something like that. I’d also like to understand if their margin of error rate is based on the original Zestimate just before a property was listed or the Zestimate after multiple price reductions. One seems like it would be more accurate than the other.
I rarely find an issue with a Zestimate unless the seller is giving the wrong square footage, number of rooms, etc., not in line with the county records, In my experience, Zillow will investigate and correct that data, even over a seller’s wishes to obscure the truth. If you’re not in red hot area of the country, you’re probably having a hard time securing a Zestimate at what you paid for your home 10-12 years ago. That’s the new normal. There is no new buyer demand coming along to push that fact aside. So don’t blame Zillow. I’m currently watching a mid-range market in PA and I’m amazed by flippers who pay 10-15% under Zestimate for a fixer and then expect to get back 100% of what they put in. Haven’t seen it happen yet. You want any potential for that degree of margin success, then you better be in a bedroom community, within commuting distance of NYC.
Thanks Bill. I appreciate hearing your take. It’s surprising to me that you rarely have an issue with the Zestimate in terms of accuracy. If your area is saturated with tract homes, I get that more though. But otherwise there are many situations where values are clearly off-base (even in very conforming markets). This isn’t a knock at Zillow or coming from a place of anger. It’s just true.
One of the big issues I have with Zilow is them chasing the list price. This artificially boosts their accuracy rate. I actually found a glaring situation the other day with a property in Waco, TX that was previously on the show Fixer Upper. This property was listed for $950,000, which is astounding for the market for a one-bedroom small home. When it was listed Zillow’s estimate was heavily influenced by the list price (which you can see at the link if you wish). I don’t have an axe to grind here, but the public needs to consider this, and there is a heavy asterisk next to any “median” accuracy rate touted by Zillow if this is the norm. I wish I could embed the image here, but here is the post as an FYI: https://www.facebook.com/SacramentoAppraiser/posts/2340496155982044