Tuesday, May 7, 2019

Sales Ratio Study is largely Ineffective, if not Counter-productive

A Sales Ratio study examines the relationship of Market Values on the Assessment Roll to Time-adjusted Sale Prices (adjusted to the Valuation/Taxable Status Date). A Sales Ratio study, unlike an Automated Valuation Model (AVM), is not an econometric solution that could be used in any meaningful decision-making. Unfortunately, sales ratio studies are often developed and used to test the metallurgy and progression of assessment rolls. 

Since sales ratios are developed using sales complexes only, two very similar homes in a given neighborhood - with very dissimilar effective ages, say 15 vs. 50 - will be evaluated alike. A properly-developed AVM, on the other hand, will effectively evaluate the differences and return values that are different, yet statistically significant and econometric.

What does a sales complex comprise? 

-Sale Price
-Sale Date (to time-adjust sales to valuation/status date)
-Sale Validation (to ensure only arms-length sales are used)
-Classification (to ensure right class of properties are used)
-Market Value (from the Tax Assessment Roll)
-Assessed Value (when Residential Assessment Ratio or RAR is also required)
Additionally, some consultants retain a few other variables like Town (to evaluate sub-markets if it is a county-wide study) and Living Area (to evaluate normalized scenarios). Of course, sub-market and/or normalized ratios are statutorily rare.

So, why does a sales ratio study become an ineffective solution? Let's consider the following reasons...

1. Sales Ratio Studies are at best Heuristic analyses -- Most large and even medium-size tax jurisdictions have moved to primarily AVM-based tax rolls. Therefore, when the raw sale prices (generally time-adjusted) are compared with the scientifically-derived AVM values (to compute the sale ratio), it is not an apples-to-apples comparison anymore. Granted, sale prices do reflect property characteristics (in addition to location, etc.), but they are nonetheless highly subjective reflecting individual (un-equalized) economic behavior, including personal tastes and preferences (e.g., when one is bent on buying a pink house, one will overpay). Exterior wall and condition are actual modeling variables, while exterior color is not. Therefore, the presence of data variables will force AVMs (hence the tax roll values) to ignore those emotional premiums, while the sandalone sale prices in sales ratio studies will fail to differentiate and ignore them.  

2. Sales Ratio Studies do not require "Representative" Tests -- The underlying assumption of a sales sample is that it statistically represents the population it is derived from. But that assumption is not necessarily valid. When the sample is large, it tends to be representative at the body of the curve (between the 25th and 75th percentiles), but not necessarily on short (<25th percentile) and long end (>75th percentile) of the curve. The reason is simple: Not all segments of the market move in tandem. When a market starts its upswing, it usually begins at the lower end of the curve, followed by the mid-range and further up. Thus, without a proper representative test, a sales ratio study is at best a hit or miss. Obviously, the additional sub-market or normalized ratios remain equally unreliable.

3. Sales Ratio Studies do not require Price Segmentation Tests -- Sales ratio studies are perfect "one size fits all" meaning only a median-based ratio does the entire trick. In the absence of the price-segmented (<25th percentile; 25th to 50th percentiles; 50th to 75th percentiles; >75th percentile) ratios, it is at best a limited scope analysis. This is the primary reason why many tax rolls are regressive, i.e., why the middle-class neighborhoods heavily subsidize the rich neighborhoods. In order to minimize the incidence of such compensating errors leading to fair and equitable adjustments, State Boards and Industry Technical bodies must additionally (in addition to the median ratio) require the full price-segmented ratios.       

4. Sales Ratio Studies do not require Champ-Challenger Validations -- Before an AVM is finalized, it is optimized and then tested against a mutually exclusive hold-out sample (Challenger). If the hold-out test results are very similar, the model is considered final (Champ) and is ready to be applied on to the population. Of course, when it comes to sales ratio studies, there are no such requirements. A forward sales sample would be an ideal challenger. For example, if the statutory ratio is developed off of 2018 calendar year sales, it could be tested against a forward sales sample (comprising validated Q1/Q2-2019 sales). In order to bolster the forward sample, seasoned listings could be added as well. The forward sample test must produce comparable (to the statutory sample) results. Before rushing to make a biblical prophecy to confirm the roll results so the dusts settle, the concerned 3rd parties like the local newspaper reporters and independent review consultants should, at least, undertake this challenger test.

5. Sales Ratio Studies do not require Stratified Time Adjustments -- As explained before, not all segments of the market move in tandem, hence time adjustment factors in each segment are often different. Applying one median factor generally distorts both ends of the curve, forcing the outer segment values to move further away from the AVM (Roll) values. Again, the State Boards and Industry Technical bodies must require all ratio analyses - from sales sampling to time adjustments to error ratios - performed and broken down into statistically significant price segments. In fast moving markets, time becomes a critical issue so the time adjustment factors must be analyzed and applied by statistical segments. Alternatively, even when a median time factor is used in an AVM, it does not pose any threat as it interacts with other variables including location and gets corrected. 

6. Sales Ratio Studies do not require any meaningful Spatial Tests -- While a system-wide median ratio could be fine for a small and mostly homogeneous jurisdiction, it is not very meaningful for large and complex jurisdictions with multiple towns, boroughs, etc. For example, a system-wide coefficient of dispersion (COD) of 15 for New York City is neither very instructive nor very helpful, as a low COD of 9 for Staten Island, a fairly homogeneous borough in the City, may compensate for Brooklyn's 20 due to its highly heterogeneous housing stock. In this example, while Staten Island passes with flying colors, Brooklyn fails despite the fact that the City's overall COD remains compliant. The study of assessment equity, therefore, requires meaningful analysis by major spatial parts as well as the aforesaid economic/market attributes and segments.  

7. Sales Ratio Studies do not require the use of MLS data to Test Data Validity -- Granted, most tax jurisdictions tend to be more careful as to the quality of the sales data (easy picking by media, etc.) than the unsold properties. Yet, this data quality is nowhere as clean and up-to-date as the MLS data that are all professionally inspected and verified. Therefore, the State Boards and other Technical bodies should urge that the jurisdictions develop the ratio-eligible database after having compared the internal sales data with those of the MLS'. Needless to say, only the (arms-length) sales data as matched and confirmed by the MLS data should qualify for the ratio study. There is a long-term benefit to this exercise as well: By studying the unmatched data, an AI logic could be developed and applied on to the unsold population to isolate (or at least narrow down) the cases requiring immediate attention. An AI-driven auto-regressive data update process is always preferable (inherently more surgical) to the traditional cyclical approach (e.g., update all data on a 5-year cycle, although only a small percent might need attention/update).
8. Sales Ratio Studies do not require any Data Convergence schema -- Despite the fact that the raw sale prices are being compared with the modeled values, no data convergence schema is required to make the sale prices closely align with the data. As indicated before, absent the data variables, it is difficult to explain why two apparently very similar homes in very close proximity of each other are fetching fairly different prices. While an AVM will correct and explain that difference, sales ratio studies will have no explanations. Therefore, sales verification must also include "Effective Age" and "GIS Implications." If the GIS implications are noted alongside the verified sales, the highly impacted sales could easily be avoided, a priori. Similarly, the Effective Age ranges could effectively serve as a  sub-stratification criteria to help compare and analyze the truly similar properties. The point is, the plain vanilla sale price alone is inadequate to form the basis for any meaningful decision making.      

9. Sales Ratio Studies have yet to factor in the Impact of Cap on SALT deductions -- The new $10,000 cap on SALT deductions (including property taxes) has started to impact the high-end residential markets, especially in high-tax coastal markets. A report from the New York Federal Reserve concluded that the caps on taxes and mortgage debts "have negatively impacted the housing market" by lowering the sales volume. Of course, the market will take a while to manifest any meaningful medium to long-term effect from it. As the volume wanes on long end of the curve, sales must be adequately replaced by hand-worked appraisals. It would be imprudent and highly regressive to re-populate that stretch of the curve by drawing from the 50th to 75th percentile ranges. That type of re-population or idea perhaps works in physical sciences, but not in economic sciences. State Boards and Technical bodies must recognize and act on this emerging trend.

Sales ratio studies urgently need a fresh and forward look. The median-based one-size-fits-all concept has to be replaced with some meaningful market segmentation analyses, coupled with a handful of spatial and economic attribute tests.

Meanwhile, as the highway signs say, "Your tax dollars are at work."

Thank You.

Sid Som
President, Homequant, Inc.
Coming Soon...Sid's New Book: 
Life, Logic and the Power of Nine (Branding)

No comments:

Post a Comment