Presentation on theme: "Valuation Office Agency Experience of Automated Valuation Modelling (AVM) in England Tim Eden BSc MRICS IRRV Deputy Director of Council Tax Valuation Office."— Presentation transcript:
Valuation Office Agency Experience of Automated Valuation Modelling (AVM) in England Tim Eden BSc MRICS IRRV Deputy Director of Council Tax Valuation Office Agency
Presentation overview A “fly through” of the VOA AVM experience including: Background The challenge The extent of data capture Changes in data management Process improvement and change Model development Training Achievements
Valuation Office Agency To be a world class organisation providing valuation and property services for the public sector
Valuation Office Agency VOA - Our purpose To provide a fair and robust basis for taxes, which help to pay for public services; and to help drive better use of property in the public sector by: compiling and maintaining accurate and comprehensive valuation lists for local taxation providing accurate valuations for national taxes delivering expert advice on valuations and strategic property management developing and maintaining a comprehensive and up to date property database advising policy makers on valuation and property issues
Valuation Office Agency VOA - Our structure CountryVOAVOA Deals with: - No. of Offices Local Domestic Property Taxation Local Non-domestic Property Taxation National Property Taxation Other Government Property Services England67Yes Wales7Yes Scotland5No Yes
Valuation Office Agency “Recent” Taxation History 1956 General National Valuation – to Point Rental Value 1963 General National Revaluation 1973 General National Revaluation 1982 General National Revaluation (Cancelled) Non-DomesticDomestic 1990 National Revaluation with Antecedent Valuation Date (AVD) 1 April 1988 Individual-based Taxation called Community Charge (“Poll Tax”) 1993 National Property Valuation for Council Tax to Capital Value Bands (AVD 1 April 1991) 1995 Revaluation AVD 1 April 1993 2000 Revaluation AVD 1 April 1998 2005 Revaluation AVD 1 April 2003 2007 Revaluation AVD 1 April 2005 (Postponed)
Valuation Office Agency Council Tax (CT) - Context CT + Non-Domestic Rates together raise c. £40bn pa CT raises £18.5bn pa 22m domestic properties in England 1.3m domestic properties in Wales Clients Department for Communities & Local Gov’t (DCLG) in England Welsh Assembly Gov’t (WAG) in Wales
Valuation Office Agency CT – 1993 List Bands (England) A - up to £40,000 B - £40,001 to £52,000 C - £52,001 to £68,000 D - £68,001 to £88,000 E - £88,001 to £120,000 F - £120,001 to £160,000 G - £160,001 to £320,000 H – over £320,001
Valuation Office Agency CT - Revaluation Local Government Act 2003 included requirement to undertake revaluations in Wales (2005) and England (2007) In Wales AVD 1 April 2005; list came into force on 1 April 2005 In England AVD 1 April 2005; list was to come into force on 1 April 2007 Draft lists were to be published 1 September 2006
Valuation Office Agency The Challenge New banding scheme not known Necessary to produce individual valuations & then overlay new bands Scale of task Complexity of housing types Vagaries of the market Inconsistent, paper-based records
Valuation Office Agency The Challenge – Market Vagaries Market analysis across England Free flow of money Historically low interest rates Fluctuating volumes in sales
Valuation Office Agency The Challenge – Housing Types Profile of Housing Stock 80% Houses/Bungalows 20% Flats/Maisonettes Over 1/3 rd of all flats in Greater London Around 1/6 th of properties at least 100 years old BUT high proportion of new properties in sales evidence!
Valuation Office Agency The Solution! Develop an AVM Skill-up staff Learn from others Cole Layer Trumble (CLT) – computer modelling expertise, AVM software KPMG – programme & project management support IAAO – statistical training Digitise data
Valuation Office Agency Began in 2002; worldwide research Mass data capture/enhancement Model development Many innovations and lessons These aspects have been very important to the success of the project Improving Business Decisions AVM design MRA models data collection 22 million values data cleansing data enhancement business case political issues value reviews staff planning and allocation VOA - AVM Journey
Valuation Office Agency Attribute Data Investigation Investigation commences in 2002 considering: Use of existing “attribute” codes already available on VOA IT database (“Group”, “Type”, “Area”) Valuer and caseworker engagement to determine useful codes Data availability & maintainability CT List Maintenance work External standards e.g. IAAO (6 year cycle) Impact of differing local practices Ability to undertake sales investigation activity Pre-contractor appointment; so limited AVM expertise
Valuation Office Agency Main Attributes to be digitised Architectural Style and construction quality – “Group” Property Type e.g. House Semi-Detached – “Type” Age (coded as era e.g. G – 1965-72) Area m² - external area (houses); internal area (flats) No. of rooms No. of bedrooms No of bathrooms Number of floors (houses); floor level (flats) Parking Conservatory (type and size) Outbuilding details Value Significant Codes
Valuation Office Agency What about Sales Data? Access to Land Registry and Stamp Duty Land Tax information Address irregularities – creates matching problems and manual effort Need to establish whether transaction at “Market Value”, special circumstances etc. Data understanding significantly assists AVM geared to support this process “Value” understanding working with AVM is key to the process
Valuation Office Agency Process Development Consider existing process The impact of AVM – scale and profile Skills Existing skills and abilities of targeted staff The need to re-train certain staff Resource availability New process to reflect Revaluation needs and business as usual IT constraints and capabilities Understand relationship between sales and model performance
Valuation Office Agency Before Sales received Suspect sales coded Sales searched to support each valuation case “on paper” At a Revaluation “Beacon Sales” reviewed and recorded “on paper” Property Data (Group, type, floor area, age, bedrooms, rooms, garage ) Property Data (Group, type, floor area, age, bedrooms, rooms, garage ) Models (MRA, algorithms and variables) Models (MRA, algorithms and variables) Locality Definition (area from which comparables derived) Locality Definition (area from which comparables derived) AVM AVM analysis output reviewed by qualified staff Decision to change, retain or include current data and coding Sales Data 1)New Sales continually updating analysis set 2) Value significance considered Sales Data 1)New Sales continually updating analysis set 2) Value significance considered After The Analysis Process
Valuation Office Agency Initial Issues with Modelling Process No first hand knowledge of sale Data time lag Data gaps due to “Permissible development” – the public need not inform the Billing Authority (BA) BAs internally not acting in a joined-up way – billing, planning & building control BAs not acting in joined-up way with VOA Are we maintaining lists or data or both? “Condition” not an attribute
Valuation Office Agency Integrating the AVM Investigation work commenced late 2002 Procurement process 2003 Collaboration of in-house team and successful contractors/partners - Cole Layer Trumble (CLT) Working on BA areas where data capture is advanced (note had only started in 2003!) Challenge of integrating AVM technology with existing VOA IT systems
Valuation Office Agency Getting Started Started with a simple additive model £ = B 0 + B 1 * size + B 2 * Detached * Size + B 3 * Terraced * Size + … + B n * Date of Sale * Size 1993 Band initially used to support analysis, but quickly stripped out of valuation models Postcode sectors used as proxy for location – VOA developed “localities” Postal areas are too crude Designed to support mail delivery, not to reflect influences on the property market!
Valuation Office Agency Performance Improvement (1) Better understanding of subject/sale data helped to: Determine usability rules for raw data Determine usability rules for a sale This understanding fed Comparable selection Model specification Consideration of rarely occurring variables Consideration of locality relationships
Valuation Office Agency High Correlations Group 55 separate codes Age 11 separate codes Type 32 separate codes Parking Floors/ Floor Level BedroomsRooms Area Bathrooms Correlation among variables High correlation compromises modelling stability
Valuation Office Agency Localities – illustrative photograph Local Authority housing estate Privately built housing estate Created bespoke localities (neighbourhoods)
Valuation Office Agency Mapping & Localities In excess of 10,000 localities Regular boundary review required Thematic mapping as part of process X-Y co-ordinate becomes necessary data X-Y suitable for comparable selection Issue of who maintains the data
Valuation Office Agency Performance Improvement (2) Move to multiplicative (log linear) model structure Log (£) = B 0 + B 1 * log (size) + B 2 * Detached + B 3 * Terraced + … + B n * Date of Sale + Bp * Log (Locality Adjustment Factor) where Bi determined from sales set using MRA This enabled improvements in: - Locality Adjustment Factor (time adjusted median price per square metre) Locality Grouping – support for comparable selection Central Modelling across the whole country
Valuation Office Agency Benefits of Central Modelling Central Modelling enabled: Central recognition of national modelling patterns and data issues Modelling “constraints” could be imposed by the centre ensuring model coefficients are consistently applied Effective direction of effort Calculation of market trend information to support VOA modelling and wider government market appreciation
Valuation Office Agency Performance Improvement (3) Integrating X-Y & Mapping allowed mass calculation of “plot size” Issues with map plots are: Plot bleed Non-alignment with the property transferred Maintenance of data
Valuation Office Agency Model Development - Lessons Create a multi-skilled and focused R&D team Select several representative areas to test Ensure proper debate on proposals Predict overall likely modelling gains Sense check & gather feedback from local staff Business model to relate model performance and cost/benefit Produce individual cost/benefit for each proposal Promote external verification e.g. IAAO
Valuation Office Agency Supporting Decision Making (1) VOA recognised management of a national valuation delivery required consistency in valuation decisions Property level confidence score required COD/COV inadequate existing banding too remote Score needed to support decisions at several levels: Strategic: Business case development Process definition Data collection & enhancement Tactical: Resource planning Operational: Resource allocation Value review or band review
Valuation Office Agency Supporting Decision Making (2) Confidence estimate for each subject property needed to reflect the aspects which would reduce accuracy: Data quality Data availability Market variability Model accuracy Adequate coverage for accuracy in the MRA Adequate comparables
Valuation Office Agency Confidence Model Confidence model based upon indicators from MRA and comparables Related actual errors to the dispersion amongst the comparables Comparable sales approach provides a number of other indicators for confidences: comparability distance weighted estimate (average adjusted) vs. MRA estimate the overall COV for the model on the sales set which tells us something about the underlying uniformity in the market and sales base MRA modelling includes a control model which considers current CT band So Confidence Model is: Likely Error = A + B dispersion of comparables + C x average distance between comparables + D x absolute value of (ln (mkt est. / control model est.) ) + E x model standard error + F x absolute value of (ln (weighted est. / MRA est.) )
Valuation Office Agency Confidence Model Maintenance Re-calibrated with every calibration iteration Periodically calibrated to valuer judgement of estimate output Can be used individually or aggregated for decisions at varying levels Developed between VOA, John Thompson (Cole Layer Trumble), Dr Jim Abbott (EDS) Presented at IAAO CAMA/GIS conference February 2006
Valuation Office Agency Summary of Models (1) Multiplicative MRA and comparable selection Operational delivery Undertaken locally Specified centrally Broad Based Model Market Trends Linearisations e.g. Group and Type variables to allow wider comparability Develop constraints for local models Quality assurance on operational delivery
Valuation Office Agency Summary of Models (2) Control Model Operates in background of local model Highlights data irregularities Confidence Model Uses data from all models Calibrated to valuer opinion Provides information for use at all levels
Valuation Office Agency Managing Business Change Addressing the time lag Review whole data process Use electronic transfer to bring in sales data Consider how to engage with taxpayer New skills Control modelling and train professionals Develop support and maintenance staff Rigorous application of project management methodology
Valuation Office Agency IT Development VOA has successfully integrated AVM technology with existing IT; this required: Improving data flow and management Enhanced mapping tools “Customising” 3 rd party “off-the-shelf” product Introducing workflow to manage delivery Lead-in time has created tensions IT supplier’s understanding of business and new technology does not happen overnight!
Valuation Office Agency Conclusions/Lessons Learned (1) Other data sources create issues: Often aggregated Currency – frequency of update Niche provider with limited depth Definition – what creates the data Cost – Is it worth it to you? Your records may be adequate and better than elsewhere anyway!
Valuation Office Agency Conclusions/Lessons Learned (2) How do you measure data quality Sale data quality Mostly measured by model Dependent upon proper sales review process Subject data quality Adoption of external standards not always possible e.g. IAAO 6 year cycle Data sampling using existing maintenance activities can support the view
Valuation Office Agency Conclusions/Lessons Learned (3) Time lag Understand the delivery requirement Address steps in the data process Work closer with data “partners” Central modelling can identify areas of concern Proximity to data You cannot easily replace local knowledge of market and meaning of sales.
Valuation Office Agency Conclusions/Lessons Learned (4) Modelling Define “attributes” with knowledge of AVM techniques Balance modellers desire for more data of imperfect market and cost to complete/maintain Stabilise data prior to commencement of modelling Valuing to “Band” does not loosen the rigour of modelling and data management
Valuation Office Agency Conclusions/Lessons Learned (5) Use AVM to direct effort Raw data analysis COD & COV are not the only measures “Frequently used comparables” Thematic mapping Use formal modelling hierarchy Don’t rush to deliver. Inefficiencies result Stabilise Model approach to ensure consistency Data stability and consistency
Valuation Office Agency Conclusions/Lessons Learned (6) Create clear lines of communication Local Management Local technical/modelling Training Make timely Consider delivery mechanism (e-learning, workshop) Project Management Structure A quality model can be let down by a poorly defined and delivered process!
Valuation Office Agency Some VOA achievements Digitisation of over 22 Million records Data completeness approaching 100% Over 120 surveyors trained in AVM techniques Market analysis nationwide, including mapping localities across the whole country Model performance well within recognised standards e.g. median COD of 353 BAs is 9.97 Proven IT platform for mass appraisal at national level.
Valuation Office Agency Questions? Tim Eden BSc MRICS IRRV Deputy Director of Council Tax email@example.com