How we know: checking our heat pump numbers against real homes
Any calculator can print a savings figure. This page shows how ours is made, how it compares with measured homes, every error we have found and fixed, and what it still cannot do.
Data as of . Hover any number to see the file it came from, or download them all.
HeatPushers estimates what a heat pump would cost to run in your house, and what it would save against the heat you have now, hour by hour through a year of your local weather, for each listed product.
It is good for comparing products and for answering "is this worth it here?" before you call an installer. Fed a year of monthly bills, it found heat loss within 20% of what daily data showed for 75% of the UK heat pump homes and 77% of the Scottish gas homes we tested.
It is least certain about ducted systems, where it still reads more efficient than field studies measured, and about weather colder than a unit was ever tested in. It does not replace an installer's room-by-room load calculation.
It will also tell you when a heat pump does not pay. With gas at $1.06 a therm and power at $0.17 a kWh in Minneapolis, gas heat costs about $11 per million BTU and heat pump heat about $17, so none of our products beat a high-efficiency furnace there.
We use the last year of hourly temperatures from the nearest NOAA weather station that has data. Every hour counts: a heat pump that does fine on a mild day but struggles on the coldest ones is judged on how many hours your town actually spends at each temperature.
People, appliances and lights warm a house, so it needs no heating until it is well below the thermostat setting outside. We use the standard reference home's internal gains. For the Boston example house that puts the start of heating at 63.3°F outdoors.
Without bills, we estimate heat loss from the size, number of stories and age of the house, using typical insulation and air leakage for each era. With bills (paste them, or load a Green Button file), we fit your house to them instead. The fit runs in your browser; your bills never leave your device. Gas, oil and propane bills usually include the water heater, so we take out 20% unless monthly bills let us measure it.
Typical houses by age
Built
Wall R
Attic R
Window U
Air leakage (ACH50)
Before 1950
4
4
0.90
14.5
1950 to 1969
5
9
0.90
11
1970 to 1989
14
16
0.55
8.5
1990 to 2009
16
30
0.35
5
2010 or later
22
44
0.30
3.5
How the bill fit works
fuel per day = base use + slope × degree-days per day below the balance temperature
We try every balance temperature and keep the one that fits best (the PRISM method, Fels 1986). The slope, times the fuel's heat content and the furnace efficiency, is the house's heat loss. A robust fit keeps one odd bill from steering the result, and resampling gives the range we show.
Each product's heat output and efficiency come from its listing in the NEEP cold-climate heat pump list: lab tests at 5°F, 17°F and 47°F, plus the coldest temperature the maker tested. Between tests we draw straight lines.
Rules we added after checking the data
Listing points that can't be right are dropped: heating COP below 1 or above 7, or above 5 at 17°F and colder.
Below the coldest test, output fades in a straight line to zero 30°F lower, and efficiency never rises as it gets colder.
Running below minimum output means cycling on and off; efficiency is cut using the industry default cycling factor (Cd = 0.25).
Ducts in an attic or crawlspace lose heat before it reaches the rooms. We charge that loss to whatever uses the ducts: your furnace, your central AC and a ducted heat pump alike. A ductless heat pump skips it, and gets credit for that.
Share of heat that reaches the rooms, by duct location
Ducts run through
Heat delivered
The heated space
100%
An unheated basement
70%
A crawlspace
78%
The attic or garage
72%
Seasonal distribution efficiency with typical leakage, LBNL-43371 Table A3 (ASHRAE 152).
Field studies consistently measure less than the lab rating. So beyond duct losses, we scale each heat pump's listed efficiency: ductless to 92%, ducted on ducts sized for it to 95%, and ducted on reused furnace ducts to 85%. These are judgments, set against the field studies below, not measurements of each product.
You can enter the blower-door number (ACH50) from an energy audit. We divide it by 17 to get the year-round natural rate the model uses, a rule of thumb from Lawrence Berkeley Lab.
Each hour, the heat pump carries the load up to its output at that temperature, and your current heat covers the rest. By default each hour uses whichever heat is cheaper. We pick the number of units with the lowest cost over their lifetime: equipment, installation and energy. Prices for gas, oil and propane are state averages from the EIA unless you type your own.
We took two public datasets with detailed meter data, added it up into monthly "bills", fitted each house exactly the way the site fits your bills, and compared the result with a fit of the same home's daily data.
93 UK homes with heat pumps and heat meters from heatpumpmonitor.org (of 150 we downloaded, the ones with a complete heating year). Heat meters record the heat actually delivered each day.
108 Edinburgh homes with gas boilers from the IDEAL study (of 255), with gas meters and room temperatures.
Heat loss, mostly. In the median home, monthly bills gave 1.11 times the daily-data heat loss in the UK heat pump homes and 1.00 times in the Scottish gas homes. That was 1.19 and 1.12 before the fixes in our changelog.
The range we show contained the daily-data value in 99% and 100% of homes. Honest, but wide: a median of 82% and 126% of the value itself, from bottom to top. A year of bills narrows down your house; it does not pin it.
Exactly when your house starts needing heat. The Scottish homes kept a median of 64.4°F indoors, but the fitted start of heating did not follow indoor temperature (correlation 0.14). Only the typical value is trustworthy.
Your non-heating use, where summers are cool. In 38 of 108 Scottish homes the fit could not separate hot water and cooking from heating.
Next winter's bill. Fitted on one winter, the model predicted the next one off by a median 8.3% (19 UK homes) and 20% (13 Scottish homes). Simply scaling last winter by how cold each winter was did as well or better: 7.4% and 15%. We don't use the fit to forecast bills.
On each UK home's 5 coldest days (a median of 38°F), heat loss from monthly bills predicted 1.08 times the heat the meter recorded; the daily-data fit predicted 1.02. Before the fixes it was 1.20.
The likely cause: compared with the fit, homes used +19% in October and -20% in April. Spring sun and heating switched off for the season both make monthly use look more sensitive to weather than it is. The bias errs on the safe side for "will it keep up?", but can suggest one unit more than you need.
For context:84 of these homes list their installer's design heat loss. The measured heat loss was a median 1.03 times the installer's figure, but anywhere from 0.73 to 1.49 times for the middle 80% of homes. Professional estimates have wide errors too.
Limits: both datasets are British, with mild, damp winters. No public dataset pairs US utility bills with metered heat pump use in the same home.
Lab ratings are measured under fixed conditions. Real homes add ducts, cycling, defrost, snow and settings. We compared our engine's seasonal efficiency with field studies that metered heat pumps in Boston-like and Minneapolis-like winters. The engine figure is the median across every listed product of that type, for an all-electric house with ducts inside, including any backup heat.
Measured in the fieldOur first engineOur engine now
Ductless now lands close to what the field measured. Ducted is still the weakest part of the model: our median remains above the measured value in both climates, and the studies don't say where those ducts ran. If you are weighing a ducted heat pump, lean toward the low end of our estimate.
Across all 10 studies, cold-climate ductless units measured a seasonal COP of 2.2 to 3.2, ducted systems 1.6 to 2.5, and units running below 0°F 1.5 to 2.4. A COP of 2 means two units of heat for each unit of electricity, twice as good as baseboard heat.
Minnesota, Twin Cities metro + northern MN. HDD not given; most of MN has heating design temp below −10 °F. 6 ducted (4 flex-fuel with LP furnace backup and a 5-10 °F changeover; 2 all-electric Mitsubishi with electric plenum booster) + 2 ductless (Mitsubishi Hyper Heat, with baseboard backup). 8 homes.
Measured: Table 3 "Annual System COP" (includes backup): ducted flex-fuel 1.0-1.3; ducted all-electric 1.9 metro / 1.8 northern MN; ductless 2.3 metro / 2.2 northern MN (ductless serves ~50% of load)
Massachusetts and Rhode Island. Ductless mini-split, single- and multi-head. 78 of 152 units were "cold-climate" models (all single-head). 152 homes (metered n per season 57-99).
Measured: Field HSPF is shown only in figures (Figs 30-34, 49-51), and no single seasonal number appears in the text. Text: field HSPF "generally fell below the nameplate rating". The top 25% of light and heavy users "approached or exceeded a 10.0 HSPF" (about COP 2.9 [derived]). Average rated HSPF: 10.3 overall, 11.0 for cold-climate units
Pacific Northwest, 5 clusters (DOE zones 4C, 5B, 6B). Ductless, mostly single-head, displacing zonal electric-resistance heat in the main living zone. 95 homes metered; 69 with COP.
Vermont; both winters mild (−10 °F reached only about once in winter 2016). Ductless cold-climate mini-splits (single- and some multi-head). 77 units at 65 service accounts.
Measured: Field "HSPF" 10.7 kBtu/kWh (≈ COP 3.1 [derived]) vs nameplate 11.9 HSPF, i.e. "approximately 88%" (exec summary says ~90%). Excludes standby energy
Massachusetts and New York; average design temperature 5.2 °F (MA) and 1.8 °F (NY). Ductless single- and multi-zone, ducted central/compact, and mixed; whole-home vs "primary with backup" applications. 43 homes / 73 outdoor units.
All 16 Maine counties; 12 homes north of Lincoln, 9 in Presque Isle and north. Predominantly ductless mini-splits (single- and multi-zone, HESP/AHI rebated). 124 homes / 170 outdoor units / 210 indoor units.
Measured: Mean seasonal COP 2.57, including defrost. TRM assumed 2.48; earlier HESP evaluation metered 2.7
Cold/very-cold US sites and Canada (some Canadian sites were lab homes). Milder-than-normal winters; only 8 of 22 sites had ≥10 h below 0 °F. Centrally ducted ccHP prototypes (8 manufacturers), 3-5 ton, with staged electric auxiliary heat. 22 units.
Measured: No seasonal COP reported. Heat-pump-only median COP 1.9 in the 0-5 °F bin; medians 1.6-2.7 across bins below 30 °F. These exclude auxiliary heat and defrost periods
Real homes check the house side. To check the engine itself, we ran the live page in a headless browser on 193 test houses: 11 climates from Miami to Fairbanks, every kind of current heat, small and large houses, old and new, whole-house and one-room jobs, DIY and hired installs. Each one costs all 772 products: 148,996 results, each checked by script.
The median heat pump's seasonal COP in each test climate, before and after the fixes in our changelog. Warmer places should score higher, and do; the first engine scored them far too high.
Climate
First engine
Now
Chart
Miami
4.91
3.74
Phoenix
5.23
3.76
Houston
4.39
3.64
Atlanta
4.06
3.43
Denver
3.67
3.19
Seattle
4.42
3.67
Chicago
3.47
3.05
Boston
3.72
3.30
Minneapolis
3.25
2.82
Burlington, VT
3.36
2.97
Fairbanks, AK
2.63
2.41
What still looks off: 4,518 results fall outside a plausible 1.3 to 4.5, and 1,178 "keeps up all winter" verdicts depend on weather colder than the unit was tested in, where we rely on the fade described above.
Every check above found problems. Here they are, newest first, in plain words. Each has its own link.
· found November 4, 2026
A spec-sheet typo made one unit the top pick for Fairbanks, Alaska#
What was wrong
One LG listing (id 2404) gave its efficiency at −4°F as COP 21.39. Its other heating figures were 1.73 to 4.35. The engine believed it, so for an electric-heated house in Fairbanks the page called it "Best value" and said it would save $10,462 a year. It was the top pick in 23 of 36 Fairbanks test houses.
The fix
Listing points that are physically implausible are now dropped: heating COP below 1 or above 7, or above 5 at 17°F and colder. If two points list the same output, the lower efficiency is kept.
Now
That unit is now a pick in 0 test houses. The highest seasonal efficiency the engine gives any product in Fairbanks fell from 29.95 to 5.79.
Heat pumps were assumed to keep full output far below their coldest test#
What was wrong
Below the coldest temperature a unit was tested at, the engine kept its output flat. So the page said units could carry the whole house down to −45°F in Fairbanks. 2,682 such "carries the whole load" claims were at least 10°F below the unit's coldest test. With no backup heat, that is dangerous advice.
The fix
Below the coldest test, output now fades in a straight line to zero 30°F lower, and efficiency never rises as it gets colder. This is a judgment, anchored on a Minnesota field test where a small unit delivered about 20% of its rated 5°F output at −20°F to −25°F.
Now
"Keeps up" claims that rely on weather colder than the unit's test fell from 3,816 to 1,178, and those now use the faded output. The top Fairbanks pick for an electric-heated house now covers 93% of the heat, not all of it.
Units with no listed coldest test lost all their cold-weather output#
What was wrong
76 products have no "lowest tested temperature" in the listing data. The page filled in an empty test row for them, which read as zero output at 5°F and colder. This one ran the other way: good cold-climate units ranked too low.
The fix
The empty rows are gone; those products use the tests they do have.
Seasonal efficiency was too high, especially in mild climates#
What was wrong
When a heat pump runs below its minimum output it cycles on and off, which wastes energy. The engine ignored that, and kept efficiency rising in warm weather. Across all test houses, 34,495 product results had a seasonal COP outside a plausible 1.3 to 4.5, 34,281 of them too high. Median in Miami: 4.91.
The fix
Running below minimum output now costs efficiency (the industry default cycling factor, Cd = 0.25). Efficiency no longer rises above the 47°F test. Real-home factors and duct losses (below) also apply now.
Now
Results outside 1.3 to 4.5: 4,518. Median in Miami is now 3.74.
"Best value" was shown on picks that cost more every year#
What was wrong
In 31 test houses, the pick labeled "Best value" would raise the yearly bill. Example: Minneapolis with a 96% gas furnace, where cheap gas beats every heat pump.
The fix
That pick is now labeled "Lowest extra cost" when it loses money, and "Cheapest to own" when it does not pay back within its lifetime. When nothing saves money, the page says so and compares the cost of heat from each source.
Now
"Best value" on a money-losing pick: 0 test houses.
A setting hidden in a collapsed section read as blank, so the weather data was never scaled to exactly a year. It happened in 193 of 193 test houses. Houston's weather covered 346.2 days, so its costs came out about 5% low.
The fix
The setting is read correctly now.
Sources: validation/sweep/REPORT.md
Monthly bills overstated heat loss, and mild winters were refused#
What was wrong
Against 93 UK homes with heat meters, heat loss fitted from monthly totals came out 1.19 times the daily-data fit. The check for "enough cold weather" turned away 21 of 93 complete years because UK winters are mild. In 108 Scottish gas homes, the uncertainty range was 368% of the value: honest, but useless.
The fix
Daily data is added up into weeks first. The temperature where heating starts is kept between 50°F and 68°F. The cold-weather check now asks for at least 500 degree-days of heating instead of counting cold months. Fits that land far off are set aside.
Now
Refused: 0. Monthly vs daily heat loss: 1.11. The Scottish range width: 126%.
Ducted heat pumps looked far better than field studies measured#
What was wrong
The first engine had no duct losses and took spec sheets at face value. It gave a ducted heat pump in Boston a seasonal COP of 3.26. A 2022 field study in Massachusetts and New York measured 2.25.
The fix
Duct losses now depend on where the ducts run, and they are charged to whatever uses them: the furnace, the central AC and a ducted heat pump alike. So a ductless unit gets credit for skipping them. Real-home factors are applied on top (ductless 92%, ducted on ducts sized for it 95%, on reused furnace ducts 85% of the listed efficiency). Yukon retrofits on old furnace ducts measured about 1.3, against 2.5 on ducts sized for the heat pump. A later review found duct losses counted twice; that was fixed before launch.
Now
Boston ducted is now 2.71, still above the field figure. See the comparison above.
The 1960s preset showed 0.65 air changes per hour. A reader read that as a blower-door number, which would be an unusually tight house. It was the year-round natural rate, but the page did not say so.
The fix
Air leakage is now entered and shown as the blower-door number (ACH50) an energy audit reports, and divided by 17 for the natural rate the model uses (an LBL rule of thumb).
Now
The 1960s preset shows 11 ACH50; a national database of blower-door tests has a median of about 12 for houses from that decade.
An outside review before launch found 10 more bugs#
What was wrong
Before the new page replaced the old one, independent reviewers went through the code and the page as a homeowner would. They found:
Duct losses were counted twice: once on the house and again on the system that uses the ducts.
A product's detail panel could keep showing numbers from before the house was costed.
The cold-weather output fade was also applied to cooling.
Boilers were charged duct losses they don't have. Gas and oil boilers are now their own choices.
With electric baseboard as backup, the baseboard never took over when the heat pump became less efficient than the baseboard.
Daily usage data lost some days when it was added up.
Green Button downloads lost their time-zone marker.
Fuel backup was labeled in kWh.
Opening a product didn't change the page address, so it couldn't be linked.
Leaving a bill out of the fit was not always applied.
Sources: Notes/heatpushers-home-model.md
Our two tools disagreed on when a house starts needing heat#
What was wrong
The comparison assumed heating starts at 60°F outdoors, the calculator at 62°F. Same house, about 9% apart.
The fix
Both now work it out from the house: thermostat setting minus free heat divided by heat loss. Leaky houses start needing heat at a warmer outdoor temperature, tight ones later. With bills, the fit measures it directly.
Now
Both tools agree to the dollar ($1,327 a year for the Boston example house).
The equipment list is behind. NEEP revised its listings on ; we serve a copy from . In a sample of 40 products, 1,244 of 1,365 figures (91.1%) still match. A few products changed enough to move their share of your heating a lot.
Air leakage is the same all year. Real houses leak more when it is cold and windy, which matters most on the coldest days.
The real-world factors and the cold fade are judgments. They are set against field studies, not measured for each product.
No room-by-room load calculation (Manual J). We estimate energy and cost for the whole house or the part you choose; we don't design ducts or tell you which rooms a head will reach.
No thermostat interplay. How a heat pump and a furnace on separate thermostats share the work in practice is not modeled.
Sun. Free heat from sunshine is not in the age-based estimate. Bills capture it.
Not tested on US bills. Our real-home checks are British. The cleanest end-to-end test, bills before and metered heat pump use after in the same US house, does not exist in public data yet.
If you have a year of bills from before and after installing a heat pump and would share them, that is the single most useful check we could add. Found something wrong on this page or in the tool? Email tophatmonkey@gmail.com; fixes land in the changelog above.
heatpumpmonitor.org (OpenEnergyMonitor community): heat pump heat-meter and electricity data. Licensed CC BY 4.0. We summed daily data into monthly totals and fitted our model to it.
IDEAL Household Energy Dataset, University of Edinburgh, doi:10.7488/ds/2836: gas, electricity and room temperatures for 255 homes. Licensed CC BY 4.0. We cleaned gas sensor dropouts and built monthly totals from it.
Methods: ANSI/RESNET 301 internal gains; PRISM (Fels 1986); LBNL-43371 and ASHRAE 152 duct efficiency; AHRI 210/240 cycling degradation; the 10 field studies listed above.
Every number on this page is generated from our validation outputs by a script, with the source file and line kept beside it: summary.json (generated 2026-11-05). Cite this page as: HeatPushers, "How we know", https://www.heatpushers.com/how-we-know, updated November 5, 2026.