Xiaomi Auto Mode Leaves Air Unsafe for 86% of hours

For the last three years, I’ve been saying clean air doesn’t need to cost an arm and a leg. So when Xiaomi came out with a purifier for under 1,000 RMB, I assumed it works just fine. But I had never gotten around to testing it—until now. And what I found shocked me.

The Test

Smart Air co-founder Anna ran a brand new Mi2 in the same 15m2 bedroom in Chaoyangmen where we’ve tested the DIYs, IQ Air, Blue Air, and Philips. Anna ran six tests on auto mode and six tests on the highest setting (which the Xiaomi oddly calls 最爱档 -“most love mode” or “Favourites mode”).

Minion

Anna turned the purifier on in the morning when she left home, and turned it off when she returned home, so there was no one in the home during the tests. The doors and windows were closed during the tests. (More details on the test method and all original data are here.)

A Dylos Pro tracked ≥ 0.5 micron and ≥ 2.5 micron particle counts during the test. Test data shows these 0.5 micron readings correlate highly with the US Embassy’s PM2.5 readings  (r = .90).

As in my earlier tests, I calculated effectiveness as (the number of particles before turning the purifier on) versus (the average number of particles over the last four hours).

Results

The Xiaomi scored as one of the worst purifiers I’ve ever tested. On average, it removed only about 60% of 0.5 micron particles over the last 4 hours of the test.

Xiaomi effectiveness

Here’s how the Xiaomi results compare to earlier results from other purifiers using the same method, same particle counter, in the same room.

Average % of particulates removed

Below is a normal test day. I put a dashed red line representing the World Health Organization’s (WHO) 24-hour PM2.5 limit (25 micrograms).

Xiaomi room test

The Xiaomi Really Wants To Be in Auto Mode

What’s wrong? The key is that—even on the highest setting—the Xiaomi reverts back to auto mode after 3 hours. To be sure this is what the machine was doing, we put the Xiaomi on the highest setting and tracked noise levels overnight.

Room

You can see the noise remaining high for three hours, then it returns to the ups and downs of auto mode:

Xiaomi noise

Here is the same graph with particle counts (measured by the Dylos). The particle counts rise every time the machine turns off.

Xiaomi noise vs PM 2.5

The data shows that the Xiaomi has a rather low standard for “safe.” The Xiaomi is turning off when PM2.5 reaches 40 micrograms, which is far higher than even the WHO’s 24 hour limit. Then it turns on again when PM2.5 gets up to 70 micrograms—almost three times the limit.

Really? It can’t be kept on high?

The fact that the Xiaomi can’t be kept on continuously all night is so strange that Anna asked Xiaomi’s customer service about it twice to make sure we’re not making a mistake. They confirmed that, no matter what, the machine will go back to auto mode after three hours (full transcript available in the supplemental materials).

Blurred

How often is the air unsafe?

I calculated the percent of hours that the air was unsafe during the tests using this rule:

After the purifier was on for at least 1 hour, for any hour where outdoor air pollution was unsafe (> 25 micrograms – the WHO 24 hour limit), how many hours was indoor air also unsafe (>25 micrograms)?

The Xiaomi left air unsafe for a shocking 86% of the time. The other similarly sized machines in my earlier tests left air unsafe only 7-16% of the time.

% of unsafe hours

Now perhaps the Xiaomi is using the looser Chinese standard for clean air of 35 micrograms. Even with that high number, 77% of the hours the Xiaomi was running, pollution levels were still above the limit.

Wait, are you sure sure?

I wanted to be really careful about this. I’ve seen other tests showing that the Xiaomi 1 works just fine, including these tests from Dr. Saint Cyr (although I’ve seen people write about flaws too, such as this guy who kept the filters wrapped in plastic, turned it on turbo mode, and the app told him his air got miraculously cleaner). So I double and tripled checked the data:

  1. We tested with a different particle counter in a different room.
  1. I re-analyzed the data throwing out any days with large fluctuations in outdoor air.
  1. I analyzed days when outdoor pollution was low to average (< 150 micrograms).
  1. I compared it to other tests done just two weeks before in the exact same room with the exact same particle counter.

None of these analyses changed the result (see details here).  Note however, that we did only test one Xiaomi Mi2, so there is a possibility another machine would perform differently. I invite anyone with a Mi2 to replicate my tests using the same method, and I’m happy to publish the results.

What now for Xiaomi?

From this data, my conclusion is that the fan and the filter are fine, but the Xiaomi has a programming flaw. Even if I use it on the highest setting, I’ll be breathing air far above the safe limit for most of the night. That’s a problem.

Is it just a Xiaomi problem?

To be fair, I cannot recommend any auto mode I’ve tested. I’ve tested the Philips AC4072 on auto mode, and it averaged 59% reductions in 0.5 micron particles, which is pretty close to the Xiaomi results. So I think this is more of a problem with auto modes rather than the Xiaomi specifically.

Yet I can still recommend the Philips. Why? It can stay in medium or high as long as you want, and my data shows it works fine all night as long as it’s not on auto mode.

I’m confident that Xiaomi can fix this flaw by simply allowing people to run it on high without reverting back to auto mode. But until that happens, I cannot recommend using the Mi2.

Open Data

As always, I’m publishing more details on the test method and the original data for fellow nerds.

How Accurate is the 1 Micron Dylos?  Official PM2.5 Comparison Test

dylos-1-micron-guangzhou-cover
Everyone who buys an air quality monitor wants to know how it compares to the official numbers. When I bought a Dylos DC1100 Pro laser particle counter,  I did a comparison test with the US Embassy. I put my Dylos Pro outside the window of my apartment (at Nanluoguxiang) 70 times and compared the numbers to the US Embassy’s Twitter feed at the same time. I found that the two numbers were very strongly related: r = 0.89 (remember the highest possible correlation is 1).

dylos-pro-vs-us-embassy-en
I did that test with the Pro model, but there’s also a standard model that’s $70 cheaper. At Smart Air, we call it the Standard Dylos.

How does this cheaper machine compare to the government measurements? If it’s similarly accurate, great! That means people can save money and get roughly similar results.

dylos-pro-vs-standard-en

Pro vs Standard

The Dylos Pro and Standard are almost the same judging from their front looks. So what’s the real difference?

The key is the size of particles they measure. Both of them have 2 size ranges. For the small particle channel, the Dylos Pro measures particles .5 microns and above, while the Standard Dylos measures 1 micron and above. On the large particle channel, the Pro measures 2.5 microns and above, while the Standard measures 10 microns and above. That means the Pro measures an extra range from .5 to 1 micron.

dylos-pro-vs-standard-measurement-range-en

The machine governments use is called the Met One BAM. It measures particles 2.5 microns and below. If we put the measurement ranges of the BAM, the Dylos Pro, and the Standard Dylos together, here’s what it looks like:

dylos-pro-vs-embassy-vs-dylos-en

The Pro overlaps a bit more the government machine than the Standard. For that reason, I’d expect the Pro correlates better with the government machine. But just how well do they compare empirically?

The Test

Luckily I’m not the only nerd interested in air pollution problem. An American living in Guangzhou bought the Standard and teamed up with me to test it against the US Consulate.

From November 12th to 17th, she collected data with her Dylos Standard 70 times outside her apartment,which is close to Sun Yat-Sen University in Guangzhou. She also recorded the numbers of PM2.5 concentration released by the US Consulate.

dylos-1-micron-test-map

Results

Remember how well the Pro correlated with the US Embassy? Here’s what I found before (r =.89).

dylos-pro-vs-us-embassy-en

Later I did another test by putting my Dylos Pro about 100 meters away from the US Embassy (much closer than my apartment). This time it turned out that the two numbers correlated at r = .9 – slightly higher than in the first test.

dylos-pro-vs-us-embassy-closer-en

How does the Standard Dylos compare? Here’s what we found:

standard-1-micron-dylos-vs-us-embassy-en

They correlate at r = .85! Although that’s a bit lower than the correlation between Dylos Pro and US Embassy, the two numbers are still strongly related. I think that result would surprise most people.

Converting the Dylos 1 Micron Readings to PM 2.5

Using this data, we can get a rough formula to convert between the Dylos 1 micron and the official numbers. For example, the 24-hour WHO limit is 25 µg/m³, which is approximately 1,250 on the Standard Dylos.

1-micron-dylos-to-us-embassy-en

The simplest formula to convert the Dylos 1 micron to PM 2.5 micrograms is:

Dylos 1 micron count × 0.02 = PM 2.5 µg/m3

If we use that formula to convert the Dylos numbers to micrograms, here’s how well the numbers compare:

1-micron-dylos-formula-en

On average, the formula was off from the government numbers by 9.04 micrograms.

Conclusion

The Standard correlates highly with the government numbers—only slightly less well than the Pro model.

Limitations

The conclusion is based on only one test in Guangzhou. We could draw stronger conclusions if we had more tests done in different locations and in different seasons.

Open Data

As always, I’m making the data and more testing details publicly available for fellow nerds.

P.S. Thanks to Yiwen Wang for help translating and creating figures.

Cost of air pollution

Death in the Air Infographic by World Bank

The World Bank released a new report titled “The Cost of Air Pollution: strengthening the economic case for action” and in it they detail how air pollution is now the 4th leading risk factor for deaths worldwide. That’s worse than the deaths attributed to alcohol and drug use, HIV/AIDS, and even malaria. Besides the other reasons for reducing air pollution (climate change, our health, etc.) the economic one is probably the one that will communicate the strongest to everyone as air pollution costs the global economy in terms of foregone labor income to the tune of $225 Billion each year globally.

Click here for full report.

Click here to view the infographic in higher resolution.

Air pollution has emerged as the fourth-leading risk factor for deaths worldwide. While pollution-related deaths mainly strike young children and the elderly, these deaths also result in lost labor income for working-age men and women. The loss of life is tragic. The cost to the economy is substantial. The infographic below is mainly based on findings from The Cost of Air Pollution: Strengthening the economic case for action, a joint study of the World Bank and the Institute for Health Metrics and Evaluation (IHME).
Air pollution has emerged as the fourth-leading risk factor for deaths worldwide. While pollution-related deaths mainly strike young children and the elderly, these deaths also result in lost labor income for working-age men and women. The loss of life is tragic. The cost to the economy is substantial. The infographic below is mainly based on findings from The Cost of Air Pollution: Strengthening the economic case for action, a joint study of the World Bank and the Institute for Health Metrics and Evaluation (IHME).
Source Pollution

10 Facts About Air Pollution

We get many questions about air pollution in our office, and understandably so. It’s a topic that isn’t well understood or well-reported about in certain parts of the countries in which we work. In some cases, it is difficult to distinguish research-backed findings from common beliefs. To contribute to collective learning, here is a quick list of top 10 facts about air pollution.

  1. Air pollution is made up of chemicals, particulates, and biological materials. Common  components include, but are not limited to: nitrogen, sulfur, carbon monoxide, carbon dioxide, dust, and ash.
  2. Air pollution is caused by both human and natural contributors. Industries, factories, vehicles, mining, agriculture, forest fires, volcanic eruptions, and wind erosion all cause air pollution.
  3. According to the Global Burden of Disease report (2013), air pollution contributes to more than 5.5 million premature deaths every year. Another report by the International Energy Agency estimates the number to be 6.5 million deaths per year.
  4. Research has linked air pollution to multiple diseases: acute lower respiratory infections, chronic obstructive pulmonary disease, lung cancer, tuberculosis, low birth weight, asthma, and cataract.
  5. According to the WHO, 98% of cities in low- and middle-income countries with more than 100,000 habitants have unsafe levels of air pollution.
  6. Of the top twenty most polluted cities in the world, 13 are in India and 3 are in China. Delhi ranks as 11th most polluted, whereas Beijing ranks as 57th most polluted.
  7. Over half of India’s population—660 million people—live in areas with unsafe levels of air pollution.
  8. On average, Indians living in polluted areas will lose 3.2 years of their lives due to air pollution.
  9. In 2014, India and China tied at 155 among 178 nations in rankings measuring how countries are tackling air pollution in the world, despite both countries having some of the worst air quality in the world.
  10. Pregnant women who live in  high traffic areas have a 22% higher risk of having children with impaired lung function than those living in less polluted areas.

Is it possible to lower the cost of clean air without sacrificing effectiveness?

A few months after I published my DIY tests, there were already Taobao stores up and running, selling DIYs even cheaper than my 200 RMB. I was intrigued. If we could really lower the cost of clean air, that’s a win!

But we have to be sure these cheaper machines are as effective. So I ordered the cheaper machines, tested them systematically, and found they performed much worse (2).

A New Way to Lower the Cost of Clean Air

But that doesn’t mean there aren’t ways to make HEPAs even more affordable. Recently I tested one way to do that: if we can shave 1 millimeter from the size of the HEPA, we can save 6% on the price. This new size still covers the fan opening, but does it still work as well?

Method

To test this question, my collaborator Anna ran 10 overnight room tests with the 29 mm in her Beijing apartment and compared it to my earlier tests of the 30 mm HEPA in the same 15m2 room.

e

Like in my earlier tests, I calculated effectiveness as the percent reduction in particulate from the start of the test to the average of the last four hours. Here’s what one of those tests looks like with the reduction calculation laid out. The blue line is indoor 0.5 micron particles; the red line is outdoor PM 2.5

q

Results

On average, the new 29mm HEPA reduced 0.5 micron particle levels in the room by 86% and 2.5 micron levels by 91%. These results were almost identical to my prior results with the 30mm HEPA.

w

Conclusion

This new HEPA lets us lower the cost of clean air without sacrificing any effectiveness. Win! We just lowered the price of Smart Air HEPAs from 80 RMB to 75 RMB.

HEPA Cost Comparison

Next I took that price and compared it to the two biggest brands out there, Blue Air and IQ Air. (Also check out the long-run cost comparison .)

Nerd Note on Replication

As a side note, this test is now the third series of room tests I’ve published with the Original DIY (early tests; 200-day longevity test). Add that to Dr. Saint Cyr’s tests, and I’d say that’s a satisfying amount of reproducibility.

For fellow data nerds, I’m posting the original data and more details about the test below.

1

2

3

4

Outdoor PM 2.5 Fluctuations

I like these long tests more than the common 20-minute tests (such as CADR tests) because this mimics how most people use purifiers. However, one drawback of these long tests is that outdoor PM 2.5 levels sometimes change over the course of 8 hours. If outdoor PM 2.5 drops, that can artificially inflate effectiveness. If outdoor PM 2.5 rises, that can artificially decrease effectiveness.

These changes should cancel out over 10 tests, but it’s worthwhile to re-run the analysis while excluding days with large outdoor changes. Among these 10 tests, two days (9/25 and 11/10) had large outdoor PM 2.5 changes. Without those two days, the averages were almost identical: 84% on 0.5 micron particles and 91% on 2.5 micron particles.

Average Outdoor PM 2.5 During Tests

It’s also worthwhile to check out bad outdoor PM 2.5 was during the tests to see if these days were representative of normal Beijing air. The average outdoor PM 2.5 was 116 micrograms. That’s about 20 micrograms higher than Beijing’s average PM 2.5 over the last five years (according to my analysis of the US Embassy’s PM 2.5 data). Thus, if anything, these tests are tougher than the Beijing average.

5

 

DIYer Makes Cannon Quieter

Bill is an expat living in the hills northwest of Beijing, and he uses his DIY Cannon to cut the particulate in his home.

图片 1

But Bill wanted to improve on the biggest downside of the Cannon–it’s a bit loud on high:

t

So Bill designed a sound dampening box for the Cannon out of foam blocks:

图片 3

There are openings in the front and back to let air through, although my guess is this lowers effectiveness by at least a few percent.

To further cut down on noise, Bill wrapped the Cannon with a blanket:

图片 4

With all that padding, Bill measured a 6 decibel drop in noise! If you have a Cannon already, here’s an idea of what that’s like: That’s 2 decibels quieter than the regular Cannon on low (52 db).

If the noise gets to you and you don’t mind a craft project, I sanction it! However, keep in mind two things:

  1. Monitor the fan temperature. Make sure your design does not allow the fan to overheat.
  2. I suspect this will lower effectiveness by at least a few percentage points. I can’t say more precisely without testing it.

Great work, Bill!

 

Winter (Air) is Coming

Another year, another Beijing winter. As winter arrives to Beijing, I got to wondering: how much worse is winter air?

To get to the bottom of it, I analyzed the last seven years of US Embassy PM 2.5 data. I found that the capital’s air has averaged 111 micrograms in the winter versus 92 micrograms for the rest of the year.

a1

Just how bad is that? The WHO 24-hour PM 2.5 limit is 25 micrograms. That means Beijing’s summers average three times the limit, and winters average over four times limit.

b1

Breathe safe this winter!

 

Shanghai Test: Is indoor air better than outdoor air?

When I got my very first particle counter, I tested sites around Beijing to see whether indoor air was better than outdoor air. It was.

But that test had some limitations. My first particle counter didn’t have a battery, so I had to estimate outdoor particulate in some locations. I also didn’t look into any variables that could give some indoor locations better air than others.

Shanghai Test

Now I’ve got a fancy new Dylos DC1700 battery-power particle counter! Now I can easily take measurements indoors and outdoors. Here’s what it looks like:

d

I took it to Shanghai and tested 14 times in 11 locations on August 27-29. While I was there, the AQI averaged 158 (70 micrograms). I tested mostly around the French Concession, although I also made it out to Fudan University. None of the locations used air purifiers.

sample-locations

I tested in any type of place I could, and I mean any place. Here’s what I recorded in a public bathroom stall:

g

Results

I focused on the smaller 0.5 micron particles (which are highly correlated with the government’s PM 2.5 readings). Across the 14 samples, here’s what I found. The red line represents paticles in the outdoor air.tumblr_inline_nvq1uiANsH1s4lgm1_500

On average, indoor air had just 69% of the particles of outdoor air.

Why do some places have cleaner air?

This fits with my findings in Beijing–indoor air has less particulate than outdoor air. But next, I wanted to see if I could figure out why some places had cleaner air than others. For example, my unpurified apartment air (unpurified because I just got home) had just 30% of particles compared to outdoors, but the public bathroom had 134% of particles compared to outdoors. Woah!

I tested the simplest explanation possible: Were the windows and doors open? Most particulate pollution comes from outside. And in closed rooms, particles will slowly fall to the ground. Thus, indoor air should be better when the windows are doors are closed.

To test that idea, I looked only at places that had doors or windows open:

Yikes! If the doors or windows are open, I was breathing air that was basically as bad as outdoor air (92%). For example, here was how air compared indoors and outdoors in the public bathroom:tumblr_inline_nvq1x22DWj1s4lgm1_500

But things were much better in places with the doors and windows closed:

tumblr_inline_nvq1ybDXJh1s4lgm1_500

With the doors and windows closed, the air had 57% of the particles of outdoors. For example, here’s the air inside Fudan University’s Economics Institute versus outdoors:tumblr_inline_nvq1ynfTmR1s4lgm1_500

In places where the doors and windows were closed, the air was always better than outside. The one exception was the Yuanyuan Restaurant, at 115%. Those particles could be coming from the kitchen.

So what?

The conclusion here is simple: We’re usually breathing much less particulate indoors than outdoors–even without a purifier.

Clean Lung Tips

  1. Work out indoors if possible. I work out indoors in a gym rather than outdoors when I’m in China.
  2. Sit inside at cafes or bars (unless people are smoking).

Does that mean indoor air is safe?

Indoor air has less particulate, but remember that “less particulate” does not always mean “safe.” Out of all 14 tests, NONE of the numbers was below the WHO 24-hour PM 2.5 limit:tumblr_inline_nvq21vZkA11s4lgm1_500

One Last Exception

Finally, remember that pollutants can sometimes come from indoors. This is usually from the paints and chemicals used in remodeling and new furniture. If your home smells like paint or new furniture, you may be safer opening the windows (or at least using activated carbon).