Xiaomi Auto Mode Leaves Air Unsafe for 86% of hours

For the last three years, I’ve been saying clean air doesn’t need to cost an arm and a leg. So when Xiaomi came out with a purifier for under 1,000 RMB, I assumed it works just fine. But I had never gotten around to testing it—until now. And what I found shocked me.

The Test

Smart Air co-founder Anna ran a brand new Mi2 in the same 15m2 bedroom in Chaoyangmen where we’ve tested the DIYs, IQ Air, Blue Air, and Philips. Anna ran six tests on auto mode and six tests on the highest setting (which the Xiaomi oddly calls 最爱档 -“most love mode” or “Favourites mode”).

Minion

Anna turned the purifier on in the morning when she left home, and turned it off when she returned home, so there was no one in the home during the tests. The doors and windows were closed during the tests. (More details on the test method and all original data are here.)

A Dylos Pro tracked ≥ 0.5 micron and ≥ 2.5 micron particle counts during the test. Test data shows these 0.5 micron readings correlate highly with the US Embassy’s PM2.5 readings  (r = .90).

As in my earlier tests, I calculated effectiveness as (the number of particles before turning the purifier on) versus (the average number of particles over the last four hours).

Results

The Xiaomi scored as one of the worst purifiers I’ve ever tested. On average, it removed only about 60% of 0.5 micron particles over the last 4 hours of the test.

Xiaomi effectiveness

Here’s how the Xiaomi results compare to earlier results from other purifiers using the same method, same particle counter, in the same room.

Average % of particulates removed

Below is a normal test day. I put a dashed red line representing the World Health Organization’s (WHO) 24-hour PM2.5 limit (25 micrograms).

Xiaomi room test

The Xiaomi Really Wants To Be in Auto Mode

What’s wrong? The key is that—even on the highest setting—the Xiaomi reverts back to auto mode after 3 hours. To be sure this is what the machine was doing, we put the Xiaomi on the highest setting and tracked noise levels overnight.

Room

You can see the noise remaining high for three hours, then it returns to the ups and downs of auto mode:

Xiaomi noise

Here is the same graph with particle counts (measured by the Dylos). The particle counts rise every time the machine turns off.

Xiaomi noise vs PM 2.5

The data shows that the Xiaomi has a rather low standard for “safe.” The Xiaomi is turning off when PM2.5 reaches 40 micrograms, which is far higher than even the WHO’s 24 hour limit. Then it turns on again when PM2.5 gets up to 70 micrograms—almost three times the limit.

Really? It can’t be kept on high?

The fact that the Xiaomi can’t be kept on continuously all night is so strange that Anna asked Xiaomi’s customer service about it twice to make sure we’re not making a mistake. They confirmed that, no matter what, the machine will go back to auto mode after three hours (full transcript available in the supplemental materials).

Blurred

How often is the air unsafe?

I calculated the percent of hours that the air was unsafe during the tests using this rule:

After the purifier was on for at least 1 hour, for any hour where outdoor air pollution was unsafe (> 25 micrograms – the WHO 24 hour limit), how many hours was indoor air also unsafe (>25 micrograms)?

The Xiaomi left air unsafe for a shocking 86% of the time. The other similarly sized machines in my earlier tests left air unsafe only 7-16% of the time.

% of unsafe hours

Now perhaps the Xiaomi is using the looser Chinese standard for clean air of 35 micrograms. Even with that high number, 77% of the hours the Xiaomi was running, pollution levels were still above the limit.

Wait, are you sure sure?

I wanted to be really careful about this. I’ve seen other tests showing that the Xiaomi 1 works just fine, including these tests from Dr. Saint Cyr (although I’ve seen people write about flaws too, such as this guy who kept the filters wrapped in plastic, turned it on turbo mode, and the app told him his air got miraculously cleaner). So I double and tripled checked the data:

  1. We tested with a different particle counter in a different room.
  1. I re-analyzed the data throwing out any days with large fluctuations in outdoor air.
  1. I analyzed days when outdoor pollution was low to average (< 150 micrograms).
  1. I compared it to other tests done just two weeks before in the exact same room with the exact same particle counter.

None of these analyses changed the result (see details here).  Note however, that we did only test one Xiaomi Mi2, so there is a possibility another machine would perform differently. I invite anyone with a Mi2 to replicate my tests using the same method, and I’m happy to publish the results.

What now for Xiaomi?

From this data, my conclusion is that the fan and the filter are fine, but the Xiaomi has a programming flaw. Even if I use it on the highest setting, I’ll be breathing air far above the safe limit for most of the night. That’s a problem.

Is it just a Xiaomi problem?

To be fair, I cannot recommend any auto mode I’ve tested. I’ve tested the Philips AC4072 on auto mode, and it averaged 59% reductions in 0.5 micron particles, which is pretty close to the Xiaomi results. So I think this is more of a problem with auto modes rather than the Xiaomi specifically.

Yet I can still recommend the Philips. Why? It can stay in medium or high as long as you want, and my data shows it works fine all night as long as it’s not on auto mode.

I’m confident that Xiaomi can fix this flaw by simply allowing people to run it on high without reverting back to auto mode. But until that happens, I cannot recommend using the Mi2.

Open Data

As always, I’m publishing more details on the test method and the original data for fellow nerds.

How Accurate is the 1 Micron Dylos?  Official PM2.5 Comparison Test

dylos-1-micron-guangzhou-cover
Everyone who buys an air quality monitor wants to know how it compares to the official numbers. When I bought a Dylos DC1100 Pro laser particle counter,  I did a comparison test with the US Embassy. I put my Dylos Pro outside the window of my apartment (at Nanluoguxiang) 70 times and compared the numbers to the US Embassy’s Twitter feed at the same time. I found that the two numbers were very strongly related: r = 0.89 (remember the highest possible correlation is 1).

dylos-pro-vs-us-embassy-en
I did that test with the Pro model, but there’s also a standard model that’s $70 cheaper. At Smart Air, we call it the Standard Dylos.

How does this cheaper machine compare to the government measurements? If it’s similarly accurate, great! That means people can save money and get roughly similar results.

dylos-pro-vs-standard-en

Pro vs Standard

The Dylos Pro and Standard are almost the same judging from their front looks. So what’s the real difference?

The key is the size of particles they measure. Both of them have 2 size ranges. For the small particle channel, the Dylos Pro measures particles .5 microns and above, while the Standard Dylos measures 1 micron and above. On the large particle channel, the Pro measures 2.5 microns and above, while the Standard measures 10 microns and above. That means the Pro measures an extra range from .5 to 1 micron.

dylos-pro-vs-standard-measurement-range-en

The machine governments use is called the Met One BAM. It measures particles 2.5 microns and below. If we put the measurement ranges of the BAM, the Dylos Pro, and the Standard Dylos together, here’s what it looks like:

dylos-pro-vs-embassy-vs-dylos-en

The Pro overlaps a bit more the government machine than the Standard. For that reason, I’d expect the Pro correlates better with the government machine. But just how well do they compare empirically?

The Test

Luckily I’m not the only nerd interested in air pollution problem. An American living in Guangzhou bought the Standard and teamed up with me to test it against the US Consulate.

From November 12th to 17th, she collected data with her Dylos Standard 70 times outside her apartment,which is close to Sun Yat-Sen University in Guangzhou. She also recorded the numbers of PM2.5 concentration released by the US Consulate.

dylos-1-micron-test-map

Results

Remember how well the Pro correlated with the US Embassy? Here’s what I found before (r =.89).

dylos-pro-vs-us-embassy-en

Later I did another test by putting my Dylos Pro about 100 meters away from the US Embassy (much closer than my apartment). This time it turned out that the two numbers correlated at r = .9 – slightly higher than in the first test.

dylos-pro-vs-us-embassy-closer-en

How does the Standard Dylos compare? Here’s what we found:

standard-1-micron-dylos-vs-us-embassy-en

They correlate at r = .85! Although that’s a bit lower than the correlation between Dylos Pro and US Embassy, the two numbers are still strongly related. I think that result would surprise most people.

Converting the Dylos 1 Micron Readings to PM 2.5

Using this data, we can get a rough formula to convert between the Dylos 1 micron and the official numbers. For example, the 24-hour WHO limit is 25 µg/m³, which is approximately 1,250 on the Standard Dylos.

1-micron-dylos-to-us-embassy-en

The simplest formula to convert the Dylos 1 micron to PM 2.5 micrograms is:

Dylos 1 micron count × 0.02 = PM 2.5 µg/m3

If we use that formula to convert the Dylos numbers to micrograms, here’s how well the numbers compare:

1-micron-dylos-formula-en

On average, the formula was off from the government numbers by 9.04 micrograms.

Conclusion

The Standard correlates highly with the government numbers—only slightly less well than the Pro model.

Limitations

The conclusion is based on only one test in Guangzhou. We could draw stronger conclusions if we had more tests done in different locations and in different seasons.

Open Data

As always, I’m making the data and more testing details publicly available for fellow nerds.

P.S. Thanks to Yiwen Wang for help translating and creating figures.

How accurate are common particle counters? Comparison test

When I started Smart Air in 2013, I wanted to buy a particle counter, and I had basically two options. I could buy a US$260 Dylos or spend thousands of dollars on the crazy expensive particle counters.

Since then, the market has exploded with new particle counters as cheap as 99 RMB. But are they any good?

Putting Particle Counters to the Test

To get to the bottom of it, we tested three popular particle counters on the market. We tested the Dylos DC1700, the Origins Laser Egg, and a new particle counter called the AirVisual Node.

 

Dylos

Dylos DC1100
Dylos DC1100

 

The Dylos is the trusty particle counter Smart Air has been using since 2013.

 

Laser Egg

Origins Laser Egg
Origins Laser Egg

 

The Laser Egg is the popular, more technologically savvy device.

Node

 

Air Visual Node
Air Visual Node

And the Node is a fancier version, including a large screen, richer information, pollution forecasts, better user experience and even a CO2 monitor.

 

allthree

The Government Comparison

We placed the machines outside the Smart Air office on Dongzhimen Waidajie, about 1.3km away from the government PM2.5 monitor at the Agricultural Exhibition Center.

 

Map of Smart Air Office and Agricultural Exhibition Centre

 

 

We ran the machines for six days. The Laser Egg and the Node give output in PM2.5 micrograms. The Dylos gives number of 0.5 micron particles, so we converted it to PM2.5 micrograms using the semi-official formula (0.5 microns – 2.5 microns)/100.

Here are the results for the first (72-hour) test outside our office in Beijing:

 

out-en-official-pm2-5-ag-exhib

 

Low-Concentration Tests

 

Next we tested on days with extraordinarily low PM2.5. That’s helpful because concentrations in homes—where most people use particle counters—are also typically low. So this data is good for testing how good the devices are at low concentration levels. We ran tests for 48 hours whilst the skies were clear.

Low concentration tests
Low concentration tests

 

Eyeballing both graphs, all three machines did a pretty good job of tracking the official numbers. Combining both tests, we found that both the AirVisual Node and the Laser Egg correlated r = 0.98 with the official PM2.5 numbers. For non-nerds, 0.98 is incredibly close to identical! The Dylos had the lowest correlation at r = 0.90, but still incredibly high (and similar to our previous test). These correlations are all extremely high and suggest that these particles counters are tracking government data well.

 

Average Deviation

 

Another way to measure accuracy is to look at on average how far the numbers were from the government data. The Node was the closest: it was off from the official numbers by an average of 4.8µg/m3. The Laser Egg was consistently further than the government machine, with an average deviation of 6.5µg/m3. The Dylos was off by an average of 9.1µg/m3.

 

Low concentration PM2.5 deviation from Agricultural Center

 

Perhaps one worry to note is that the Laser Egg was consistently under-estimating PM2.5 while air pollution was in the lower range. This means there could be a risk that the Laser Egg underestimates the real pollution levels in the home, giving a false sense of security. However, even these deviations were not large.

 

picture2 picture1 picture3

 

The Airpocalypse Test

To test accuracy at extremely high concentrations, we burned a cigarette in a closed 15m3 room. Our goal here was to see how well the particle counters were at reading concentration levels over a whole range of values, including toxic levels. With the help of cigarettes and a partner NGO in Beijing, we managed to get the concentration above 1,000µg/m3!

Airpocalypse test setup

For this test we also has another machine (Sibata LD-6S) on hand as a reference. This is an industrial PM2.5 dust indicator, with an accuracy of ±10% and repeatability error of ±2%. Thus, we used the LD-6S as our baseline.

 

Airpocalypse test results
Airpocalypse test results

 

Looking at the data, it’s immediately clear that the Laser Egg and the Dylos had a hard time keeping up with these really high levels of concentration. In contrast, the Node and the LD-6S matched very closely, and were able to measure values over 1,000µg/m3. The chances of you needing to measure these values outside of experiments are very slim, but it shows that the Node is more accurate at these high levels.

 

Take-Home Message

 

Overall, the three particle counters were reasonably accurate compared to the government machines. In the estimation of the Smart Air team, all of them are suitable for giving an approximate AQI value in your home. Of all three, the Node scored the highest, with the lowest deviation from the government machines in both outdoor tests and the highest accuracy in the “crazy bad” test.

 

Usability

 

Since all three machines are reasonably accurate, the question then really comes down to: How easy it is to use the device? And what features do they have?

 

The Dylos (1800 RMB)

The Dylos easily loses this fight. It has no phone connectivity, and downloading the data is a terrible pain—and that’s if you have one of the old school pin connecter cables.

 

The Laser Egg (499RMB)

The Laser Egg is an entry-point particle counter. It gives reasonably accurate results with a simple interface. It’s not feature rich, but it does what it says on the box.

 

The AirVisual Node (988RMB)

 To our eyes, the Node offers the best features. For starters, it can measure CO2, temperature, and humidity. That makes it more of an ‘environment monitor’ than just a particle monitor. CO2 is important if you have lots of people in a small space as it can give an indication of how confined the space is. If you have indoor sources of air pollution (VOCs) like new furniture or remodelling, high CO2 levels can mean that those indoor pollutants are building up. Its user design shows AQI and CO2 for the past 24 hours both indoors and outdoors, a forecast for the coming days, and suggestions to help you decide when to open your windows and wear a mask. We’ve found these features helpful in our office.


Conclusion

After passing our tests, we will start shipping the AirVisual Node through our Taobao shop  and website. It’s a great option for anybody wanting a solid device for both home use and research (if you’re a nerd like us). Go take a look!

Over the next few months, we hope to get a larger pool of particle counters together and run more extensive tests. This is only the beginning! Once we’ve independently verified more devices, we may well be adding them to our shop as well.

 

Do pollution masks work?

When a billion people in China (and quite a few expats) woke up to the severe air pollution in almost every city in China, it forced a billion people to become experts in a complicated scientific question. Do masks work?

Since then, I’ve given talks with hundreds of people all around China about how to protect themselves from air pollution. In those talks, I’ve heard doubts from smart, skeptical people. Here I’ll answer those doubts because, fortunately, smart, skeptical scientists (plus one dedicated nerd—yours truly) have empirically tested these questions.

Here are the two most frequent skepticisms I hear about masks.

  1. “There’s no way they capture the really small particles”

The skeptic case:

The most dangerous particles are the smallest particles, but masks are so thin. How could they possibly get the smallest particles?

The scientific test:

Researchers from the University of Edinburgh tested different common masks by running a diesel generator (to mimic car exhaust) and piping the exhaust through different masks. They used a particle counter to see how many particles made it through the mask. Here’s my super scientific rendering of the setup:

图片 1

One important detail: the particle counter they used measures down to .007 microns. We’re talking about truly tiny particles here!

First they tried a simple cotton handkerchief. Sometimes I see bikers in China wearing these.

2

Not great, 28% of particles blocked.

Next they tried a cheap surgical mask.

3

Surprisingly good! (Fit tests generally show lower results–see below–but still a lot higher than most people’s intuition.)

Next they tried several bike masks.

4

Most were around 80%.

Then they tried several cheap 3M masks.

5

They all scored over 95%. Pretty good!

Conclusion: masks capture even very small particles.

  1. “OK, they capture the small particles, but when you wear them, all the air just leaks in the side.”

The skeptic case:

Masks work in theory, but those tests aren’t on real faces! When you actually wear them, you can’t get a good enough fit, so they’re basically useless.

The scientific test:

This question is tougher to answer because you have to measure the mask while you’re actually wearing it. For that, you need a really expensive fit test machine. Fortunately, I begged and begged 3M until they let me use their lab in Beijing:

6

The blue tube is sampling air outside the mask, while the white tube is sampling air from inside the mask (more details on the methods here).

Beijing-based Dr. Richard Saint Cyr also tested masks, so I’ll combine my data with his. Here’s how well the masks worked on our faces:

7

How well do masks work for the broader population?

It’s important to make clear: fit test results on my face won’t always be the same for other people’s faces. However, there is evidence from a broader population that masks fit most people well. A scientific study of 3M masks on 22 Chinese people found a median fit score of 99.5%–essentially the same as the top results from Dr. Saint Cyr and me.

Best yet, effective masks don’t cost a lot of money. And you certainly don’t need to buy the most expensive masks on the market to breathe clean air.

8

A note on gases: Note that these tests are about particulate pollution. Most commercially available masks don’t target gas pollutants like NO2 and O3, so it’s not 100% protection.

  1. Is there a documented health benefit of wearing a mask?

This is probably the hardest question to answer. However, there are two solid studies that have randomly assigned people in Beijing to wear masks or not and measured their heart rate and blood pressure (1,  2).

9

While wearing masks, people had lower blood pressure and better-regulated heart rates.

10

Conclusion: Masks capture even the smallest particles—even while you’re wearing them. And they have documented health benefits. That should be enough to satisfy even the skeptics!

Poor Man’s Fit Test

Which mask works best on your face? I was fortunate enough to visit a lab to do a super fancy fit test, but very few of us have access to this $10,000 machine. So what should normal folks do?

图片 1

While visiting the 3M lab, I learned about what I’m calling the poor man’s fit test. It’s not as accurate as a real fit test, but it will help you identify big leaks. It’s pretty simple:

  1. Put on the mask. Make sure the metal is bent tightly around your nose.
  1. If the mask has two straps, make sure one strap is below your ears and one above like this:maxresdefault
  2. Lightly hold the mask in place and inhale sharply. While inhaling, pay attention to see if you feel a sensation of air or coolness around the edge of the mask. Pay particular attention to the area around the nose.
  3. If you feel air leaking, adjust the mask and try again. If further adjustment does not solve the problem, try a different mask.

3m-fitting

If your mask does not have an exhalation valve, you can also do the test while exhaling sharply.

Breathe safe!

Is it possible to lower the cost of clean air without sacrificing effectiveness?

A few months after I published my DIY tests, there were already Taobao stores up and running, selling DIYs even cheaper than my 200 RMB. I was intrigued. If we could really lower the cost of clean air, that’s a win!

But we have to be sure these cheaper machines are as effective. So I ordered the cheaper machines, tested them systematically, and found they performed much worse (2).

A New Way to Lower the Cost of Clean Air

But that doesn’t mean there aren’t ways to make HEPAs even more affordable. Recently I tested one way to do that: if we can shave 1 millimeter from the size of the HEPA, we can save 6% on the price. This new size still covers the fan opening, but does it still work as well?

Method

To test this question, my collaborator Anna ran 10 overnight room tests with the 29 mm in her Beijing apartment and compared it to my earlier tests of the 30 mm HEPA in the same 15m2 room.

e

Like in my earlier tests, I calculated effectiveness as the percent reduction in particulate from the start of the test to the average of the last four hours. Here’s what one of those tests looks like with the reduction calculation laid out. The blue line is indoor 0.5 micron particles; the red line is outdoor PM 2.5

q

Results

On average, the new 29mm HEPA reduced 0.5 micron particle levels in the room by 86% and 2.5 micron levels by 91%. These results were almost identical to my prior results with the 30mm HEPA.

w

Conclusion

This new HEPA lets us lower the cost of clean air without sacrificing any effectiveness. Win! We just lowered the price of Smart Air HEPAs from 80 RMB to 75 RMB.

HEPA Cost Comparison

Next I took that price and compared it to the two biggest brands out there, Blue Air and IQ Air. (Also check out the long-run cost comparison .)

Nerd Note on Replication

As a side note, this test is now the third series of room tests I’ve published with the Original DIY (early tests; 200-day longevity test). Add that to Dr. Saint Cyr’s tests, and I’d say that’s a satisfying amount of reproducibility.

For fellow data nerds, I’m posting the original data and more details about the test below.

1

2

3

4

Outdoor PM 2.5 Fluctuations

I like these long tests more than the common 20-minute tests (such as CADR tests) because this mimics how most people use purifiers. However, one drawback of these long tests is that outdoor PM 2.5 levels sometimes change over the course of 8 hours. If outdoor PM 2.5 drops, that can artificially inflate effectiveness. If outdoor PM 2.5 rises, that can artificially decrease effectiveness.

These changes should cancel out over 10 tests, but it’s worthwhile to re-run the analysis while excluding days with large outdoor changes. Among these 10 tests, two days (9/25 and 11/10) had large outdoor PM 2.5 changes. Without those two days, the averages were almost identical: 84% on 0.5 micron particles and 91% on 2.5 micron particles.

Average Outdoor PM 2.5 During Tests

It’s also worthwhile to check out bad outdoor PM 2.5 was during the tests to see if these days were representative of normal Beijing air. The average outdoor PM 2.5 was 116 micrograms. That’s about 20 micrograms higher than Beijing’s average PM 2.5 over the last five years (according to my analysis of the US Embassy’s PM 2.5 data). Thus, if anything, these tests are tougher than the Beijing average.

5

 

Shanghai Test: Is indoor air better than outdoor air?

When I got my very first particle counter, I tested sites around Beijing to see whether indoor air was better than outdoor air. It was.

But that test had some limitations. My first particle counter didn’t have a battery, so I had to estimate outdoor particulate in some locations. I also didn’t look into any variables that could give some indoor locations better air than others.

Shanghai Test

Now I’ve got a fancy new Dylos DC1700 battery-power particle counter! Now I can easily take measurements indoors and outdoors. Here’s what it looks like:

d

I took it to Shanghai and tested 14 times in 11 locations on August 27-29. While I was there, the AQI averaged 158 (70 micrograms). I tested mostly around the French Concession, although I also made it out to Fudan University. None of the locations used air purifiers.

sample-locations

I tested in any type of place I could, and I mean any place. Here’s what I recorded in a public bathroom stall:

g

Results

I focused on the smaller 0.5 micron particles (which are highly correlated with the government’s PM 2.5 readings). Across the 14 samples, here’s what I found. The red line represents paticles in the outdoor air.tumblr_inline_nvq1uiANsH1s4lgm1_500

On average, indoor air had just 69% of the particles of outdoor air.

Why do some places have cleaner air?

This fits with my findings in Beijing–indoor air has less particulate than outdoor air. But next, I wanted to see if I could figure out why some places had cleaner air than others. For example, my unpurified apartment air (unpurified because I just got home) had just 30% of particles compared to outdoors, but the public bathroom had 134% of particles compared to outdoors. Woah!

I tested the simplest explanation possible: Were the windows and doors open? Most particulate pollution comes from outside. And in closed rooms, particles will slowly fall to the ground. Thus, indoor air should be better when the windows are doors are closed.

To test that idea, I looked only at places that had doors or windows open:

Yikes! If the doors or windows are open, I was breathing air that was basically as bad as outdoor air (92%). For example, here was how air compared indoors and outdoors in the public bathroom:tumblr_inline_nvq1x22DWj1s4lgm1_500

But things were much better in places with the doors and windows closed:

tumblr_inline_nvq1ybDXJh1s4lgm1_500

With the doors and windows closed, the air had 57% of the particles of outdoors. For example, here’s the air inside Fudan University’s Economics Institute versus outdoors:tumblr_inline_nvq1ynfTmR1s4lgm1_500

In places where the doors and windows were closed, the air was always better than outside. The one exception was the Yuanyuan Restaurant, at 115%. Those particles could be coming from the kitchen.

So what?

The conclusion here is simple: We’re usually breathing much less particulate indoors than outdoors–even without a purifier.

Clean Lung Tips

  1. Work out indoors if possible. I work out indoors in a gym rather than outdoors when I’m in China.
  2. Sit inside at cafes or bars (unless people are smoking).

Does that mean indoor air is safe?

Indoor air has less particulate, but remember that “less particulate” does not always mean “safe.” Out of all 14 tests, NONE of the numbers was below the WHO 24-hour PM 2.5 limit:tumblr_inline_nvq21vZkA11s4lgm1_500

One Last Exception

Finally, remember that pollutants can sometimes come from indoors. This is usually from the paints and chemicals used in remodeling and new furniture. If your home smells like paint or new furniture, you may be safer opening the windows (or at least using activated carbon).

Can Activated Carbon Remove Formaldehyde?

“Formaldehyde” (甲醛) is a surprisingly common word in China. I say surprising because I’m from the US, where only scientists and high schoolers dissecting frogs tend to be familiar with the chemical.

But it’s not because Chinese people are all nerds. At dinner with Chinese colleagues, my friend David once used the word “carbohydrate” (碳水化合物) and was instantly made fun of for using a “science word” in casual conversation. Why the double standard? Because in China, just like carbs in America, formaldehyde is an everyday health concern.

What’s the big deal?

Why are they so concerned? Formaldehyde causes scary health problems like ulcers and cancer, yet it’s common in construction materials and new furniture. The people who should worry the most are people in new or recently renovated homes. Tests of new and renovated homes routinely find high levels of formaldehyde.

Why formaldehyde is so hard to remove

Formaldehyde is a tricky problem because it escapes from materials as a gas, and HEPA filters aren’t made to capture gases. I’ve heard companies claim that activated carbon can remove formaldehyde, but I’ve also seen companies selling sprays that claim to clear formaldehyde from your home. That sounds a lot like snake oil to me.

Since there’s a profit motivation for companies to claim they can get rid of formaldehyde, I wanted to empirically test whether carbon actually works.

Tests

In my quest for an answer, the first stumbling block was detecting formaldehyde. Many companies on Taobao will sell you a “formaldehyde detector”, but they’re actually general VOC detectors. (There are lots of volatile organic compounds; formaldehyde is one type.) So even if the machine tells you it’s detecting formaldehyde, you have no way of knowing whether it’s formaldehyde or some other VOC.

Thus, to be absolutely sure we were detecting formaldehyde, my collaborator Anna bought bottles of liquid formaldehyde–risking our health for science!

2

To spread it in the room, we put it in a rice cooker along with 250 ml of water in a 4.14m2 porch (volume 10.35m3). When the rice cooker heats up, it releases formaldehyde as a gas into the air.

n

The Industrial Scientific MX6 detects different types of VOCs, not formaldehyde specifically. But because we released formaldehyde in the room, we can be sure that VOC is formaldehyde.

3

To attack the formaldehyde, Anna put a composite activated carbon filter on the Cannon.

c

Anna turned the cooker and the fan on at the same time and let them run until the VOC level fell back to zero. We also ran a control test with a fan only. We ran a total of three carbon tests and two fan-only tests.

Results

Here’s what one of the carbon tests looked like, starting from the peak formaldehyde level:

v

But we need to be sure that’s the effect of the carbon, not just the formaldehyde dispersing over time. To do that, we need to compare those results to the fan-only condition. Here’s what the two tests look like side by side:

b

The formaldehyde levels stayed higher for longer in the fan-only condition, but the levels dropped much quicker when we used carbon.

I averaged across all three carbon tests and compared the average reduction compared to the fan-only condition. On average, the carbon reduced formaldehyde levels to 50% within 15 minutes of the peak formaldehyde levels and then down 0% by 25 minutes.

a

Conclusion

These composite activated carbon filters removed formaldehyde from the air. My earlier tests show that these filters remove other types of VOCs too.

Does everyone need carbon?

Studies have found that formaldehyde is much more common in new and recently remodeled homes, so people in new or remodeled homes probably need carbon. However, my MX6 found zero VOCs in ALL apartments I tested except for places that had recent renovation or smoking.

Now, there could be VOCs at levels lower than the MX6 can detect, so I’m not confident to say there are ZERO VOCs in most houses. But I think it’s reasonable to say that carbon filters are not mandatory for homes without obvious sources of formaldehyde or other VOCs.

Do I still need a HEPA?

Activated carbon is made to get smells and gases like VOCs. It is not designed to get particulate in general. In fact, activated carbon is made to be as porous as possible to get as much air into contact with the carbon. And my tests with a carbon filter alone show that it is does not remove high amounts of small particles. Thus, we still need a HEPA.

As always, I’m including the raw data and more details on the testing for fellow nerds.

more

Data

The raw data is a large file, so I’m making it available as a download. Here is the summary data:

s

Methods

Check out my earlier VOC tests for more details on the MX6 detector, placement of the detector, and the fan-only control condition. Conditions were identical except for the pollution source.

How do we know the detector was actually detecting formaldehyde?

I wanted to test whether the MX6 was detecting the formaldehyde and not the heat and humidity coming from the electric cooker. To test that, I also ran a condition where I filled the rice cooker with water, but no formaldehyde. In that condition, the MX6 read zero:

d

That tells us that the MX6 wasn’t mistaking heat or water for formaldehyde.

Limitations

Similar to my earlier VOC tests, one limitation is that the formaldehyde here was not naturally occurring. It would be ideal to find a house that was recently remodeled and already has formaldehyde in it because that would be a closer approximation of how most people would use it.

However, one difficulty of that sort of test is that the VOC detector wouldn’t be able to tell us if we’re detecting formaldehyde or other VOCs. In one sense, that’s not important–we want to get rid of all of them. But it would also be interesting to know if we’re getting formaldehyde specifically. To do that, we would need to take gas samples and have them sent to a lab (although if anyone knows of an easier way to detect formaldehyde and only formaldehyde, let me know).

Are DIY purifiers still cheap in the long run?

When I published tests of the DIY compared to the expensive machines, a couple people on Zhihu (China’s Quora) asked if you really save money in the long run with the DIY once you calculate the cost of changing the HEPA. After all, the IQ Air HEPA costs 1,782 RMB, but you can use it for more than a year.

Long-Term DIY HEPA Costs

At that time I didn’t have an answer. To get an honest answer, I needed to do tests in the real world, but that took almost a year to complete. (That’s more than I can say for the numbers IQ Air and Blue Air give. If filters last 6 months in Sweden, are they going to last 6 months in Beijing?)

Now it’s a year later, and I have that data. Smart Air co-founder Gus ran his Original DIY for eight hours a day and tracked what percentage of particulate it removed from the air each day with a Dylos particle counter.

1

Based on that data, I recommend changing HEPAs after 140 days at 8 hours per day (about 1,000 hours of use). Since that HEPA costs 80 RMB, that averages to .57 RMB per day and 208.6 RMB per year.

To get an idea of what that means, we can compare that to the cost of drinking a bottle of water a day:

2

Big Brand HEPA Costs

I’m highly skeptical that HEPA recommendations for Sweden can be mapped onto China. But to be conservative, I’ll assume in my calculations that their numbers work the same in China.

I calculated costs for the Blue Air 203 and IQ Air Health Pro Plus using the same conditions I used for the Original DIY longevity test:

  1. Highest setting
  2. 8 hours of use per day
  3. Extend those costs over a year
  4. Include the cost of pre-filters but not carbon filters

Blue Air

Blue Air’s HEPA costs 300 RMB and lasts 6 months. That works out to 1.67 RMB per day and 609.6 RMB per year.

3

IQ Air

IQ Air’s HEPA is more expensive: 1,782 RMB. On the sixth setting, it lasts 4,968 hours. If you use it 8 hours a day, that works out to 2.87 RMB per day and 1,047.4 RMB per year.

However, we have to add the cost of the pre-filter because the HEPA lifespan depends on the pre-filter. The pre-filter costs 645 RMB and lasts 2,016 hours on the sixth setting. That works out to 2.56 RMB per day and 934.3 RMB per year.

IQ Air grand total: 1,982 RMB per year.

4

Conclusion

The long-term HEPA cost for the Original DIY is 66-89% less than the Blue Air and IQ Air.

It should be noted that these results may not map onto the Cannon. I’m still working on the Cannon longevity test, so we’ll need to wait until that test is done.

As always, I’m writing more details about the data below for fellow nerds.

DIY HEPA Costs

DIY HEPA costs are based on the 200-day longevity test in real Beijing air. All 200 days of original data are available in that post.

IQ Air Costs

The IQ Air cost calculations are based on a table from the North American IQ Air authorized retailer. That data is available here. I welcome other nerds to double check my calculations!

One thing to note: the calculations above are based on the sixth setting. If the IQ Air runs on setting three, the effectiveness will be lower, but the HEPA will last longer. In my calculations, I totaled the costs for one year one setting three: 901 RMB. That’s lower than the calculation above (because it’s processing less air), but it’s still more than four times the cost of the DIY HEPA.

Blue Air Costs

Calculating costs for the Blue Air is more difficult. That’s because Blue Air doesn’t seem to publish longevity recommendations for different settings or how many hours you can use the HEPA. Instead, Blue Air says you should replace the HEPA after six months.

Thus, I sent Blue Air an email asking how many days I should replace the HEPA if I run my machine eight hours a day. A Blue Air representative told me six months and told me I would void the warranty if I used it more than six months. Thus, I used their recommendation in my calculations (original email).

If Blue Air is wrong and the HEPA lasts long at eight hours per day, the calculation changes. I have alternative calculations here.