How accurate are common particle counters? Comparison test

When I started Smart Air in 2013, I wanted to buy a particle counter, and I had basically two options. I could buy a US$260 Dylos or spend thousands of dollars on the crazy expensive particle counters.

Since then, the market has exploded with new particle counters as cheap as 99 RMB. But are they any good?

Putting Particle Counters to the Test

To get to the bottom of it, we tested three popular particle counters on the market. We tested the Dylos DC1700, the Origins Laser Egg, and a new particle counter called the AirVisual Node.

 

Dylos

Dylos DC1100
Dylos DC1100

 

The Dylos is the trusty particle counter Smart Air has been using since 2013.

 

Laser Egg

Origins Laser Egg
Origins Laser Egg

 

The Laser Egg is the popular, more technologically savvy device.

Node

 

Air Visual Node
Air Visual Node

And the Node is a fancier version, including a large screen, richer information, pollution forecasts, better user experience and even a CO2 monitor.

 

allthree

The Government Comparison

We placed the machines outside the Smart Air office on Dongzhimen Waidajie, about 1.3km away from the government PM2.5 monitor at the Agricultural Exhibition Center.

 

Map of Smart Air Office and Agricultural Exhibition Centre

 

 

We ran the machines for six days. The Laser Egg and the Node give output in PM2.5 micrograms. The Dylos gives number of 0.5 micron particles, so we converted it to PM2.5 micrograms using the semi-official formula (0.5 microns – 2.5 microns)/100.

Here are the results for the first (72-hour) test outside our office in Beijing:

 

out-en-official-pm2-5-ag-exhib

 

Low-Concentration Tests

 

Next we tested on days with extraordinarily low PM2.5. That’s helpful because concentrations in homes—where most people use particle counters—are also typically low. So this data is good for testing how good the devices are at low concentration levels. We ran tests for 48 hours whilst the skies were clear.

Low concentration tests
Low concentration tests

 

Eyeballing both graphs, all three machines did a pretty good job of tracking the official numbers. Combining both tests, we found that both the AirVisual Node and the Laser Egg correlated r = 0.98 with the official PM2.5 numbers. For non-nerds, 0.98 is incredibly close to identical! The Dylos had the lowest correlation at r = 0.90, but still incredibly high (and similar to our previous test). These correlations are all extremely high and suggest that these particles counters are tracking government data well.

 

Average Deviation

 

Another way to measure accuracy is to look at on average how far the numbers were from the government data. The Node was the closest: it was off from the official numbers by an average of 4.8µg/m3. The Laser Egg was consistently further than the government machine, with an average deviation of 6.5µg/m3. The Dylos was off by an average of 9.1µg/m3.

 

Low concentration PM2.5 deviation from Agricultural Center

 

Perhaps one worry to note is that the Laser Egg was consistently under-estimating PM2.5 while air pollution was in the lower range. This means there could be a risk that the Laser Egg underestimates the real pollution levels in the home, giving a false sense of security. However, even these deviations were not large.

 

picture2 picture1 picture3

 

The Airpocalypse Test

To test accuracy at extremely high concentrations, we burned a cigarette in a closed 15m3 room. Our goal here was to see how well the particle counters were at reading concentration levels over a whole range of values, including toxic levels. With the help of cigarettes and a partner NGO in Beijing, we managed to get the concentration above 1,000µg/m3!

Airpocalypse test setup

For this test we also has another machine (Sibata LD-6S) on hand as a reference. This is an industrial PM2.5 dust indicator, with an accuracy of ±10% and repeatability error of ±2%. Thus, we used the LD-6S as our baseline.

 

Airpocalypse test results
Airpocalypse test results

 

Looking at the data, it’s immediately clear that the Laser Egg and the Dylos had a hard time keeping up with these really high levels of concentration. In contrast, the Node and the LD-6S matched very closely, and were able to measure values over 1,000µg/m3. The chances of you needing to measure these values outside of experiments are very slim, but it shows that the Node is more accurate at these high levels.

 

Take-Home Message

 

Overall, the three particle counters were reasonably accurate compared to the government machines. In the estimation of the Smart Air team, all of them are suitable for giving an approximate AQI value in your home. Of all three, the Node scored the highest, with the lowest deviation from the government machines in both outdoor tests and the highest accuracy in the “crazy bad” test.

 

Usability

 

Since all three machines are reasonably accurate, the question then really comes down to: How easy it is to use the device? And what features do they have?

 

The Dylos (1800 RMB)

The Dylos easily loses this fight. It has no phone connectivity, and downloading the data is a terrible pain—and that’s if you have one of the old school pin connecter cables.

 

The Laser Egg (499RMB)

The Laser Egg is an entry-point particle counter. It gives reasonably accurate results with a simple interface. It’s not feature rich, but it does what it says on the box.

 

The AirVisual Node (988RMB)

 To our eyes, the Node offers the best features. For starters, it can measure CO2, temperature, and humidity. That makes it more of an ‘environment monitor’ than just a particle monitor. CO2 is important if you have lots of people in a small space as it can give an indication of how confined the space is. If you have indoor sources of air pollution (VOCs) like new furniture or remodelling, high CO2 levels can mean that those indoor pollutants are building up. Its user design shows AQI and CO2 for the past 24 hours both indoors and outdoors, a forecast for the coming days, and suggestions to help you decide when to open your windows and wear a mask. We’ve found these features helpful in our office.


Conclusion

After passing our tests, we will start shipping the AirVisual Node through our Taobao shop  and website. It’s a great option for anybody wanting a solid device for both home use and research (if you’re a nerd like us). Go take a look!

Over the next few months, we hope to get a larger pool of particle counters together and run more extensive tests. This is only the beginning! Once we’ve independently verified more devices, we may well be adding them to our shop as well.

 

Source Pollution

10 Facts About Air Pollution

We get many questions about air pollution in our office, and understandably so. It’s a topic that isn’t well understood or well-reported about in certain parts of the countries in which we work. In some cases, it is difficult to distinguish research-backed findings from common beliefs. To contribute to collective learning, here is a quick list of top 10 facts about air pollution.

  1. Air pollution is made up of chemicals, particulates, and biological materials. Common  components include, but are not limited to: nitrogen, sulfur, carbon monoxide, carbon dioxide, dust, and ash.
  2. Air pollution is caused by both human and natural contributors. Industries, factories, vehicles, mining, agriculture, forest fires, volcanic eruptions, and wind erosion all cause air pollution.
  3. According to the Global Burden of Disease report (2013), air pollution contributes to more than 5.5 million premature deaths every year. Another report by the International Energy Agency estimates the number to be 6.5 million deaths per year.
  4. Research has linked air pollution to multiple diseases: acute lower respiratory infections, chronic obstructive pulmonary disease, lung cancer, tuberculosis, low birth weight, asthma, and cataract.
  5. According to the WHO, 98% of cities in low- and middle-income countries with more than 100,000 habitants have unsafe levels of air pollution.
  6. Of the top twenty most polluted cities in the world, 13 are in India and 3 are in China. Delhi ranks as 11th most polluted, whereas Beijing ranks as 57th most polluted.
  7. Over half of India’s population—660 million people—live in areas with unsafe levels of air pollution.
  8. On average, Indians living in polluted areas will lose 3.2 years of their lives due to air pollution.
  9. In 2014, India and China tied at 155 among 178 nations in rankings measuring how countries are tackling air pollution in the world, despite both countries having some of the worst air quality in the world.
  10. Pregnant women who live in  high traffic areas have a 22% higher risk of having children with impaired lung function than those living in less polluted areas.
cost-effective

Is it possible to lower the cost of clean air without sacrificing effectiveness?

A few months after I published my DIY tests, there were already Taobao stores up and running, selling DIYs even cheaper than my 200 RMB. I was intrigued. If we could really lower the cost of clean air, that’s a win!

But we have to be sure these cheaper machines are as effective. So I ordered the cheaper machines, tested them systematically, and found they performed much worse (2).

A New Way to Lower the Cost of Clean Air

But that doesn’t mean there aren’t ways to make HEPAs even more affordable. Recently I tested one way to do that: if we can shave 1 millimeter from the size of the HEPA, we can save 6% on the price. This new size still covers the fan opening, but does it still work as well?

Method

To test this question, my collaborator Anna ran 10 overnight room tests with the 29 mm in her Beijing apartment and compared it to my earlier tests of the 30 mm HEPA in the same 15m2 room.

e

Like in my earlier tests, I calculated effectiveness as the percent reduction in particulate from the start of the test to the average of the last four hours. Here’s what one of those tests looks like with the reduction calculation laid out. The blue line is indoor 0.5 micron particles; the red line is outdoor PM 2.5

q

Results

On average, the new 29mm HEPA reduced 0.5 micron particle levels in the room by 86% and 2.5 micron levels by 91%. These results were almost identical to my prior results with the 30mm HEPA.

w

Conclusion

This new HEPA lets us lower the cost of clean air without sacrificing any effectiveness. Win! We just lowered the price of Smart Air HEPAs from 80 RMB to 75 RMB.

HEPA Cost Comparison

Next I took that price and compared it to the two biggest brands out there, Blue Air and IQ Air. (Also check out the long-run cost comparison .)

Nerd Note on Replication

As a side note, this test is now the third series of room tests I’ve published with the Original DIY (early tests; 200-day longevity test). Add that to Dr. Saint Cyr’s tests, and I’d say that’s a satisfying amount of reproducibility.

For fellow data nerds, I’m posting the original data and more details about the test below.

1

2

3

4

Outdoor PM 2.5 Fluctuations

I like these long tests more than the common 20-minute tests (such as CADR tests) because this mimics how most people use purifiers. However, one drawback of these long tests is that outdoor PM 2.5 levels sometimes change over the course of 8 hours. If outdoor PM 2.5 drops, that can artificially inflate effectiveness. If outdoor PM 2.5 rises, that can artificially decrease effectiveness.

These changes should cancel out over 10 tests, but it’s worthwhile to re-run the analysis while excluding days with large outdoor changes. Among these 10 tests, two days (9/25 and 11/10) had large outdoor PM 2.5 changes. Without those two days, the averages were almost identical: 84% on 0.5 micron particles and 91% on 2.5 micron particles.

Average Outdoor PM 2.5 During Tests

It’s also worthwhile to check out bad outdoor PM 2.5 was during the tests to see if these days were representative of normal Beijing air. The average outdoor PM 2.5 was 116 micrograms. That’s about 20 micrograms higher than Beijing’s average PM 2.5 over the last five years (according to my analysis of the US Embassy’s PM 2.5 data). Thus, if anything, these tests are tougher than the Beijing average.

5

 

净化器需要全天开着吗

Test: Do you have to run your purifier all day?

Some purifier companies recommend running your purifier all day. But is that really necessary? If so, isn’t that a big waste of your filters and energy?

Method

To get to the bottom of it, I set a Smart Air Cannon on a timer to turn on for two hours everyday in a 13.5m2 Beijing bedroom:

1

I put a particle counter in the room to take measurements every minute. I did the test while I was on vacation, so there was no influence of me opening and closing doors.

Results

After six days, I came back and saw how long it took to the Cannon to clean the air each time it came on. Here’s what six days of data looked like for the small .5 micron particles:

5

Over the six test days, the air in Beijing became progressively worse. But on each day, it was clear when the Cannon turned on and off. The dropoffs were sharp, showing the Cannon was working quickly.

I averaged over the six test days to find out how long it took the Cannon to clean the air on average.

6

Conclusion

On average, the Cannon cut .5 micron particulates in half in 10 minutes. By 20 minutes, it removed 80%.

Bottom line: Powerful purifiers like the Cannon clean the air very quickly, so I see no need to run the purifier while I’m not at home.

Can I turn it off while I sleep?

The data can also answer another question a few people have asked me: “I don’t want to hear the fan while I sleep, so can I run it for an hour and then turn it off while I sleep?

In the tests, the air got dirty very quickly after the Cannon turned off (even though the windows and doors were closed). Dirty air is entering our homes constantly, even though we can’t see it.

Bottom line: I do NOT recommend turning the purifier off while you sleep.

Open Data

As always, I’m publishing the raw data and more details on the methods below. I’ll also be publishing data from similar tests in a much larger room (30.5m2) and for tests when I’m at home and moving in the room.

4

(more…)

5-9-1024x580

How Long do Smart Air HEPAs last? Longevity Test Day 130

How long do Smart Air HEPAs last? My quest to give a solid empirical answer has now completed 130 days! Gus is still dutifully tracking the effectiveness of the HEPA he’s been using in his bedroom since January (with a trip to the US in the middle), calculating the percent reduction in particulates every night.

I reported a few weeks ago that the HEPA was just as effective at Day 1 and Day 90. Now after 130 days and 951 hours of usage, here’s what the HEPA looks like:

1

Results: At 100 days, the overall effectiveness declined by about 4%. Even so, the HEPA is still removing 80% of .5 micron particles from the overall room air:

5

Conclusion: 

  1. On the Original DIY, the Smart Air HEPA lasted about 100 days without losing any effectiveness. That was for a total of 729 hours of usage or an average of about 8 hours a day (7.29 to be precise).
  2. Between days 100-130, the effectiveness dropped by about 4%. I think it’s up to you to decide whether a 4% drop is enough to make you change the HEPA.
  3. You can adjust these numbers to the number of hours you use the machine. For example, if you run the DIY 24 hours a day, you can get about 30 days with full effectiveness and at least 40 days with a roughly 4% reduction.
  4. As I’ve argued before, I prefer this type of test over the context-less numbers that some big purifier companies give because (1) it is based on air in China and (2) it gives the concrete number of hours you can use it rather than just the number of months. Of course, there will be some variation across different seasons, apartments, and cities in China.Limitations:
    1. This number doesn’t necessarily apply to the Cannon. On the one hand, the Cannon filter might have a shorter life because it pushes more air and therefore should collect particles faster. However, our Cannon HEPAs have a pre-filter, which extends the life of the HEPA. The next step is to do longevity tests with the Cannon!
    2. Gus changed apartments on Day 102, which is not ideal. However, the average effectiveness didn’t seem to be affected by his move (see the points at Day 100 and Day 110). We’ll repeat the tests in the future, and that will give us more reliable estimates.

    For now, the test continues! Stay tuned for the next report.

    I’m posting the raw data and detailed methods below for fellow nerds.

    more

    Because there is so much data, I’m making the raw data available here on my personal site. The methods are the same as in my previous tests.

    Basics: Gus did the tests overnight while he slept. The room door was closed during the tests. I calculated effectiveness by averaging the number of .5 micron particles over the last three hours before Gus woke up. Then I divided that number by the baseline measurement before Gus turned on the purifier.

    Effect of Apartment-mate’s Purifier: As I wrote in my last update, we discovered on Day 57 that Gus’s roommate was running a purifier 24 hours a day in his room. This may artificially boost our estimate of the lifespan. It also artificially lowered the baseline particle counts and thereby lowered the estimates of effectiveness for the first 57 days.

    Therefore, the raw data points for days 0-60 would give people a false sense that the HEPA increases in effectiveness after 60 days. To correct for that, I averaged difference between Days 0-57 and Days 58-90. The original graph is in my earlier post, and all of the raw data is available here.

    Outdoor Fluctuation: On Day 94 and Day 113, the outdoor PM 2.5 concentration increased by almost 100 micrograms during the test, so I removed these days from the calculation. Keeping those datapoints lowers those two 10-day estimates by about 3%. This data is retained in the raw data file.

    Room Change: For days 1-101, Gus was in his old apartment, which had a 12.3 m2 bedroom. For days 102-130, he was in his new apartment. Obviously that’s not ideal, but his landlord was taking back his old apartment! We will repeat the tests in his new apartment from scratch after this test ends.

    Calculating room size for Gus’s new bedroom is a little tricky because the ceiling is slanted. The floor area is 13.7 m2–slightly larger than his old room. But the slope of his ceiling gives it a volume of 26.3 m3 compared to 33.5 m3 if it were perfectly rectangular and 30.1 m3 in his old room.

    Thus, his new room may be easier to purify, although I suspect the roof is not very well sealed, so it may leak more than his old room.

1-17

How long do Smart Air HEPAs last? At least 90 days

The most frequent question people ask me these days is: how long does the HEPA last?

This question is important because replacement HEPAs are the biggest long-term cost of clean air. IQ Air charges $370 for its filters. So if you have to replace the HEPA every two weeks, the DIY might not save you money in the long run.

What’s Wrong with the Standard Numbers? I’ve been slow to answer because I wanted good data. Lots of filter companies give a single number (often 6 months), but they don’t say whether those numbers are for European air or Chinese air. If they last 6 months in Swedish air, I highly doubt they’d last 6 months in China’s thick air.

Another problem is that “6 months” doesn’t tell you how much you can use it per day. Is that 6 months for 24 hours a day? 8 hours a day?

A More Empirical Answer: So to get a concrete answer to this question, I think the best way is to get a new HEPA and test how effective it is every day. Over time, we can see when it starts to lose its effectiveness.

It’s been tough to start that test because I’ve been busy running tests of new HEPAs, fans, and other purifiers. But thankfully my collaborator Gus volunteered his bedroom as a laboratory, and we now have data from 90 days of tests.

Limitation: However, before I reveal the results, I discovered a flaw in the study. I was looking at Gus’s data, and I noticed the particle counts in his room before he turned on the purifier were a lot lower than they were in my room. Why?

Then we realized that Gus’s roommate was running his large Blue Air 24 hours a day. That made the air in his house cleaner than it would be in a normal house. Fortunately for science, the roommate moved his Blue Air out on Day 57, so we now have a normal testing environment.

However, this created two limitations in the data:

  1. It may inflate our estimates of the longevity of the filter because the air was cleaner than it would be in a normal Beijing apartment.
  2. It artificially lowered the estimates of effectiveness for the first 60 days. (At the end of the post, I compare data from when the Blue Air was being used nearby and after it was gone.) This also makes it hard to tell whether there were any slight changes in effectiveness over time.

Gus will start completely new tests after we finish these tests, so we’ll have a better answer later. We debated whether publishing this data would be useful. In the end, I decided that it’s better to give people some idea of the longevity, as long as I make it clear that there are limitations with the data.

Method: Gus used the Original DIY and the same HEPAs we ship from Smart Air every night in the 12.3 m2 bedroom in his Beijing apartment.

Gus used the same method as my previous tests to calculate effectiveness–the percentage reduction of particulates from the room air overnight.

To smooth out the variability in any single datapoint, I averaged the effectiveness over each 10-day period.

Here’s an example of what one of the 90 days looks like:

4

Here’s what the HEPA looks like after 90 days:

1

Results: After 90 days, the HEPA is still getting as many .5 micron particles as in my earlier tests:

5

Conclusion: This suggests Smart Air HEPAs last at least 90 days without losing effectiveness. Stay tuned to see how the next 90 days turn out!

Remember: these results can only be generalized to Smart Air HEPAs. HEPAs from different manufacturers vary in thickness and quality.

Number of hours: At the end of 90 days, Gus has used the HEPA for a total of 660 hours, an average of 7.33 hours per day. When we get a

total estimate of the number of hours the HEPA lasts, you can plug in the number of hours you use the filter per day to get an exact number of days for your own use.

For fellow data nerds, I’m posting more details about the tests below.

more

The Effect of Having a Second Purifier on Nearby: As I described in the main text, Gus’s roommate had a large Blue Air running for the first 56 testing days. My hypothesis was that the Blue Air was artificially lowering the baseline particle counts, which makes it harder to get high-percentage reductions.

We were able to test this hypothesis after his roommate prepared to move to Hong Kong and shipped his Blue Air there ahead of time.

To test my hypothesis, I compared the effectiveness (1) during the 56 days that the Blue Air was running and (2) the 34 days after his roommate stopped using the Blue Air:

3

The estimates of effectiveness rose to 86%–even higher than my original tests. This suggests that the earlier numbers were artificially low.

It also means that we can’t interpret any changes before Day 70:

6

The numbers go up when Gus’s roommate stopped using his purifier, but I think it’s wrong to say the DIY became more effective.

What we can say is that, even after 90 days of usage, the HEPA is still working as well as in my original tests.

Can we generalize the results? These tests are great for people living in China because the tests are based on Beijing air. However, the filter may last longer in places with cleaner air (for example, Hong Kong) and shorter in places with dirtier air (for example, Taiyuan).

In addition, fluctuations in AQI could make these estimates longer or shorter. For example, winter air tends to be more polluted.

Dropoff After 10 Days? The dot for the first 10 days is slightly higher than the later days, which suggests a dropoff after 10 days. However, I think that is not the case. In the first 10 days, Gus was still learning how to use the particle counter, and he failed to set it properly or get a good baseline reading on four of the days, so we couldn’t calculate effectiveness. That means the estimate for the first 10 days is less reliable.

What’s Going on at Day 30? The third datapoint (days 21-30) is the lowest. That is probably because the outdoor air was particularly bad in Beijing during that time. The numbers rebound slightly during the next 20 days.

Why HEPAs get better with age (to a point). Many people have the intuition that filters get worse over time, but HEPA filters actually get better at removing particles. That’s because the more you use it, the more particles fill up the pores in the HEPA, and it becomes harder for particles to get through.

The problem is that it also becomes harder for clean air to get through. So eventually, the HEPA will get so full of particles that it won’t let enough air through. That’s how the HEPA eventually dies.

Raw Data: The file is too large to display all of the raw data here, so I’m making it downloadable from my University of Virginia homepage (click here to download).

2015061632121471

How Safe is Indoor Air?

I recently had a conversation in Beijing that went something like this:

Friend: I’m not sure if I can make badminton tomorrow. I have a basketball game in the day.

Me: Oh man, do you play outside?

Friend: Nah, it’s inside.

Me: Oh, phew. Good.

Friend: Wait, why do you say that?

Me: Oh, the air is way worse outside. I used to feel like I had asthma after playing basketball outside.

Friend: Really? No, they’re not that much different. I saw it’s just 20% different.

Seeing as how nerds cannot let matters of fact go, I started using my particle counter to take measurements of inside and outside air at different locations around Beijing. This answer is important: it tells you if it’s any safer to exercise indoors and how much damage you’re doing to your lungs by choosing that seat outdoors at your favorite cafe or restaurant.

So I took measurements in six locations around Beijing, in apartments, cafes, and my gym. I only chose bad days (pollution concentration above the WHO standard of 25), and I avoided days where it rained (because rain can cause quick changes in air quality). Here’s what I found:

1

On average, indoor air had only 36% of the pollution outdoors.

Things were a little worse for the smaller .5 micron particles, but still much better than outside:

2

On average, indoor air had only 51% of the .5 micron particulates of outside air. My guess is that the .5 micron data was worse than 2.5 micron data because it’s easier for smaller particles to get into your home and stay suspended in the air.

There is a lot of variation between places. For the 2.5 micron particles, the locations varied from 14% to 58%. Dr. Saint Cyr also found significant variation between two apartments he lived in, 50% to 70%.

Conclusion: In terms of particulate pollution, you’re safer snagging an indoor seat and working out indoors, particularly on bad days (I’ve seen some argue that we are particularly vulnerable when we work out because we breathe more deeply than normal).

But remember that doesn’t mean indoor air is safe, just better than outside. For example, if your air at home had 40% of Beijing’s concentration last night at 11pm (8/15), you would’ve had 64 g/m3 in your home, which is more than twice the WHO standard of 25.

As usual, I’m posting more on my methods and raw data below.

(more…)