How accurate are common particle counters? Comparison test

When I started Smart Air in 2013, I wanted to buy a particle counter, and I had basically two options. I could buy a US$260 Dylos or spend thousands of dollars on the crazy expensive particle counters.

Since then, the market has exploded with new particle counters as cheap as 99 RMB. But are they any good?

Putting Particle Counters to the Test

To get to the bottom of it, we tested three popular particle counters on the market. We tested the Dylos DC1700, the Origins Laser Egg, and a new particle counter called the AirVisual Node.

 

Dylos

Dylos DC1100
Dylos DC1100

 

The Dylos is the trusty particle counter Smart Air has been using since 2013.

 

Laser Egg

Origins Laser Egg
Origins Laser Egg

 

The Laser Egg is the popular, more technologically savvy device.

Node

 

Air Visual Node
Air Visual Node

And the Node is a fancier version, including a large screen, richer information, pollution forecasts, better user experience and even a CO2 monitor.

 

allthree

The Government Comparison

We placed the machines outside the Smart Air office on Dongzhimen Waidajie, about 1.3km away from the government PM2.5 monitor at the Agricultural Exhibition Center.

 

Map of Smart Air Office and Agricultural Exhibition Centre

 

 

We ran the machines for six days. The Laser Egg and the Node give output in PM2.5 micrograms. The Dylos gives number of 0.5 micron particles, so we converted it to PM2.5 micrograms using the semi-official formula (0.5 microns – 2.5 microns)/100.

Here are the results for the first (72-hour) test outside our office in Beijing:

 

out-en-official-pm2-5-ag-exhib

 

Low-Concentration Tests

 

Next we tested on days with extraordinarily low PM2.5. That’s helpful because concentrations in homes—where most people use particle counters—are also typically low. So this data is good for testing how good the devices are at low concentration levels. We ran tests for 48 hours whilst the skies were clear.

Low concentration tests
Low concentration tests

 

Eyeballing both graphs, all three machines did a pretty good job of tracking the official numbers. Combining both tests, we found that both the AirVisual Node and the Laser Egg correlated r = 0.98 with the official PM2.5 numbers. For non-nerds, 0.98 is incredibly close to identical! The Dylos had the lowest correlation at r = 0.90, but still incredibly high (and similar to our previous test). These correlations are all extremely high and suggest that these particles counters are tracking government data well.

 

Average Deviation

 

Another way to measure accuracy is to look at on average how far the numbers were from the government data. The Node was the closest: it was off from the official numbers by an average of 4.8µg/m3. The Laser Egg was consistently further than the government machine, with an average deviation of 6.5µg/m3. The Dylos was off by an average of 9.1µg/m3.

 

Low concentration PM2.5 deviation from Agricultural Center

 

Perhaps one worry to note is that the Laser Egg was consistently under-estimating PM2.5 while air pollution was in the lower range. This means there could be a risk that the Laser Egg underestimates the real pollution levels in the home, giving a false sense of security. However, even these deviations were not large.

 

picture2 picture1 picture3

 

The Airpocalypse Test

To test accuracy at extremely high concentrations, we burned a cigarette in a closed 15m3 room. Our goal here was to see how well the particle counters were at reading concentration levels over a whole range of values, including toxic levels. With the help of cigarettes and a partner NGO in Beijing, we managed to get the concentration above 1,000µg/m3!

Airpocalypse test setup

For this test we also has another machine (Sibata LD-6S) on hand as a reference. This is an industrial PM2.5 dust indicator, with an accuracy of ±10% and repeatability error of ±2%. Thus, we used the LD-6S as our baseline.

 

Airpocalypse test results
Airpocalypse test results

 

Looking at the data, it’s immediately clear that the Laser Egg and the Dylos had a hard time keeping up with these really high levels of concentration. In contrast, the Node and the LD-6S matched very closely, and were able to measure values over 1,000µg/m3. The chances of you needing to measure these values outside of experiments are very slim, but it shows that the Node is more accurate at these high levels.

 

Take-Home Message

 

Overall, the three particle counters were reasonably accurate compared to the government machines. In the estimation of the Smart Air team, all of them are suitable for giving an approximate AQI value in your home. Of all three, the Node scored the highest, with the lowest deviation from the government machines in both outdoor tests and the highest accuracy in the “crazy bad” test.

 

Usability

 

Since all three machines are reasonably accurate, the question then really comes down to: How easy it is to use the device? And what features do they have?

 

The Dylos (1800 RMB)

The Dylos easily loses this fight. It has no phone connectivity, and downloading the data is a terrible pain—and that’s if you have one of the old school pin connecter cables.

 

The Laser Egg (499RMB)

The Laser Egg is an entry-point particle counter. It gives reasonably accurate results with a simple interface. It’s not feature rich, but it does what it says on the box.

 

The AirVisual Node (988RMB)

 To our eyes, the Node offers the best features. For starters, it can measure CO2, temperature, and humidity. That makes it more of an ‘environment monitor’ than just a particle monitor. CO2 is important if you have lots of people in a small space as it can give an indication of how confined the space is. If you have indoor sources of air pollution (VOCs) like new furniture or remodelling, high CO2 levels can mean that those indoor pollutants are building up. Its user design shows AQI and CO2 for the past 24 hours both indoors and outdoors, a forecast for the coming days, and suggestions to help you decide when to open your windows and wear a mask. We’ve found these features helpful in our office.


Conclusion

After passing our tests, we will start shipping the AirVisual Node through our Taobao shop  and website. It’s a great option for anybody wanting a solid device for both home use and research (if you’re a nerd like us). Go take a look!

Over the next few months, we hope to get a larger pool of particle counters together and run more extensive tests. This is only the beginning! Once we’ve independently verified more devices, we may well be adding them to our shop as well.

 

30元滤网靠谱吗

Is a 30 RMB HEPA Possible?

After I published directions for how to make your own purifier, people asked me: “Which HEPAs should I use? Is this one trustworthy?”

That’s the type of question that makes a nerd like me happy because it means we need to get more data. So I ordered HEPAs from every manufacturer I could find, and I tested them all. After all the testing was done, I found I could ship the HEPAs that worked best for 80 RMB, which was cheaper than the 110 RMB HEPAs I was buying–quality and price!

Can HEPAs be even Cheaper? 

Later I found HEPAs for 20 RMB wholesale. I was excited. If HEPAs are that cheap, we can make the DIY even cheaper!

But the test data was terrible. These HEPAs weren’t anywhere close to getting 99% of particles, so I passed on them. If didn’t want to use it in my home, why would I want to ship them to other people?

A 99.97% HEPA for 30 RMB?

Thus I wasn’t surprised a couple weeks ago when I saw a store on Taobao selling HEPAs for 30 RMB and claiming that my HEPAs are 暴利 (aggressively overpriced). They claim that their HEPAs get 99.97% of particles, and if that’s true, maybe these were the holy grail of HEPAs I was looking for all along!

So I ordered two online and put them to the test. The first shock was seeing that it doesn’t have a frame:

1

That makes it harder retain its shape, but it might still be effective without a frame, so I put it through the tests.

Air Outlet Test

First, I tested it by putting it on the Cannon and testing the air coming out of the HEPA with a Met One Aerocet 531S. (The Met One is useful because it has a pump to regulate airflow. In air outlet tests, the particle counter is sitting in a stream of air, so using a pump maintains constant readings.)

2

The results weren’t pretty. Smart Air HEPAs got over 99.9% of particles, but the 30 RMB HEPA was below 90%–far below their claim of 99.97%.

Air Speed

But particle effectiveness isn’t everything. A HEPA in the 80% range might work better if it has better air flow. In that case, maybe the HEPA could process the air more times and clean the room air as well as a real HEPA.

To test that possibility, I put each HEPA on the Cannon and used a tool to measure air speed (fancy name “anemometer”). I placed the anemometer on the HEPA at four locations (left, right, top, bottom) and took the average air speed.

3

Again, the results weren’t pretty. So not only was the 30 RMB HEPA getting far fewer particles, it was letting much less air through.

Conclusion

Quality HEPAs for 30 RMB are still a dream. They’re not useless, but using this 30 RMB HEPA would expose people to significantly more particles.

I still hold out hope that manufacturers will be able to innovate cheaper HEPAs without sacrificing quality, but I haven’t seen those HEPAs yet.

Is the Taobao Store Owner Being Dishonest? 

The 30 RMB HEPA store makes claims that their HEPAs get 99.97% of particles, and the data clearly contradicts that, so it’s tempting to think that they’re lying.

But are they? I don’t know what’s in their mind, but my guess is that they simply didn’t go through the hassle of buying a particle counter and testing the HEPAs. I suspect that half of what seems like dishonesty on Taobao is actually just sloppiness.

As usual, I’m posting the raw data below. (more…)

2-19-1024x768

Are All DIYs the Same? Poor Results from 还我蓝天

When I did my first experiments, several people told me not to publish the data. “Don’t give it away for free,” they told me. “Use it to make money!” I decided then that my main goal wasn’t to make money. I almost got tricked into paying $1,000 for clean air, and I wanted to help people avoid getting tricked too, so I published the data anyway.

Of course, publishing the instructions online has made it easy for people to copy the idea. 还我蓝天 (Huanwo Lantian) was one of the first to follow in our footsteps, selling a DIY filter a few months back. They even use a screen capture of Gus’s appearance on Chinese TV on their shop:

1

I was curious to see how their filter works, so I decided to order one off of Taobao and put it to the test.

Now I’m in an awkward position because I found that their HEPA was not working nearly as well as my Smart Air HEPAs. It’s awkward because, if I publish the data, will people think I’m just trying to attack a competitor?

In the end, I think it’s better to publish the results and be honest about my conflict of interest. At the very least, I think people have an interest in knowing how well other DIYs work–especially when some of those websites use graphs that are lifted from my site, which can mislead people into thinking the test results are from their machines.

And as always, I’m publishing my raw data and testing methods at the end of this post, so fellow nerds can replicate my studies.

Method: My collaborator Anna used the same methods as our earlier tests in her 15m2 room. Anna did five overnight tests with the same Dylos Pro particle counter, and I calculated effectiveness as the percent reduction of particles in the room air, averaging the last three hours (more info). Then I compared the results to my earlier tests in the same room.

Here’s what I found:

2

Results: The 还我蓝天 DIY removed 21% fewer particles 0.5 microns and above and 11% less 2.5 micron particles than the Original.

Is it the fan? The 还我蓝天 fan is slightly smaller than the Smart Air Original, so one explanation could be that the 还我蓝天 fan is just moving less air. Anna tested that by strapping the 还我蓝天 HEPA onto our Original fan.

Here’s what I found:

3

Result: There wasn’t much difference. With the new fan, it was getting 4% more 0.5 micron particles and 2% less 2.5 micron particles. Thus, the fan doesn’t seem to be the reason.

Is it the HEPA? The second possibility is that the 还我蓝天 HEPA isn’t as good. Anna tested the HEPA by doing air outlet tests with a Met One GT-521, which measures down to 0.3 microns. Anna tested the air coming out of the HEPA for 10 seconds, and I averaged the results from three tests. (More details at the end of the post.)

Here’s what I found:

4

Results: The 还我蓝天 HEPA performed worse, about 7% lower than the HEPA standard. The major source of the 还我蓝天’s poor performance seems to be the quality of the filter.

Conclusion: In room tests, the 还我蓝天 DIY removes about 21% fewer particles than the Original DIY, and the data suggests that the reason is that 还我蓝天 HEPA is lower quality.

Now, don’t get me wrong. The 还我蓝天 DIY is making the room air cleaner. I’d rather have a 还我蓝天 than nothing. But the results show that this DIY copycat is cutting corners by using cheap HEPAs.

As always, I’m posting the raw data and more detail on the methods for fellow nerds.

(more…)

1-24-1024x768

A New Way to Test Whether the DIY Works

So far, I’ve been testing air purifiers by taking a baseline measurement of particulate pollution in a room, and then turning on the purifier and testing whether the counts drop. I’ve used that method to test the DIY and more expensive machines.

However, I recently bought a second particle counter, so my collaborator Gus suggested another method: run one particle counter in the bedroom that has the purifier, and run another particle counter in a different room that does NOT have a purifier. The benefit of this method is that the control room represents the counterfactual–what would have happened if we hadn’t turned on the air purifier.

Thus, if a northwest wind hits Beijing and makes the outdoor air a lot cleaner, we can separate the effect of the outdoor air fluctuations from the effect of the purifier. In that situation, my old method would artificially raise our estimates of effectiveness. Changes in outdoor air can also artificially lower our estimates of effectiveness if the outdoor air gets dirtier after we turn on the purifier.

In previous tests, I corrected for this by averaging over multiple tests. I also analyzed the data after removing days in which outdoor air pollution fluctuated a lot (for example, I do that sort of analysis in the extra nerd notes here).

But it’s always nice to use different types of tests to make sure an effect is real, so Gus did this experiment. He set up one particle counter in his room and one in his kitchen: 1

He let the particle counters run for several hours, and then a timer turned on the Original DIY in his room. (The kitchen had no air purifier.) Here’s what happened:

2

The difference between the bedroom and the kitchen air quality can approximate the effect of the air purifier. It looks like Gus would have been breathing 16,000 of these 0.5 micron particles in the air in his bedroom if he hadn’t turned on his DIY purifier.

And it’s pretty clear that the kitchen air quality (where we don’t have a purifier running) is following outdoor air quality:

3

(Be aware that I’m overlaying these two lines on the same graph, but the Y-axes are different. This is NOT saying that indoor air is as bad as outdoor air. Indoor air is usually cleaner than outdoor air.)

Conclusion: Similar to earlier tests, the double particle counter test shows that the DIY purifier is removing particulate pollution from the air.

As always, I’m including more details for fellow data nerds below.

more

What about the 2.5 micron particles? In the main text, I present the 0.5 micron results, but what about the larger particles? The results there are similar:

4

Remember that because these particles are larger, they are more affected by people moving around. That’s probably why there are large spikes from 7 to 9pm, and why counts level off after midnight. However, even after the numbers level off, the purifier is reducing the counts in bedroom even further.

Timing: Careful readers will notice a slight lag between the bedroom and kitchen numbers. That’s because after entering its hourly mode, the Dylos particle counter takes its first measurement after a random number of minutes. (After that, it’s every 60 minutes.)

Therefore, we cannot guarantee the kitchen and bedroom numbers are taken at the exact same time. However, after the fact, we can tell when the measurements were taken. Thus, we know that the bedroom numbers were taken at 27 minutes past every hour, and the kitchen numbers were taken at 36 minutes past the hour. Those numbers are reflected on the X-axis.

Percentage effectiveness: I also computed the percentage reductions in particulates using my regular method of averaging the last four hours before the DIY was turned off. Then I compared that as a percentage of particulate in the kitchen.

According to that calculation, the DIY removed 86% of 2.5 micron particles and 87% of 0.5 micron particles. Compare that to 92% of 2.5 micron particles and 84% of 0.5 micron particles in my earlier tests. However, I am cautious about drawing strong conclusions from one day’s data. It would be best to average the results of several testing days.