How accurate are common particle counters? Comparison test

When I started Smart Air in 2013, I wanted to buy a particle counter, and I had basically two options. I could buy a US$260 Dylos or spend thousands of dollars on the crazy expensive particle counters.

Since then, the market has exploded with new particle counters as cheap as 99 RMB. But are they any good?

Putting Particle Counters to the Test

To get to the bottom of it, we tested three popular particle counters on the market. We tested the Dylos DC1700, the Origins Laser Egg, and a new particle counter called the AirVisual Node.

 

Dylos

Dylos DC1100
Dylos DC1100

 

The Dylos is the trusty particle counter Smart Air has been using since 2013.

 

Laser Egg

Origins Laser Egg
Origins Laser Egg

 

The Laser Egg is the popular, more technologically savvy device.

Node

 

Air Visual Node
Air Visual Node

And the Node is a fancier version, including a large screen, richer information, pollution forecasts, better user experience and even a CO2 monitor.

 

allthree

The Government Comparison

We placed the machines outside the Smart Air office on Dongzhimen Waidajie, about 1.3km away from the government PM2.5 monitor at the Agricultural Exhibition Center.

 

Map of Smart Air Office and Agricultural Exhibition Centre

 

 

We ran the machines for six days. The Laser Egg and the Node give output in PM2.5 micrograms. The Dylos gives number of 0.5 micron particles, so we converted it to PM2.5 micrograms using the semi-official formula (0.5 microns – 2.5 microns)/100.

Here are the results for the first (72-hour) test outside our office in Beijing:

 

out-en-official-pm2-5-ag-exhib

 

Low-Concentration Tests

 

Next we tested on days with extraordinarily low PM2.5. That’s helpful because concentrations in homes—where most people use particle counters—are also typically low. So this data is good for testing how good the devices are at low concentration levels. We ran tests for 48 hours whilst the skies were clear.

Low concentration tests
Low concentration tests

 

Eyeballing both graphs, all three machines did a pretty good job of tracking the official numbers. Combining both tests, we found that both the AirVisual Node and the Laser Egg correlated r = 0.98 with the official PM2.5 numbers. For non-nerds, 0.98 is incredibly close to identical! The Dylos had the lowest correlation at r = 0.90, but still incredibly high (and similar to our previous test). These correlations are all extremely high and suggest that these particles counters are tracking government data well.

 

Average Deviation

 

Another way to measure accuracy is to look at on average how far the numbers were from the government data. The Node was the closest: it was off from the official numbers by an average of 4.8µg/m3. The Laser Egg was consistently further than the government machine, with an average deviation of 6.5µg/m3. The Dylos was off by an average of 9.1µg/m3.

 

Low concentration PM2.5 deviation from Agricultural Center

 

Perhaps one worry to note is that the Laser Egg was consistently under-estimating PM2.5 while air pollution was in the lower range. This means there could be a risk that the Laser Egg underestimates the real pollution levels in the home, giving a false sense of security. However, even these deviations were not large.

 

picture2 picture1 picture3

 

The Airpocalypse Test

To test accuracy at extremely high concentrations, we burned a cigarette in a closed 15m3 room. Our goal here was to see how well the particle counters were at reading concentration levels over a whole range of values, including toxic levels. With the help of cigarettes and a partner NGO in Beijing, we managed to get the concentration above 1,000µg/m3!

Airpocalypse test setup

For this test we also has another machine (Sibata LD-6S) on hand as a reference. This is an industrial PM2.5 dust indicator, with an accuracy of ±10% and repeatability error of ±2%. Thus, we used the LD-6S as our baseline.

 

Airpocalypse test results
Airpocalypse test results

 

Looking at the data, it’s immediately clear that the Laser Egg and the Dylos had a hard time keeping up with these really high levels of concentration. In contrast, the Node and the LD-6S matched very closely, and were able to measure values over 1,000µg/m3. The chances of you needing to measure these values outside of experiments are very slim, but it shows that the Node is more accurate at these high levels.

 

Take-Home Message

 

Overall, the three particle counters were reasonably accurate compared to the government machines. In the estimation of the Smart Air team, all of them are suitable for giving an approximate AQI value in your home. Of all three, the Node scored the highest, with the lowest deviation from the government machines in both outdoor tests and the highest accuracy in the “crazy bad” test.

 

Usability

 

Since all three machines are reasonably accurate, the question then really comes down to: How easy it is to use the device? And what features do they have?

 

The Dylos (1800 RMB)

The Dylos easily loses this fight. It has no phone connectivity, and downloading the data is a terrible pain—and that’s if you have one of the old school pin connecter cables.

 

The Laser Egg (499RMB)

The Laser Egg is an entry-point particle counter. It gives reasonably accurate results with a simple interface. It’s not feature rich, but it does what it says on the box.

 

The AirVisual Node (988RMB)

 To our eyes, the Node offers the best features. For starters, it can measure CO2, temperature, and humidity. That makes it more of an ‘environment monitor’ than just a particle monitor. CO2 is important if you have lots of people in a small space as it can give an indication of how confined the space is. If you have indoor sources of air pollution (VOCs) like new furniture or remodelling, high CO2 levels can mean that those indoor pollutants are building up. Its user design shows AQI and CO2 for the past 24 hours both indoors and outdoors, a forecast for the coming days, and suggestions to help you decide when to open your windows and wear a mask. We’ve found these features helpful in our office.


Conclusion

After passing our tests, we will start shipping the AirVisual Node through our Taobao shop  and website. It’s a great option for anybody wanting a solid device for both home use and research (if you’re a nerd like us). Go take a look!

Over the next few months, we hope to get a larger pool of particle counters together and run more extensive tests. This is only the beginning! Once we’ve independently verified more devices, we may well be adding them to our shop as well.

 

Source Pollution

10 Facts About Air Pollution

We get many questions about air pollution in our office, and understandably so. It’s a topic that isn’t well understood or well-reported about in certain parts of the countries in which we work. In some cases, it is difficult to distinguish research-backed findings from common beliefs. To contribute to collective learning, here is a quick list of top 10 facts about air pollution.

  1. Air pollution is made up of chemicals, particulates, and biological materials. Common  components include, but are not limited to: nitrogen, sulfur, carbon monoxide, carbon dioxide, dust, and ash.
  2. Air pollution is caused by both human and natural contributors. Industries, factories, vehicles, mining, agriculture, forest fires, volcanic eruptions, and wind erosion all cause air pollution.
  3. According to the Global Burden of Disease report (2013), air pollution contributes to more than 5.5 million premature deaths every year. Another report by the International Energy Agency estimates the number to be 6.5 million deaths per year.
  4. Research has linked air pollution to multiple diseases: acute lower respiratory infections, chronic obstructive pulmonary disease, lung cancer, tuberculosis, low birth weight, asthma, and cataract.
  5. According to the WHO, 98% of cities in low- and middle-income countries with more than 100,000 habitants have unsafe levels of air pollution.
  6. Of the top twenty most polluted cities in the world, 13 are in India and 3 are in China. Delhi ranks as 11th most polluted, whereas Beijing ranks as 57th most polluted.
  7. Over half of India’s population—660 million people—live in areas with unsafe levels of air pollution.
  8. On average, Indians living in polluted areas will lose 3.2 years of their lives due to air pollution.
  9. In 2014, India and China tied at 155 among 178 nations in rankings measuring how countries are tackling air pollution in the world, despite both countries having some of the worst air quality in the world.
  10. Pregnant women who live in  high traffic areas have a 22% higher risk of having children with impaired lung function than those living in less polluted areas.
Flickr Photo

Is Summer Air Better than Winter Air?

 

Summer is here, bringing with it clearer skies and certainly cleaner air. Right?

Summer always seems to drive out the dense clouds of pollution that suffocate many Indian cities. However, while summer air is in fact cleaner than air during other seasons, it’s still far from safe according to the standards set by the World Health Organization (WHO).

During the winter, cold air traps pollutants close to the ground, a process called an “inversion.” Summer heat prevents this inversion, which does improve the air quality. However, average air conditions in India are still clearly not ideal.

Here’s a map of today’s pollution levels across India:

 

pollution levels
Source: https://aqicn.org/map/india/

 

On a day like today, when the AQI in Chennai, Hyderabad, Kolkata, Mumbai and New Delhi is in the ‘unhealthy’ or ‘very unhealthy’ range, we often wonder at Smart Air if the pollution in summer really is any better than the winter.

We got to the bottom of it by analyzing the US Embassy’s data in New Delhi and US consulates’ data in Mumbai, Chennai, Hyderabad, and Kolkata. So is summer air really better than winter air? We took the data from the past two years (June 2014 to June 2016) and broke it down into four seasons: winter (December to February), summer (March to June), monsoon (July to September), and post-monsoon (October to November). Next, we calculated the average particulate pollution (PM2.5) levels for each season.

Across the five cities we looked at, PM 2.5 levels were 26% better in the summer—118 micrograms in the winter compared to 49 micrograms in the summer. That means summer air is better.

Let’s take a look at the difference in PM2.5 between the five cities during different seasons:

 

 

US Embassy Air Quality Data
U.S. Department of State Data, June 2014 – June 2016. Air quality data may not be validated or verified

 

But how good is “better?” Here in India, “better” is nowhere near “safe.” Over the course of the two years we analyzed, average annual pollution levels in all five cities never fell below even the WHO’s more lenient (24-hour) exposure limit (25 micrograms per cubic meter). In fact, the average pollution levels across all the cities we tested was about 500% the WHO annual limit (10 micrograms) and 200% of the more lenient 24-hour limit (25)!

 

The lowest summer pollution level we found was Chennai (31 micrograms). But even that lowest summer level still surpassed the WHO limits.

Below are the 2-year graphs for each city. You can see that each city has two distinct swells in PM2.5 levels during the winter, each followed by 2 clear dips during the summer. Interestingly enough, comparing the summer and winter levels of each city from 2014-2015 to 2015-2016 shows some cities’ PM2.5 levels improving, while others’ increase between years. Most notably, Chennai’s winter pollution levels dropped significantly between years as did Hyderabad’s, while New Delhi and Kolkata experienced clear increases. However, we’re not sure whether or not this improvement and worsening of PM2.5 levels can be attributed to cities’ environmental efforts (or lack thereof).

The conclusion? The evidence is quite clear: summer air is in fact better than winter air. However, despite all the blue skies and warm days we’ve been having lately, there’s still a need to protect yourself inside and outside the house. Don’t mistake “better” for “safe.” Neither summer nor winter air meets WHO health standards and summer air is still of significant concern to public health.

 

Chennai US Department of State
U.S. State Department Data – June 2014 to June 2016. Data may not be fully verified or validated.

 

US Embassy Air Quality
U.S. State Department Data – June 2014 to June 2016. Data may not be fully verified or validated.

 

US Embassy Air Quality Data
U.S. State Department Data – June 2014 to June 2016. Data may not be fully verified or validated.

 

US Embassy Air Quality Data
U.S. State Department Data – June 2014 to June 2016. Data may not be fully verified or validated.

 

US Embassy Air quality data
U.S. State Department Data – June 2014 to June 2016. Data may not be fully verified or validated.

 

cost-effective

Is it possible to lower the cost of clean air without sacrificing effectiveness?

A few months after I published my DIY tests, there were already Taobao stores up and running, selling DIYs even cheaper than my 200 RMB. I was intrigued. If we could really lower the cost of clean air, that’s a win!

But we have to be sure these cheaper machines are as effective. So I ordered the cheaper machines, tested them systematically, and found they performed much worse (2).

A New Way to Lower the Cost of Clean Air

But that doesn’t mean there aren’t ways to make HEPAs even more affordable. Recently I tested one way to do that: if we can shave 1 millimeter from the size of the HEPA, we can save 6% on the price. This new size still covers the fan opening, but does it still work as well?

Method

To test this question, my collaborator Anna ran 10 overnight room tests with the 29 mm in her Beijing apartment and compared it to my earlier tests of the 30 mm HEPA in the same 15m2 room.

e

Like in my earlier tests, I calculated effectiveness as the percent reduction in particulate from the start of the test to the average of the last four hours. Here’s what one of those tests looks like with the reduction calculation laid out. The blue line is indoor 0.5 micron particles; the red line is outdoor PM 2.5

q

Results

On average, the new 29mm HEPA reduced 0.5 micron particle levels in the room by 86% and 2.5 micron levels by 91%. These results were almost identical to my prior results with the 30mm HEPA.

w

Conclusion

This new HEPA lets us lower the cost of clean air without sacrificing any effectiveness. Win! We just lowered the price of Smart Air HEPAs from 80 RMB to 75 RMB.

HEPA Cost Comparison

Next I took that price and compared it to the two biggest brands out there, Blue Air and IQ Air. (Also check out the long-run cost comparison .)

Nerd Note on Replication

As a side note, this test is now the third series of room tests I’ve published with the Original DIY (early tests; 200-day longevity test). Add that to Dr. Saint Cyr’s tests, and I’d say that’s a satisfying amount of reproducibility.

For fellow data nerds, I’m posting the original data and more details about the test below.

1

2

3

4

Outdoor PM 2.5 Fluctuations

I like these long tests more than the common 20-minute tests (such as CADR tests) because this mimics how most people use purifiers. However, one drawback of these long tests is that outdoor PM 2.5 levels sometimes change over the course of 8 hours. If outdoor PM 2.5 drops, that can artificially inflate effectiveness. If outdoor PM 2.5 rises, that can artificially decrease effectiveness.

These changes should cancel out over 10 tests, but it’s worthwhile to re-run the analysis while excluding days with large outdoor changes. Among these 10 tests, two days (9/25 and 11/10) had large outdoor PM 2.5 changes. Without those two days, the averages were almost identical: 84% on 0.5 micron particles and 91% on 2.5 micron particles.

Average Outdoor PM 2.5 During Tests

It’s also worthwhile to check out bad outdoor PM 2.5 was during the tests to see if these days were representative of normal Beijing air. The average outdoor PM 2.5 was 116 micrograms. That’s about 20 micrograms higher than Beijing’s average PM 2.5 over the last five years (according to my analysis of the US Embassy’s PM 2.5 data). Thus, if anything, these tests are tougher than the Beijing average.

5

 

indoor vs outdoor

Shanghai Test: Is Indoor air better than outdoor air?

When I got my very first particle counter, I tested sites around Beijing to see whether indoor air was better than outdoor air. It was.

But that test had some limitations. My first particle counter didn’t have a battery, so I had to estimate outdoor particulate in some locations. I also didn’t look into any variables that could give some indoor locations better air than others.

Shanghai Test

Now I’ve got a fancy new Dylos DC1700 battery-power particle counter! Now I can easily take measurements indoors and outdoors. Here’s what it looks like:

d

I took it to Shanghai and tested 14 times in 11 locations on August 27-29. While I was there, the AQI averaged 158 (70 micrograms). I tested mostly around the French Concession, although I also made it out to Fudan University. None of the locations use air purifiers.

f

I tested in any type of place I could, and I mean any place. Here’s what I recorded in a public bathroom stall:

g

Results

I focused on the smaller 0.5 micron particles (which are highly correlated with government PM 2.5 readings). Across the 14 samples, here’s what I found. The red line represents how many particles were in outdoor air.

http://smartairfilters.com/cn/

On average, indoor air had just 69% of the particles of outdoor air.

Why do some places have cleaner air?

This fits with my findings in Beijing–indoor air has less particulate than outdoor air. But next I wanted to see if I could figure out why some places had cleaner air than others. For example, my unpurified apartment air (unpurified because I just got home) had just 30% of particles compared to outdoors, but the public bathroom had 134% of particles compared to outdoors. Woah!

I tested the simplest explanation possible: Were the windows and doors open? Most particulate pollution comes from outside. And in closed rooms, particles will slowly fall to the ground. Thus, indoor air should be better when the windows are doors are closed.

To test that idea, I looked only at places that had doors or windows open:

j

Yikes! If the doors or windows are open, I was breathing air that was basically as bad as outdoor air (92%). For example, here was how air compared indoors and outdoors in the public bathroom:

k

But things were much better in places with the doors and windows closed:l

With the doors and windows closed, the air had 57% of the particles of outdoors. For example, here’s the air inside Fudan University’s Economics Institute versus outdoors:z

In places where the doors and windows were closed, the air was always better than outside. The one exception was the Yuanyuan Restaurant, at 115%. Those particles could be coming from the kitchen.

So what?

The conclusion here is simple: We’re usually breathing much less particulate indoors than outdoors–even without a purifier.

Clean Lung Tips

  1. Work out indoors if possible. I work out indoors in a gym rather than outdoors when I’m in China.
  2. Sit inside at cafes or bars (unless people are smoking).

Does that mean indoor air is safe?

Indoor air has less particulate, but remember that “less particulate” does not always mean “safe.” Out of all 14 tests, NONE of the numbers was below the WHO 24-hour PM 2.5 limit:

x

One Last Exception

Finally, remember that pollutants can sometimes come from indoors. This is usually from the paints and chemicals used in remodeling and new furniture. If your home smells like paint or new furniture, you may be safer opening the windows (or at least using activated carbon).

2

Is Air Pollution a Problem in Mongolia?

图片 1

According to The Guardian, Ulaan Bator is the world’s second most polluted mega city. From 2008-2011, the average PM 2.5 level was nearly 150 micrograms. In 2014, Beijing averaged 98 micrograms; the WHO 24-hour limit is 25 micrograms.

 

2

Is it a bustling manufacturing sector? Manufacturing is still a tiny percentage of the Mongolian economy. Cars filling the streets? Not that either.

3

It turns out, most of Ulaan Baatar’s pollution is from people burning coal to keep warm. A little heat is necessary where winter temperatures dip to -40 C.

Yet all that air pollution causes 1 in every 10 deaths in the capital, according to an academic study. That makes clean air an urgent need in this city of 1.2 million people.

Smart Air will host its first-ever Mongolia workshops in Ulaan Baatar! Join us in the Mongolian capital September 11th and 12th.

Join us to learn how DIY purifiers can help remove dangerous particulate pollution from inside the home. Smart Air founder Thomas Talhelm will explain how even the priciest purifiers use a fan to push air through a HEPA filter. Workshop participants will make their own in just 10 minutes, and they’ll take it home to start breathing easier right away.

4-4

Are DIY purifiers still cheap in the long run?

When I published tests of the DIY compared to the expensive machines, a couple people on Zhihu (China’s Quora) asked if you really save money in the long run with the DIY once you calculate in the cost of changing the HEPA. After all, the IQ Air HEPA costs 1,782 RMB, but you can use it for more than a year.

Long-Term DIY HEPA Costs

At that time I didn’t have an answer. To get an honest answer, I needed to do tests in the real world, but that took almost a year to complete. (That’s more than I can say for the numbers IQ Air and Blue Air give. If filters last 6 months in Sweden, are they going to last 6 months in Beijing?)

Now it’s a year later, and I have that data. Smart Air co-founder Gus ran his Original DIY for eight hours a day and tracked what percentage of particulate it removed from the air each day with a Dylos particle counter.

1

Based on that data, I recommend changing HEPAs after 140 days at 8 hours per day (about 1,000 hours of use). Since that HEPA costs 80 RMB, that averages to .57 RMB per day and 208.6 RMB per year.

To get an idea of what that means, we can compare that to the cost of drinking a bottle of water a day:

2

Big Brand HEPA Costs

I’m highly skeptical that HEPA recommendations for Sweden can be mapped onto China. But to be conservative, I’ll assume in my calculations that their numbers work the same in China.

I calculated costs for the Blue Air 203 and IQ Air Health Pro Plus using the same conditions I used for the Original DIY longevity test:

  1. Highest setting
  2. 8 hours of use per day
  3. Extend those costs over a year
  4. Include the cost of pre-filters but not carbon filters

Blue Air

Blue Air’s HEPA costs 300 RMB and lasts 6 months. That works out to 1.67 RMB per day and 609.6 RMB per year.

3

IQ Air

IQ Air’s HEPA is more expensive: 1,782 RMB. On the sixth setting, it lasts 4,968 hours. If you use it 8 hours a day, that works out to 2.87 RMB per day and 1,047.4 RMB per year.

However, we have to add the cost of the pre-filter because the HEPA lifespan depends on the pre-filter. The pre-filter costs 645 RMB and lasts 2,016 hours on the sixth setting. That works out to 2.56 RMB per day and 934.3 RMB per year.

IQ Air grand total: 1,982 RMB per year.

4

Conclusion

The long-term HEPA cost for the Original DIY is 66-89% less than the Blue Air and IQ Air.

It should be noted that these results may not map onto the Cannon. I’m still working on the Cannon longevity test, so we’ll need to wait until that test is done.

As always, I’m writing more details about the data below for fellow nerds.

DIY HEPA Costs

DIY HEPA costs are based on the 200-day longevity test in real Beijing air. All 200 days of original data are available in that post.

IQ Air Costs

The IQ Air cost calculations are based on a table from the North American IQ Air authorized retailer. That data is available here. I welcome other nerds to double check my calculations!

One thing to note: the calculations above are based on the sixth setting. If the IQ Air runs on setting three, the effectiveness will be lower, but the HEPA will last longer. In my calculations, I totaled the costs for one year one setting three: 901 RMB. That’s lower than the calculation above (because it’s processing less air), but it’s still more than four times the cost of the DIY HEPA.

Blue Air Costs

Calculating costs for the Blue Air is more difficult. That’s because Blue Air doesn’t seem to publish longevity recommendations for different settings or how many hours you can use the HEPA. Instead, Blue Air says you should replace the HEPA after six months.

Thus, I sent Blue Air an email asking how many days I should replace the HEPA if I run my machine eight hours a day. A Blue Air representative told me six months and told me I would void the warranty if I used it more than six months. Thus, I used their recommendation in my calculations (original email).

If Blue Air is wrong and the HEPA lasts long at eight hours per day, the calculation changes. I have alternative calculations here.

2-6

What time of day is PM 2.5 the lowest?

My intuition has always been that air pollution is lowest at night because there are fewer cars on the road and fewer factories humming. Apparently I’m not alone: 139 voted for their guess about what time of day has the lowest PM 2.5, and night time came in first:

1

I also know people who arrange their schedules to work out in the morning to avoid the worst pollution. But how accurate are our intuitions?

To get to the bottom of it, I analyzed thousands of hours of PM 2.5 data from the US Embassy in Beijing. When I finally got the answer, I was surprised:

2

Instead of during the quiet of night, it’s the afternoon–right around rush hour–that PM 2.5 is the lowest. So if you’re planning a picnic or insist on exercising outside, you’re usually best off between noon and 6pm.

What about other cities?

Is that how PM 2.5 generally works, or is it unique to Beijing’s activity or climate? Fortunately, US consulates in several other cities publish their historical data.

Shenyang

In terms of climate, Shenyang is pretty similar to Beijing, and its daily PM 2.5 patterns are very similar (although night time isn’t quite as bad as Beijing):

3

Chengdu

The pattern in Chengdu is similar. PM 2.5 is lowest in the afternoon and highest in the morning.

4

Shanghai

The picture starts changing when you get to the south. In Shanghai, PM 2.5 is lowest in the afternoon (like the north), but there’s also a dip in the very early morning:

5

Guangzhou

Guangzhou’s pattern was virtually identical to Shanghai’s: a dip in the very early morning and a dip in the afternoon.

6

Bottom Line: When is it safest to be outside?

In all five cities, the afternoon had the lowest PM 2.5 levels. And in contrast to many people’s intuition, the night time had the worst air in several cities. Thus, you’re usually best off organizing your picnic or tai chi in the afternoon.

But keep in mind that the air is NEVER safe on average in any of the five cities at any time of day. So take “safest” with a grain of salt!

7

Open Data

All of the data is available from the US embassy and consulate websites. Thanks to Josh Malina for collecting and analyzing the consulate data.

1-8

Does adding a carbon layer reduce particulate effectiveness?

I recently published tests showing that carbon actually removes VOCs. But adding that carbon layer raises a question: adding carbon means the fan has to power through yet another layer of resistance. Does that make purifier less effective at removing particulates?

Methods

To test this question, I ran 10 room tests with the Cannon and 10 tests with the Original DIY in the same 15m2 Beijing apartment as my earlier tests without a carbon layer.

I measured how much particulate it removed with a Dylos particle counter and compared the particle counts (1) before I turned on the purifier at night and (2) the average of the last four hours before I woke up in the morning.

Results

With the additional carbon, the Cannon particulate effectiveness dropped 1-2%. Thus carbon has a very minor negative effect on the Cannon.

1

However, the Original DIY had a harder time powering through the extra layer. Its 0.5 micron effectiveness dropped 19%, and 2.5 micron effectiveness dropped 15%.

2

Conclusion

For people who need carbon (and that is not everyone), I would recommend adding the carbon to the Cannon, but I would think twice about adding carbon to the Original.

As always, I’m posting the raw data and more details on the test for fellow nerds below.

(more…)