{"id":15179,"date":"2019-04-25T11:00:47","date_gmt":"2019-04-25T19:00:47","guid":{"rendered":"http:\/\/www.palada.net\/index.php\/2019\/04\/25\/news-8928\/"},"modified":"2019-04-25T11:00:47","modified_gmt":"2019-04-25T19:00:47","slug":"news-8928","status":"publish","type":"post","link":"http:\/\/www.palada.net\/index.php\/2019\/04\/25\/news-8928\/","title":{"rendered":"Results are in! Our Study Measuring the Different Levels of Data Center Air Containment"},"content":{"rendered":"<p><strong>Credit to Author: Victor Avelar| Date: Thu, 25 Apr 2019 15:18:07 +0000<\/strong><\/p>\n<p>In <a href=\"https:\/\/blog.schneider-electric.com\/datacenter\/2019\/03\/19\/getting-it-right-how-much-air-containment-is-enough-in-your-data-center\/\" target=\"_blank\" rel=\"noopener noreferrer\">an earlier blog<\/a> I explained the problem with \u201cpercent leakage\u201d in regards to air containment and I proposed some more practical metrics. To quantify these metrics, we set up a test at the <a href=\"https:\/\/www.schneider-electric.com\/en\/work\/support\/showroom\/setc\/\" target=\"_blank\" rel=\"noopener noreferrer\">Innovation Executive Briefing Center<\/a> in St. Louis. In this blog, I discuss that study we performed in our cooling lab \u2013 with the help of Scott Buell and his awesome team.<\/p>\n<h2>Our Containment Study: the setup<\/h2>\n<p>We set up a 2-row pod, including four racks in each row with hot aisle containment (HAC) ducted back to the CRAH. \u00a0Although we had a raised floor, we took a hard floor approach where the CRAH supply air flooded the room (no perforated tiles anywhere). This \u201cbowling alley\u201d layout, where the CRAH supply jet crashes against the side of the pod, provided a worst case for our testing. We had 6 temperature sensors attached vertically across each rack. Other sensor arrays were placed in the hot aisle, CRAH return, CRAH supply, and in the cold aisle. We used a power meter for the CRAH and monitored power for all the server simulators. We also used a very sensitive <a href=\"https:\/\/www.schneider-electric.com\/en\/product\/ACAC22000_APC\/active-flow-controller\/?range=61775-thermal-containment&amp;filter=business-3-critical-power,-cooling-and-racks&amp;parent-category-id=7200&amp;parent-subcategory-id=7230\" target=\"_blank\" rel=\"noopener noreferrer\">pressure sensor<\/a> in the hot aisle that gave us an indication of the balance between CRAH supply air and IT airflow. This pressure sensor becomes less useful with less containment, but if you have 100% blanking panels, use aisle doors, and duct the hot or cold aisle, I would recommend using a pressure sensor. All this data was collected with two data loggers and fed back to a workstation.<\/p>\n<h2>Removing Variables \u2013 Controlling the Test-environment<\/h2>\n<p>Our tests started with finding the minimal airflow required to cool the servers in an ideal \u201cairtight\u201d case (yes, with lots of tape). Then we introduced practices that resulted in leakage. In all, we ran 44 cases. The ideal \u201cairtight\u201d case started with a standard <a href=\"https:\/\/www.schneider-electric.us\/en\/work\/solutions\/for-business\/data-centers-and-networks\/hyperpod\/\" target=\"_blank\" rel=\"noopener noreferrer\">HyperPod<\/a> containment solution with brush strips, doors, hot aisle duct to the CRAH return, and 8 racks at kW densities 7, 2, 5, 9.5, 2, 12, 4, and 3.5. (a total of 45kW with an average of 5.7kW\/rack). We used 15 server simulators, 10U each, to simulate the stated densities. Then, to make it airtight, we used cardboard and gaff tape to cover up every single penetration we could see. This took about one week to seal everything, including cable penetrations. Every gap was covered in cardboard and tape except for the server simulators. I don\u2019t recommend this\u2026 super time consuming! Measurement is also time consuming because you have to wait 25-100 minutes for the temperatures to reach steady state, for every change you make.<\/p>\n<p>Note that when the test called for missing blanking panels, we color-coded the panels to ensure we consistently removed the same panels. We had three cases: 100% panels installed (186 total panels), 87% installed, and 60% installed.<\/p>\n<p> <img decoding=\"async\" style=\"width: 192px;\" src=\"https:\/\/coschedule.s3.amazonaws.com\/167392\/80ee65fc-8f4c-4e77-ab83-a838fcf4999c\/1556112397252.jpeg\" width=\"189\" \/><img decoding=\"async\" style=\"width: 193px;\" src=\"https:\/\/coschedule.s3.amazonaws.com\/167392\/4fb4cb8c-44fd-4431-bf44-c52727c0c8ff\/1556112397440.png\" width=\"269\" \/> <\/p>\n<p>&nbsp;<\/p>\n<p> <img decoding=\"async\" style=\"width: 192px;\" src=\"https:\/\/coschedule.s3.amazonaws.com\/167392\/db6b1ba5-0857-4335-86a1-a2bc55367069\/1556112397647.png\" width=\"253\" \/><img decoding=\"async\" style=\"width: 340px;\" src=\"https:\/\/coschedule.s3.amazonaws.com\/167392\/9ea13182-f114-4031-832e-2e2285fb26a3\/1556112397873.jpeg\" width=\"333\" \/> <\/p>\n<h2>Setting the Benchmark with the Ideal \u2018Airtight\u2019 Case<\/h2>\n<p>By changing the CRAH fan speed we found the airflow at which there was nearly a zero-pressure difference between the hot and cold aisle (this gives us a good indication that the CRAH and IT airflows are about the same). Then we opened the CRAH chilled water valve to bring the average IT supply air temperature at the rack fronts to 21\u00b0C\/70\u00b0F (our target for all cases). Once we measured our ideal \u201cairtight\u201d case, we were able to benchmark the subsequent \u201cleaky\u201d cases to this baseline. What where the metrics for our ideal airtight baseline?<\/p>\n<ul>\n<li><strong>average IT supply temperature at the rack fronts was 21.1<\/strong><strong>\u00b0C<\/strong>\/<strong>70<\/strong><strong>\u00b0F<\/strong>,<\/li>\n<li><strong>maximum temperature was 22.1<\/strong><strong>\u00b0C<\/strong>\/<strong>71.8<\/strong><strong>\u00b0F<\/strong><\/li>\n<li><strong><a href=\"https:\/\/en.wikipedia.org\/wiki\/Standard_deviation\" target=\"_blank\" rel=\"noopener noreferrer\">standard deviation<\/a><\/strong><strong>\u00a0of 0.4<\/strong><strong>\u00b0C\/0.8<\/strong><strong>\u00b0F<\/strong><\/li>\n<\/ul>\n<p>For reference, the average CRAH power consumption was 616 watts, chilled water valve 24% open, and CRAH fan speed 36%. With each case hereafter, we started the test with the same valve position and fan speed as the ideal airtight case, and then increased both from there to find the point that brought us closest to the ideal temperature and standard deviation.<\/p>\n<h2>The Results Are In \u2013 Measured Outcomes Across 5 Cases<\/h2>\n<p>Each of the following cases tested questions we had to air containment. Check out our findings.<\/p>\n<p>Case 1 \u2013 What is the minimal required containment possible?<\/p>\n<ul>\n<li><em>Used<\/em>: 60% blanking panels, no hot aisle ducting, pod doors closed, \u201cV\u201d shaped diffuser in front of CRAH<\/li>\n<li><em>Result<\/em>: average IT supply air temperature was <u>28.5<\/u><u>\u00b0C\/83.2<\/u><u>\u00b0F<\/u> with a standard deviation of <u>4.1<\/u><u>\u00b0C\/7.4<\/u><u>\u00b0F<\/u>.<\/li>\n<li>Findings: this standard deviation is almost <strong>10 times higher<\/strong> and the average temperature is <strong>7.6<\/strong><strong>\u00b0C\/13.6<\/strong><strong>\u00b0F higher<\/strong> than the \u201cairtight\u201d case.<\/li>\n<\/ul>\n<p>Case 2 \u2013 What happens when I add the rest of the blanking panels?<\/p>\n<ul>\n<li><em>Used<\/em>: 100% blanking panels, no hot aisle ducting, pod doors closed, \u201cV\u201d shaped diffuser in front of CRAH<em>\u00a0<\/em><\/li>\n<li><em>Result<\/em>: the average temperature decreased to <u>25.2<\/u><u>\u00b0C\/77.3<\/u><u>\u00b0F<\/u> with a standard deviation of <u>1.3<\/u><u>\u00b0C\/2.4<\/u><u>\u00b0F<\/u><\/li>\n<li><em>Findings<\/em>: The standard deviation is only about <strong>3 times higher<\/strong> and the average temperature is <strong>4.3<\/strong><strong>\u00b0C\/7.7<\/strong><strong>\u00b0F higher<\/strong> than the \u201cairtight\u201d case. So blanking panels gave us a big step in the right direction.<\/li>\n<\/ul>\n<p>Case 3 \u2013 What if I fully ducted the hot aisle to the CRAH in Case 1?<\/p>\n<ul>\n<li><em>Used<\/em>: 60% blanking panels, full ducting, pod doors closed, \u201cV\u201d shaped diffuser in front of CRAH<em>\u00a0<\/em><\/li>\n<li><em>Results<\/em>: the average temperature decreased to <u>22.3<\/u><u>\u00b0C\/72.2<\/u><u>\u00b0F<\/u> with a standard deviation of <u>2.3<\/u><u>\u00b0C\/4.1<\/u><u>\u00b0F<\/u><\/li>\n<li><em>Findings<\/em>: The standard deviation is now over <strong>5 times higher<\/strong> and the average temperature is <strong>1.4<\/strong><strong>\u00b0C\/2.5<\/strong><strong>\u00b0F higher<\/strong> than the \u201cairtight\u201d case. What this seems to indicate (for our test) is that 100% blanking panels reduce the temperature variation, while ducting reduces the average temperature.<\/li>\n<\/ul>\n<p>Case 4 \u2013 What\u2019s the impact of diffusing the CRAH supply air?<\/p>\n<ul>\n<li><em>Used<\/em>: 60% blanking panels, full ducting, pod doors closed, diffuser in front of CRAH<\/li>\n<li><em>Results<\/em>: the average temperature increased to <u>23.2<\/u><u>\u00b0C\/73.7<\/u><u>\u00b0F<\/u> with a standard deviation of <u>2.9<\/u><u>\u00b0C\/5.2<\/u><u>\u00b0F<\/u><\/li>\n<li><em>Findings<\/em>: the standard deviation is now almost <strong>7 times higher<\/strong> and the average temperature is <strong>2.3<\/strong><strong>\u00b0C\/4.1<\/strong><strong>\u00b0F higher<\/strong> than the \u201cairtight\u201d case. This tells us that the diffuser helps improve both the variation and average temperature.<\/li>\n<\/ul>\n<p>Case 5 \u2013 Combine all best practices except diffuser?<\/p>\n<ul>\n<li><em>Used<\/em>: 100% blanking panels, full ducting, pod doors closed<\/li>\n<li><em>Results<\/em>: doing this resulted in an average temperature <u>21.3<\/u><u>\u00b0C\/70.3<\/u><u>\u00b0F<\/u> with a standard deviation of <u>0.6<\/u><u>\u00b0C\/1.1<\/u><u>\u00b0F<\/u><\/li>\n<li><em>Findings<\/em>: The standard deviation is now <strong>1.5 times higher<\/strong> and the average temperature is <strong>0.4<\/strong><strong>\u00b0C\/0.7<\/strong><strong>\u00b0F higher<\/strong> than the \u201cairtight\u201d case.<\/li>\n<\/ul>\n<p>At the start of our testing, we assumed that air jets had little impact on temperatures when everything was well-sealed. Unfortunately, this was a bad assumption and we didn\u2019t measure the impact of this in Case 5. Had we inserted a diffuser in the ideal airtight case and Case 5, I believe we would\u2019ve seen an improvement in the standard deviation and average temperature.<\/p>\n<h2>Drawing Conclusions from the 5 Air Containment Cases<\/h2>\n<p>The table below summarizes the values for each experiment:<\/p>\n<p> <img decoding=\"async\" style=\"width: 616px;\" src=\"https:\/\/coschedule.s3.amazonaws.com\/167392\/6f8af926-aaf9-45a9-9b34-d58477aae019\/1556112477412.png\" \/> <\/p>\n<p>The table includes the maximum rack temperatures for each case. Maximum temperature is obviously important if you care about hot spots. Chances are that if you\u2019re experiencing hot spots, you\u2019re missing blanking panels. So, what does this tell us? It tells us that blanking panels are a huge driver in reducing maximum inlet temperatures. Consider Case 2 and 3. Case 2 has <strong>no return duct<\/strong>, but has 100% blanking panels. Case 3 has a fully ducted return but missing only 75 of the 186 panels. Yet, Case 2 (un-ducted) has a lower maximum temperature than Case 3. But notice that the average for Case 3 is lower than Case 2, thanks to the fully ducted return.<\/p>\n<p>In comparing the effect of the diffuser (Case 3 and 4), we see that the diffuser has a significant impact on maximum temperature as well. But this effect will be highly variable in your data center. Finally, in comparing the ideal case with Case 5, we can see that going crazy with gaff tape and cardboard is questionable in terms of the return you get on standard deviation, average, and maximum inlet temperature improvements. Also remember that this was done with a worst case \u201cbowling alley\u201d layout, therefore the effect of jets in other layouts may not be as pronounced.<\/p>\n<p><em>So, what\u2019s next?<\/em> Re-read this blog again to be sure to digest everything. Then, I\u2019m following up with a summary of best practices you can use for your data center, based on the data and findings from this project. Please leave a comment below and let me know what you think of our containment test and the results we documented!<\/p>\n<p>The post <a rel=\"nofollow\" href=\"https:\/\/blog.schneider-electric.com\/datacenter\/2019\/04\/25\/results-are-in-our-study-measuring-the-different-levels-of-data-center-air-containment\/\">Results are in! Our Study Measuring the Different Levels of Data Center Air Containment<\/a> appeared first on <a rel=\"nofollow\" href=\"https:\/\/blog.schneider-electric.com\">Schneider Electric Blog<\/a>.<\/p>\n<p><a href=\"https:\/\/blog.schneider-electric.com\/datacenter\/2019\/04\/25\/results-are-in-our-study-measuring-the-different-levels-of-data-center-air-containment\/\" target=\"bwo\" >http:\/\/blog.schneider-electric.com\/feed\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p><strong>Credit to Author: Victor Avelar| Date: Thu, 25 Apr 2019 15:18:07 +0000<\/strong><\/p>\n<p>In an earlier blog I explained the problem with \u201cpercent leakage\u201d in regards to air containment and I proposed some more practical metrics. To quantify these metrics, we set up&#8230;  <a href=\"https:\/\/blog.schneider-electric.com\/datacenter\/2019\/04\/25\/results-are-in-our-study-measuring-the-different-levels-of-data-center-air-containment\/\" title=\"ReadResults are in! Our Study Measuring the Different Levels of Data Center Air Containment\">Read more &#187;<\/a><\/p>\n<p>The post <a rel=\"nofollow\" href=\"https:\/\/blog.schneider-electric.com\/datacenter\/2019\/04\/25\/results-are-in-our-study-measuring-the-different-levels-of-data-center-air-containment\/\">Results are in! Our Study Measuring the Different Levels of Data Center Air Containment<\/a> appeared first on <a rel=\"nofollow\" href=\"https:\/\/blog.schneider-electric.com\">Schneider Electric Blog<\/a>.<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"colormag_page_container_layout":"default_layout","colormag_page_sidebar_layout":"default_layout","footnotes":""},"categories":[12389,12388],"tags":[21297,12391,21298],"class_list":["post-15179","post","type-post","status-publish","format-standard","hentry","category-scadaics","category-schneider","tag-containment","tag-data-center","tag-percent-leakage"],"_links":{"self":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts\/15179","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/comments?post=15179"}],"version-history":[{"count":0,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/posts\/15179\/revisions"}],"wp:attachment":[{"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/media?parent=15179"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/categories?post=15179"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/www.palada.net\/index.php\/wp-json\/wp\/v2\/tags?post=15179"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}