#Government – HPCwire

[strategictech]

Hyperion Research: Eleven HPC Predictions for 2024

HPCwire is happy to announce a new series with Hyperion Research – a fact-based market research firm focusing on the HPC market. In addition to providing market insights and research, Hyperion also sponsors the HPC User Forum, a twice-a-year US and two international meetings where important topics are discussed. A volunteer steering committee of major user organizations in government, industry, and academia sets meeting agendas. This year, the Spring meeting is in Reston, Virginia.

@tonyshan #techinnovation https://bit.ly/tonyshan https://bit.ly/tonyshan_X

[mthrynn]

In the race to commercialize quantum computing, IBM is one of several companies leading the pack. Today, IBM announced it had signed JPMorgan Chase, Daimler AG, Samsung and a number of other corporations to its IBM Q Network, which provides online access to IBM’s experimental quantum computing systems. IBM is also establishing regional research hubs at IBM Research in New York, Oak Ridge National Lab in Tennessee, Keio University in Japan, Oxford University in the United Kingdom, and the University of Melbourne in Australia.

IBM Q system control panel (photo: IBM)

Twelve organizations in total will be using the IBM prototype quantum computer via IBM’s BlueMix cloud to accelerate quantum development as they explore a broad set of industrial and scientific applications. Other partners include JSR Corporation, Barclays, Hitachi Metals, Honda, and Nagase.

Partners currently have access to the 20 qubit IBM Q system, which IBM announced last month, but Big Blue is also building an operational prototype 50 qubit processor, which will be made available in next generation IBM Q systems. The partners will specifically be looking to identify applications that will elicit a quantum advantage, such that they perform better or faster on a quantum machine than a classical one.

IBM leadership believes we are at the dawn of the commercial quantum era. “The IBM Q Network will serve as a vehicle to make quantum computing more accessible to businesses and organizations through access to the most advanced IBM Q systems and quantum ecosystem,” said Dario Gil, vice president of AI and IBM Q, IBM Research in a statement. “Working closely with our clients, together we can begin to explore the ways big and small quantum computing can address previously unsolvable problems applicable to industries such as financial services, automotive or chemistry. There will be a shared focus on discovering areas of quantum advantage that may lead to commercial, intellectual and societal benefit in the future.”

Experts from the newly formed IBM Q Consulting will be able to provide support and offer customized roadmaps to help clients become quantum-ready, says IBM.

With IBM Q, IBM seeks to be the first tech company to deliver a commercial universal quantum computing systems for and in tandem with industry and research users. Although this marks the start of its commercial network, IBM has been providing scientists, researchers, and developers with free access to IBM Q processors for over a year via the IBM Q Experience. According to the company, 60,000 registered users have collectively run more than 1.7 million experiments and generated over 35 third-party research publications.

The post IBM Launches Commercial Quantum Network with Samsung, ORNL appeared first on HPCwire.

via Government – HPCwire

[yogeshmalik]

Slovak Government Provides Capital Infusion of $17 Million to Tachyum - HPCwire https://ift.tt/2EQedtG

[naivelocus]

NVIDIA to Train 100,000 Developers on Deep Learning in 2017

SAN JOSE, Calif., May 9, 2017 — To meet surging demand for expertise in the field of AI, NVIDIA today announced that it plans to train 100,000 developers this year — a tenfold increase over 2016 — through the NVIDIA Deep Learning Institute.

Analyst firm IDC estimates that 80 percent of all applications will have an AI component by 2020. The NVIDIA Deep Learning Institute provides developers, data scientists and researchers with practical training on the use of the latest AI tools and technology.

The institute has trained developers around the world at sold-out public events and onsite training at companies such as Adobe, Alibaba and SAP; at government research institutions like the U.S. National Institute of Health, National Institute of Science and Technology, and the Barcelona Supercomputing Center; and at institutes of higher learning such as Temasek Polytechnic Singapore and India Institute of Technology, Bombay.

In addition to instructor-led workshops, developers have on-demand access to training on the latest deep learning technology, using NVIDIA software and high-performance Amazon Web Services (AWS) EC2 P2 GPU instances in the cloud. More than 10,000 developers have already been trained by NVIDIA using AWS on the applied use of deep learning.

“AI is the defining technology of our generation,” said Greg Estes, vice president of Developer Programs at NVIDIA. “To meet overwhelming demand from enterprises, government agencies and universities, we are dramatically expanding the breadth and depth of our offerings, so developers worldwide can learn how to leverage this transformative technology.”

NVIDIA is broadening the Deep Learning Institute’s curriculum to include the applied use of deep learning for self-driving cars, healthcare, web services, robotics, video analytics and financial services. Coursework is being delivered online using NVIDIA GPUs in the cloud through Amazon Web Services and Google’s Qwiklabs, as well as through instructor-led seminars, workshops and classes to reach developers across Asia, Europe and the Americas. NVIDIA currently partners with Udacity to offer Deep Learning Institute content for developing self-driving cars.

“There is a real demand for developers who not only understand artificial intelligence, but know how to apply it in commercial applications,” said Christian Plagemann, vice president of Content at Udacity. “NVIDIA is a leader in the application of deep learning technologies and we’re excited to work closely with their experts to train the next generation of artificial intelligence practitioners.”

Deep Learning Institute hands-on labs are taught by certified expert instructors from NVIDIA, partner companies and universities. Each lab covers a fundamental tenet of deep learning, such as using AI for object detection or image classification; applying AI to determine the best approach to cancer treatment; or, in the most advanced courses, using technologies such as NVIDIA DRIVE PX 2 and DriveWorks to develop autonomous vehicles.

To meet its 2017 goal, NVIDIA is expanding the Deep Learning Institute through:

New Deep Learning Training Labs: NVIDIA is working with Amazon Web Services, Facebook, Google, the Mayo Clinic, Stanford University, as well as the communities supporting major deep learning frameworks to co-develop training labs using Caffe2, MXNet and TensorFlow.

New Courseware for Educators: NVIDIA has partnered with Yann LeCun, director of AI research at Facebook and computer science professor at New York University, to develop the DLI Teaching Kit, which covers the academic theory and application of deep learning on GPUs using the PyTorch framework. Hundreds of educators are already using the DLI Teaching Kit, including the University of Oxford and the University of California, Berkeley.

New DLI Certified Training Partners: NVIDIA is expanding the Deep Learning Institute ecosystem by providing materials and certifying instructors from Hewlett Packard Enterprise, IBM and Microsoft.

NVIDIA is also working with Microsoft Azure, IBM Power and IBM Cloud teams to port lab content to their cloud solutions.

At this week’s GPU Technology Conference, in Silicon Valley, the Deep Learning Institute will offer 14 different labs and train more than 2,000 developers on the applied use of AI. View the schedule and register for a session at www.nvidia.com/dli.

Instructors can access the DLI Teaching Kits, which also cover accelerated computing and robotics, at www.developer.nvidia.com/teaching-kits.

More information on course offerings is available at NVDLI@nvidia.com.

About NVIDIA

NVIDIA‘s (NASDAQ: NVDA) invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined modern computer graphics and revolutionized parallel computing. More recently, GPU deep learning ignited modern AI — the next era of computing — with the GPU acting as the brain of computers, robots and self-driving cars that can perceive and understand the world. More information at http://nvidianews.nvidia.com/.

Source: NVIDIA

The post NVIDIA to Train 100,000 Developers on Deep Learning in 2017 appeared first on HPCwire.

— HPCwire

[mthrynn]

WASHINGTON, Dec. 7 – On Friday, December 15, the Council on Competitiveness will release its annual Clarion Call for Competitiveness. This will be Council’s 6th year publishing the document, which provides policymakers inside and outside the beltway with policy recommendations to address the most critical competitiveness challenges and opportunities.

The Council will release this year’s Clarion Call at its annual “National Competitiveness Forum.” Building on the momentum of the Council’s 30th anniversary in 2016, this year’s NCF will assemble over one hundred leaders from America’s private and public sectors to discuss, debate and inform a national competitiveness agenda.

This year’s event will focus not only on where America stands with regard to growth and prosperity, but also on where we need to go in terms of future drivers of competitiveness. The event will feature an interactive program with panels, discussions and presentations on current competitiveness issues at the Newseum.

Confirmed speakers include Sam Allen, chairman & CEO of Deere & Company and chairman of the Council on Competitiveness, Michael Crow, president of Arizona State University, George Fischer, group president of Verizon Enterprise Solutions, Nick Akins, chairman, president and CEO of American Electric Power and many more representatives from industry, academia, government and our national labs.

Want to learn more? Check out our 2017 NCF website and read last year’s 2016 Clarion Call for Competitiveness and check back on December 15 for the full 2017 Clarion Call.

Source: Council on Competitiveness

The post Council on Competitiveness to Release Annual Clarion Call for Competitiveness at 2017 National Competitiveness Forum appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

Traffic jams and mishaps are often painful and sometimes dangerous facts of life. At this week’s IEEE International Conference on Big Data being held in Boston, researchers from TACC and colleagues will present a new deep learning tool that uses raw traffic camera footage from City of Austin cameras to recognize objects – people, cars, buses, trucks, bicycles, motorcycles and traffic lights – and characterize how those objects move and interact.

The researchers from Texas Advanced Computing Center (TACC), the University of Texas Center for Transportation Research and the City of Austin have been collaborating to develop tools that allow sophisticated, searchable traffic analyses using deep learning and data mining. An account of the work (Artificial Intelligence and Supercomputers to Help Alleviate Urban Traffic Problems), written by Aaron Dubrow, was posted this week on the TACC website.

Their work is being tested in parts of Austin where cameras on signal lights automatically counted vehicles in a 10-minute video clip, and preliminary results showed that their tool was 95 percent accurate overall.

“We are hoping to develop a flexible and efficient system to aid traffic researchers and decision-makers for dynamic, real-life analysis needs,” said Weijia Xu, a research scientist who leads the Data Mining & Statistics Group at TACC. “We don’t want to build a turn-key solution for a single, specific problem. We want to explore means that may be helpful for a number of analytical needs, even those that may pop up in the future.” The algorithm they developed for traffic analysis automatically labels all potential objects from the raw data, tracks objects by comparing them with other previously recognized objects and compares the outputs from each frame to uncover relationships among the objects.

The team used the open-source YOLO library and neural network developed by University of Washington and Facebook researchers for real-time object detection. According to the team, this is the first time YOLO has been applied to traffic data. For the data analysis and query component, they incorporated HiveQL, a query language maintained by the Apache Software Foundation that lets individuals search and compare data in the system.

Once researchers had developed a system capable of labeling, tracking and analyzing traffic, they applied it to two practical examples: counting how many moving vehicles traveled down a road and identifying close encounters between vehicles and pedestrians.

“Current practice often relies on the use of expensive sensors for continuous data collection or on traffic studies that sample traffic volumes for a few days during selected time periods,” Natalia Ruiz Juri, a research associate and director of the Network Modeling Center at UT’s Center for Transportation Research. “The use of artificial intelligence to automatically generate traffic volumes from existing cameras would provide a much broader spatial and temporal coverage of the transportation network, facilitating the generation of valuable datasets to support innovative research and to understand the impact of traffic management and operation decisions.”

Whether autonomous vehicles will mitigate the problem is an ongoing debate and Juri notes, “The highly anticipated introduction of self-driving and connected cars may lead to significant changes in the behavior of vehicles and pedestrians and on the performance of roadways. Video data will play a key role in understanding such changes, and artificial intelligence may be central to enabling comprehensive large-scale studies that truly capture the impact of the new technologies.”

Link to full article: http://ift.tt/2Cbhphn

Link to video on the work: http://ift.tt/2BTGX1M

Images: TACC

The post TACC Researchers Test AI Traffic Monitoring Tool in Austin appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

Optimizing ESnet (Energy Sciences Network), the world’s fastest network for science, is an ongoing process. Recently a two-year collaboration by ESnet users – the Petascale DTN Project – achieved its ambitious goal to deliver sustained data transfers at over the target rate of 1 petabyte per week. ESnet is managed by Lawrence Berkeley National Laboratory for the Department of Energy.

During the past two years ESnet engineers have been working with staff at DOE labs to fine tune the specially configured systems called data transfer nodes (DTNs) that move data in and out of the National Energy Research Scientific Computing Center (NERSC) at LBNL and the leadership computing facilities at Argonne National Laboratory and Oak Ridge National Laboratory. A good article describing the ESnet project (ESnet’s Petascale DTN project speeds up data transfers between leading HPC centers) was posted yesterday on Phys.org.

A variety of software and hardware upgrades and expansion were required to achieve the speedup. Here are two examples taken from the article:

At NERSC, the DTN project resulted in adding eight more nodes, tripling the number, in order achieve enough internal bandwidth to meet the project’s goals. “It’s a fairly complicated thing to do,” said Damian Hazen, head of NERSC’s Storage Systems Group. “It involves adding infrastructure and tuning as we connected our border routers to internal routers to the switches connected to the DTNs. Then we needed to install the software, get rid of some bugs and tune the entire system for optimal performance.”

Oak Ridge Leadership Computing Facility now has 28 transfer nodes in production on 40-Gigabit Ethernet. The nodes are deployed under a new model—a diskless boot—which makes it easy for OLCF staff to move resources around, reallocating as needed to respond to users’ needs. “The Petascale DTN project basically helped us increase the ‘horsepower under the hood’ of network services we provide and make them more resilient,” said Jason Anderson, an HPC UNIX/storage systems administrator at OLCF. “For example, we recently moved 12TB of science data from OLCF to NCSA in less than 30 minutes. That’s fast!”

The Petascale DTN collaboration also includes the National Center for Supercomputing Applications (NCSA) at the University of Illinois in Urbana-Champaign, funded by the National Science Foundation (NSF). Together, the collaboration aims to achieve regular disk-to-disk, end-to-end transfer rates of one petabyte per week between major facilities, which translates to achievable throughput rates of about 15 Gbps on real world science data sets. The number of sites with this base capability is also expanding, with Brookhaven National Laboratory in New York now testing its transfer capabilities with encouraging results. Future plans including bringing the NSF-funded San Diego Supercomputer Center and other big data sites into the mix.

Performance measurements from November 2017 at the end of the Petascale DTN project. All of the sites met or exceed project goals. Credit: Eli Dart, ESnet

“This increase in data transfer capability benefits projects across the DOE mission science portfolio” said Eli Dart, an ESnet network engineer and leader of the project. “HPC facilities are central to many collaborations, and they are becoming more important to more scientists as data rates and volumes increase. The ability to move data in and out of HPC facilities at scale is critical to the success of an ever-growing set of projects.”

Link to full Phys.org article: http://ift.tt/2nS0ZHA

The post ESnet Now Moving More Than 1 Petabyte/wk appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

Cutting edge advanced computing hardware (aka big iron) does not stand by itself. These computers are the pinnacle of a myriad of technologies that must be carefully woven together by people to create the computational capabilities that are used to deliver insights into the behaviors of complex systems. This collection of technologies and people has been called the High Performance Computing (HPC) ecosystem. This is an appropriate metaphor because it evokes the complicated nature of the interdependent elements needed to deliver first of a kind computing systems.

The idea of the HPC ecosystem has been around for years and most recently appeared in one of the objectives for the National Strategic Computing Initiative (NSCI). The 4th objective calls for “Increasing the capacity and capability of an enduring national HPC ecosystem.” This leads to the questions of, “what makes up the HPC ecosystem” and why is it so important? Perhaps the more important question is, why does the United States need to be careful about letting its HPC ecosystem diminish?

The heart of the HPC ecosystem is clearly the “big humming boxes” that contain the advanced computing hardware. The rows upon rows of cabinets are the focal point of the electronic components, operating software, and application programs that provide the capabilities that produce the results used to create new scientific and engineering insights that are the real purpose of the HPC ecosystem. However, it is misleading to think that any one computer at any one time is sufficient to make up an ecosystem. Rather, the HPC ecosystem requires a continuous pipeline of computer hardware and software. It is that continuous flow of developing technologies that keeps HPC progressing on the cutting edge.

The hardware element of the pipeline includes systems and components that are under development, but are not currently available. This includes the basic research that will create the scientific discoveries that enable new approaches to computer designs. The ongoing demand for “cutting edge” systems is important to keep system and component designers pushing the performance envelope. The pipeline also includes the currently installed highest performance systems. These are the systems that are being tested and optimized. Every time a system like this is installed, technology surprises are found that must be identified and accommodated. The hardware pipeline also includes systems on the trailing edge. At this point, the computer hardware is quite stable and allows a focus on developing and optimizing modeling and simulation applications.

One of the greatest challenges of maintaining the HPC ecosystem is recognizing that there are significant financial commitments needed to keep the pipeline filled. There are many examples of organizations that believed that buying a single big computer would make them part of the ecosystem. In those cases, they were right, but only temporarily. Being part of the HPC ecosystem requires being committed to buying the next cutting-edge system based on the lessons learned from the last system.

Another critical element of the HPC ecosystem is software. This generally falls into two categories – software needed to operate the computer (also called middleware or the “stack”) and software that provides insights into end user questions (called applications). Middleware plays the critical role of managing the operations of the hardware systems and enabling the execution of applications software. Middleware includes computer operating systems, file systems and network controllers. This type of software also includes compilers that translate application programs into the machine language that will be executed on hardware. There are quite a number of other pieces of middleware software that include libraries of commonly needed functions, programming tools, performance monitors, and debuggers.

Applications software span a wide range and are as varied as the problems users want to address through computation. Some applications are quick “throwaway” (prototype) attempts to explore potential ways in which computers may be used to address a problem. Other applications software is written, sometimes with different solution methods, to simulate physical behaviors of complex systems. This software will sometimes last for decades and will be progressively improved. An important aspect of these types of applications is the experimental validation data that provide confidence that the results can be trusted. For this type of applications software, setting up the problem that can include finite element mesh generation, populating that mesh with material properties and launching the execution are important parts of the ecosystem. Other elements of usability of application software include the computers, software, and displays that allow users to visualize and explore simulation results.

Data is yet another essential element of the HPC ecosystem. Data is the lifeblood in the circulatory system that flows through the system to keep it doing useful things. The HPC ecosystem includes systems that hold and move data from one element to another. Hardware aspects of the data system include memory, storage devices, and networking. Also software device drivers and file systems are needed to keep track of the data. With the growing trend to add machine learning and artificial intelligence to the HPC ecosystem, its ability to process and productively use data are becoming increasingly significant.

Finally, and most importantly, trained and highly skilled people are an essential part of the HPC ecosystem. Just like computing systems, these people make up a “pipeline” that starts in elementary school and continues through undergraduate and then advanced degrees. Attracting and educating these people in computing technologies is critical. Another important part of the people pipeline of the HPC ecosystem are the jobs offered by academia, national labs, government, and industry. These professional experiences provide the opportunities needed to practice and hone HPC skills.

The origins of the United States’ HPC ecosystem dates back to the decision by the U.S. Army Research Lab to procure an electronic computer to calculate ballistic tables for its artillery during World War II (i.e. ENIAC). That event led to finding and training the people, who in many cases were women, to program and operate the computer. The ENIAC was just the start of the nation’s significant investment in hardware, middleware software, and applications. However, just because the United States was the first does not mean that it was alone. Europe and Japan also have robust HPC ecosystems for years and most recently China has determinedly set out to create one of their own.

The United States and other countries made the necessary investments in their HPC ecosystems because they understood the strategic advantages that staying at the cutting edge of computing provides. These well-document advantages apply to many areas that include: national security, discovery science, economic competitiveness, energy security and curing diseases.

The challenge of maintaining the HPC ecosystem is that, just like a natural ecosystem, the HPC version can be threatened by becoming too narrow and lacking diversity. This applies to the hardware, middleware, and applications software. Betting on just a few types of technologies can be disastrous if one approach fails. Diversity also means having and using a healthy range of systems that covers the highest performance cutting edge systems to wide deployment of mid and low-end production systems. Another aspect of diversity is the range of applications that can productively use on advanced computing resources.

Perhaps the greatest challenge to an ecosystem is complacency and assuming that it, and the necessary people, will always be there. This can take the form of an attitude that it is good enough to become a HPC technology follower and acceptable to purchase HPC systems and services from other nations. Once a HPC ecosystem has been lost, it is not clear if it can be regained. Having a robust HPC ecosystem can last for decades, through many “half lives” of hardware. A healthy ecosystem allows puts countries in a leadership position and this means the ability to influence HPC technologies in ways that best serve their strategic goals. Happily, the 4th NSCI objective signals that the United States understands these challenges and the importance of maintaining a healthy HPC ecosystem.

About the Author

Alex Larzelere is a senior fellow at the U.S. Council on Competitiveness, the president of Larzelere & Associates Consulting and HPCwire’s policy editor. He is currently a technologist, speaker and author on a number of disruptive technologies that include: advanced modeling and simulation; high performance computing; artificial intelligence; the Internet of Things; and additive manufacturing. Alex’s career has included time in federal service (working closely with DOE national labs), private industry, and as founder of a small business. Throughout that time, he led programs that implemented the use of cutting edge advanced computing technologies to enable high resolution, multi-physics simulations of complex physical systems. Alex is the author of “Delivering Insight: The History of the Accelerated Strategic Computing Initiative (ASCI).”

The post HPC Iron, Soft, Data, People – It Takes an Ecosystem! appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

AMD’s return to the data center received a boost today when Microsoft Azure announced introduction of instances based on AMD’s EPYC microprocessors. The new instances – Lv2-Series of Virtual Machine – use the EPYC 7551 processor. Adoption of EPYC by a major cloud provider adds weight to AMD’s argument that it has returned to the data center with a long-term commitment and product roadmap. AMD had been absent from that segment for a number of years.

Writing in a blog, Corey Sanders director of compute, Azure, said, “We’ve worked closely with AMD to develop the next generation of storage optimized VMs called Lv2-Series, powered by AMD’s EPYC processors. The Lv2-Series is designed to support customers with demanding workloads like MongoDB, Cassandra, and Cloudera that are storage intensive and demand high levels of I/O.” The EPYC line was launched last June (see HPCwire article, AMD Charges Back into the Datacenter and HPC Workflows with EPYC Processor.)

The instances make use of Microsoft’s Project Olympus intended to deliver a next generation open source cloud hardware design developed with the Open Compute Community (OCP). “We think Project Olympus will be the basis for future innovation between Microsoft and AMD, and we look forward to adding more instance types in the future benefiting from the core density, memory bandwidth and I/O capabilities of AMD EPYC processors,” said Sanders, quoted in the AMD’s announcement of the new instances.

It is an important win for AMD. Gaining a foothold in the X86 landscape today probably requires adoption by hyperscalers. No doubt some “tire kicking” is going on here but use of an Olympus design adds incentive for Microsoft Azure to court customers for the instances. HPE has also announced servers using the EPYC line.

AMD EPYC chip lineup at the June launch

The Lv2-Series instances run on the AMD EPYC 7551 processor featuring a base core frequency of 2.2 GHz and a maximum single-core turbo frequency of 3.0 GHz. “With support for 128 lanes of PCIe connections per processor, AMD provides over 33 percent more connectivity than available two-socket solutions to address an unprecedented number of NVMe drives directly,” says AMD.

The Lv2 VMs will be available starting at eight and ranging to 64 vCPU sizes, with the largest size featuring direct access to 4TB of memory. These sizes will support Azure premium storage disks by default and will also support accelerated networking capabilities for the highest throughput of any cloud.

Scott Aylor, AMD corporate vice president and general manager of Enterprise Solutions said, “There is tremendous opportunity for users to tap into the capabilities we can deliver across storage and other workloads through the combination of AMD EPYC processors on Azure. We look forward to the continued close collaboration with Microsoft Azure on future instances throughout 2018.”

Link to AMD release: http://ift.tt/2jTYZtp

Link to Azure blog: http://ift.tt/2jScGsF

The post Azure Debuts AMD EPYC Instances for Storage Optimized Workloads appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

Last August, cloud giant Microsoft acquired HPC cloud orchestration pioneer Cycle Computing. Since then the focus has been on integrating Cycle’s organization, mapping out its new role as a core Microsoft Azure product, and deciding what to do with those Cycle customers who currently use non-Azure cloud providers. At SC17, HPCwire caught up with Brett Tanzer, head of Microsoft Azure Specialized Compute Group (ASCG, which used to be Big Compute) in which Cycle now lives, and Tim Carroll, formerly Cycle VP of sales and ecosystem development and now a ‘principal’ in ASCG, for a snapshot of emerging plans for Cycle.

Much has already been accomplished they emphasize – for starters “the Cycle organization has settled in” and most are relocating to Seattle. Much also remains to be done – it will probably be a year or so before Cycle is deeply integrated across Azure’s extensive capabilities. In some ways, it’s best not to think of the Cycle acquisition in isolation but as part of Microsoft’s aggressively evolving strategy to make Azure all things for all users and that includes the HPC community writ large. Cycle is just one of the latest, and a significant, piece of the puzzle.

Founded in 2005 by Jason Stowe and Rob Futrick, Cycle Computing was one of the first companies to target HPC orchestration in the cloud; its software, CycleCloud, enables users to burst and manage HPC workloads (and data) into the cloud. Till now, cloud provider agnosticism has been a key Cycle value proposition. That will change but how quickly is uncertain. Tanzer assures there will be no disruption of existing Cycle customers, but also emphasizes Microsoft intends Cycle to become an Azure-only product over time. Cycle founder Stowe has taken on a new role as a solutions architect in the Specialized Compute Group. The financial details of the Cycle acquisition weren’t made public.

Far more than in the past HPC is seen as an important opportunity for the big cloud providers. The eruption of demand for running AI and deep learning workflows has also been a major driving force for cloud providers.

Nvidia V100 GPU

Microsoft, like Google and Amazon (and others), has been investing heavily in advanced scale technology. The immediate goal is to attract HPC and AI/deep learning customers. One indicator is the way they have all been loading up on GPUs. Azure is no exception and offers a growing list of GPU instances (M60, K80, P100, P40, and V100 (announced)); it also offers InfiniBand high speed interconnect. In October, Microsoft extended its high performance gambit further via a partnership with Cray to offer supercomputing in the cloud (see HPCwire article, Cray+Azure: Can Cloud Propel Supercomputing?).

How the latter bet will play out is unclear – Tanzer says, “We are hearing from customers there are some workloads they need to get into the cloud that require a Cray. And Cray itself is a pretty innovative company. We think the partnership has longer legs. Look for more to come.” One wonders what interesting offerings may sprout from that alliance.

For now the plan for Cycle is ever deeper integration with Azure’s many offerings, perhaps eventually including Cray. It’s still early days, of course. Tanzer says, “If Tim looks like he hasn’t slept much for past three months, it’s because he hasn’t.  Strategically, all of these products – Cycle, Azure Batch, HPC pack (cluster tool) – will work together and drive orchestration across all the key workloads.”

“The company is rallying behind the [HPC] category and customers are responding very well,” says Tanzer. “We are investing in all phases of the maturity curve, so if you are somebody who wants a Cray, we now have an answer for you. If you are rewriting your petrochemical workload and want to make it cloud friendly, then Batch is a great solution. We are really just taking care, wherever we can, to take friction out of using the cloud. We looked at Cycle and its fantastic people and knowledge. The relationship with Cycle is very symbiotic. We look at where our customers are and see [that for many], Cycle helps them bootstrap the process.”

It’s not hard to see why Cycle was an attractive target. Cycle brings extensive understanding of HPC workloads, key customer and ISV relationships, and a robust product. Recently it’s been working to build closer relationships with systems builders (e.g. Dell EMC) and HPC ISVs (e.g. ANSYS). From an operations and support perspective, not much has changed for Cycle customers, says Carroll, although he emphasizes having now gained access to Microsoft’s deep bench of resources. No decision has been made on name changes and Tanzer says, “Cycle is actually a pretty good name.”

Cycle’s new home, the Azure’s Specialized Compute Group seems to be a new organization encompassing what was previously Big Compute. As of this writing, there was still no Specialized Compute Group web page, but from the tone of Tanzer and Carroll it seemed that things could still be in flux. SCG seems to have a fairly broad mission to smooth the path to cloud computing across all segments with so-called “specialized needs” – that, of course, includes HPC but also crosses over into enterprise computing as well. To a significant extent, says Tanzer, it is part of Microsoft’s company-wide mantra to meet-the-customer-where-she/he-is to minimize disruption.

“Quite frankly we are finding customers, even in the HPC space, need a lot of help and it’s also an area where Microsoft has a many differentiated offerings,” Tanzer says. “You should expect us to integrate Cycle’s capabilities more natively into Azure. There is much more that can be done in the category to help customers take advantage of the cloud, from providing best practices about how your workloads move, through governance, and more. Cloud consumption is getting more sophisticated and it’s going require tools to help users maximize their efforts even though the usage models will be very different.”

One can imagine many expanded uses for Cycle functionality, not least close integration with an HPC applications and closer collaboration with ISVs to drive adoption. Microsoft has the clout and understanding of both software and infrastructure businesses to help drive that, says Carroll. “Those two things are important because this is a space that’s always struggled to figure out how to build partnerships between the infrastructure providers and software providers; Microsoft’s ability to talk to some of the significant and important ISVs and figure out ways to work with them from a Microsoft perspective is a huge benefit.”

It probably bears repeating that Tanzer’s expectations seem much broader than HPC or Cycle’s role as an enabler. He says rather matter of factly, “Customers are recognizing the cloud is the destination and thinking in more detail about that. It will be interesting to see how that plays out.” When he says customers, one gets the sense he is talking about more than just a narrow slice of the pie.

The conversation over how best to migrate and perform HPC has a long history. Today, there seems less debate about whether it can be done effectively but more around how to do it right, how much it costs, and what types of HPC jobs are best suited for being run in the cloud. Carroll has for some time argued that technology is not the issue for the most potential HPC cloud users.

Tim Carroll

“It’s less about whether somebody is technically ready than whether they have a business model that requires them to be able to move faster and leverage more compute than they had thought they were going to need,” says Carroll. “Where we see the most growth is [among users] who have deadlines and at the end of the day what they really care about is how long will it take me to get my answer and tell me the cost and complexity to get there. That’s a different conversation than we have had in this particular segment over time.”

Some customer retraining and attitude change will be necessary, says Tanzer.

“They are going to have hybrid environments for a while so to the degree we can help them reduce some of the chaos that comes from that and help retrain the workforce easily on what it needs to take advantage of the cloud. We think that’s important. Workforces who run the workloads really understand all they want to do is to take advantage of the technology but some relearning is necessary and that’s another area where Cycle really helps because of its tools and set of APIs and they speak the language of a developer,” he says.

Cycle connections in the academic space will also be beneficial according to Tanzer. There are both structural and financial obstacles for academicians who wish to run HPC workloads in commercial cloud and Cycle insight will help Azure navigate that landscape to the benefit of Azure and academic users, he suggests. The Cray deal will help in government markets, he says.

Stay tuned.

The post Microsoft Spins Cycle Computing into Core Azure Product appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

Nov. 29, 2017 — The 9th Irish Supercomputer List was released today featuring two new world-class supercomputers. This is the first time that Ireland has four computers ranked on the Top500 list of the fastest supercomputers on Earth. Ireland is now ranked number one globally in terms of number of Top-500 supercomputers per capita (stats here). In terms of performance per capita, Ireland is ranked 4th globally (stats here). These new supercomputers boost the Irish High Performance Computing capacity by nearly one third, up from 3.01 to 4.42 Pflop/s. Ireland has ranked on the Top500 list 33 times over a history of 23 years with a total of 20 supercomputers (full history here). Over half of these rankings (19) and supercomputers (12) have been in the last 6 years, representing Ireland’s increasing pace of High Performance Computing investment. The new entrants, from two undisclosed software and web services companies, feature at spots 423 and 454 on the 50th Top500 Supercomputer List, with Linpack Rmax scores of 635 and 603 TFlop/s respectively.

Not considering Ireland’s admittedly low population (which does help the above rankings), Ireland still ranks admiraly. In terms of Top500 installations, Ireland ranks 9th place globally, tied with Australia, India and Saudi Arabia, and 18th in the world in terms of supercomputing performance.

The Irish Supercomputer List now ranks 30 machines (2 new, 1 upgraded), with a total of more than 207,000 CPU cores and over 106,000 accelerator cores.

Source: Irish Supercomputer List

The post Ireland Ranks Number One in Top500 Systems per Capita appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

We live on a planet of more than seven billion people who speak more than 7,000 languages. Most of these are “low-resource” languages for which there are a dearth of human translators and no automated translation capability. This presents a big challenge in emergency situations where information must be collected and communicated rapidly across languages.

To address this problem, linguists at Ohio State University are using the Ohio Supercomputer Center’s Owens Cluster to develop a general grammar acquisition technology.

This graph displays an algorithm that explores the space of possible probabilistic grammars and maps out the regions of this space that have the highest probability of generating understandable sentences. (Source: OSC)

The research is part of an initiative called Low Resource Languages for Emergent Incidents (LORELEI) that is funded through the Defense Advanced Research Projects Agency (DARPA). LORELEI aims to support emergent missions, e.g., humanitarian assistance/disaster relief, peacekeeping or infectious disease response by “providing situational awareness by identifying elements of information in foreign language and English sources, such as topics, names, events, sentiment and relationships.”

The Ohio State group is using high-performance computing and Bayseian methods to develop a grammar acquisition algorithm that can discover the rules of lesser-known languages.

“We need to get resources to direct disaster relief and part of that is translating news text, knowing names of cities, what’s happening in those areas,” said William Schuler, Ph.D., a linguistics professor at The Ohio State University, who is leading the project. “It’s figuring out what has happened rapidly, and that can involve automatically processing incident language.”

Schuler’s team is using Bayseian methods to discover a given language’s grammar and build a model capable of generating grammatically valid output.

“The computational requirements for learning grammar from statistics are tremendous, which is why we need a supercomputer,” Schuler said. “And it seems to be yielding positive results, which is exciting.”

The team originally used CPU-only servers but is now using GPUs in order to model a larger number of grammar categories. The goal is to have a model that can be trained on a target language in an emergency response situation, so speed is critical. In August, the team ran two simulated disaster simulations in seven days using 60 GPU nodes (one Nvidia P100 GPU per node) but a real-world situation with more realistic configurations would demand even greater computational power, according to one of the researchers.

Read the full announcement here: http://ift.tt/2zfoknO

Owens Cluster technical specs here: http://ift.tt/2AnuDcr

The post Linguists Use HPC to Develop Emergency-Response Translator appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

Nov. 20, 2017 — Croatia is the 13th country to sign the European declaration on high-performance computing (HPC). Blaženka Divjak, Croatian Minister of Science and Education signed the declaration today in Brussels in the presence of Roberto Viola, Director-General for Communications Networks, Content and Technology at the European Commission.

Vice-President Ansip, responsible for the Digital Single Market, and Mariya Gabriel, Commissioner for Digital Economy and Society welcomed this important step for EuroHPC: “We are pleased to welcome Croatia in this bold European project. By aligning our European and national strategies and pooling resources, we will put Europe in leading global position in HPC and provide access to world-class supercomputing resources for both public and private users, especially for SMEs, who use more and more HPC in their business processes. The scientific and industrial developments will have a direct positive impact on European citizens’ daily lives in areas going from biotechnological and biomedical research to personalised medicine, and from renewable energy to urban development.”

Blaženka Divjak, Croatian Minister of Science and Education added: “Republic of Croatia recognizes the need for EU integrated world-class high performance computing infrastructure which in combination with EU data and network infrastructures would upraise both Europe’s and Croatian scientific capabilities and industrial competitiveness. Therefore, we are very pleased that Croatia is now part of this ambitious European project. It is widely agreed that scientific progress as well as economic growth will increasingly rely on top level HPC-enabled methods and tools, services and products. Signing this Declaration is a step in the right direction for our country which will help Croatia to further develop our research and industrial potential. Europe needs to combine resources to overcome its fragmentation and the dilution of efforts.”

The goal of the EuroHPC agreement is to establish a competitive HPC ecosystem by acquiring and operating leading-edge high-performance computers . The ecosystem will comprise hardware and software components, applications, skills and services. It will be underpinned by a world-class HPC and data infrastructure HPC infrastructure, available across EU, no matter where supercomputers are located. This HPC infrastructure will also support the European Open Science Cloud and will allow millions of researchers to share and analyse data in a trusted environment. Focusing initially on the scientific community, the user base of the cloud will over time be enlarged to a wide range of users: scientific communities, large industry and SMEs, as well as the public sector.

The EuroHPC declaration aims at having EU exascale supercomputers, capable of at least 1018 calculations per second, in the global top three by 2022-2023.

The EuroHPC initiative was launched during the Digital Day in March 2017 and signed by France, Germany, Italy, Luxembourg, the Netherlands, Portugal and Spain (see the press statement, speech and blog post by Vice-President Ansip). Five other countries have since joined this bold European initiative: Belgiumin June, Slovenia in July, Bulgaria and Switzerland in October and Greece in November.

Why HPC matters

Supercomputers are very powerful systems with hundreds of thousands or millions of processors working in parallel to analyse billions of pieces of data in real time. They do extremely demanding computations for simulating and modelling scientific and engineering problems that cannot be performed using general-purpose computers. Therefore, access to HPC becomes essential in many areas spanning from health, biology and climate change to automotive, aerospace energy and banking.

Moreover, as the problems we want to solve are more and more complex, the demands on computational resources are growing accordingly. In this rhythm, today’s state of the art machines are obsolete after 5-7 years of operation.

Aiming at and developing a European HPC ecosystem will benefit both academia and industry. As a wide range of scientific and industrial applications will be made available at EU level, citizens will benefit from an increased level of HPC resources in areas like:

Health, demographic change and wellbeing

Secure, clean and efficient energy

Smart, green and integrated urban planning

Cybersecurity

Weather forecasting and climate change

Food security

More examples in the HPC factsheet.

Next steps

The European Commission, together with countries which have signed the declaration are preparing, by the end of 2017, a roadmap with implementation milestones to deploy the European exascale supercomputing infrastructure.

All other Member States and countries associated to Horizon 2020 are encouraged to join EuroHPC and work together, and with the European Commission, in this initiative.

Source: European Commission

The post Croatia Signs the European Declaration on High-Performance Computing appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

Sierra, the 125 petaflops machine based on IBM’s Power9 chip and being built at Lawrence Livermore National Laboratory, sometimes takes a back seat to Summit, the 180 petaflops system being built at Oak Ridge National Laboratory and expected to perhaps top the Top500 list in June. Like Sierra, Summit features a heterogeneous architecture based on Power9 and Nvidia GPUs.

Livermore today posted a brief update on Sierra’s progress along with a short video. Trucks began delivering racks and hardware over the summer with system acceptance scheduled in fiscal 2018. Sierra, part of the CORAL effort, is expected to provide four to six times the sustained performance of the Lab’s current workhorse system, Sequoia.

“Sierra is what we call an advanced technology platform,” says Mike McCoy, Program Director, Advanced Simulation and Computing, in the video. “[It] will serve the three NNSA (National Nuclear Security Administration) laboratories. So the ATS2, which is Sierra, is the second in a series of four systems that are on a roadmap to get us to exascale computing [around] 2024.”

Sierra is expected to have roughly 260 racks and will be the biggest computer installed at Livermore in size, number of racks, and speed.

“IBM analyzed our benchmark applications, showed us how the system would perform well for them, and how we would be able to achieve similar performance for our real applications,” said Bronis de Supinski, Livermore Computing’s chief technology officer and head of Livermore Lab’s Advanced Technology (AT) systems, in the article. “Another factor was that we had a high probability, given our estimates of the risks associated with that proposal, of meeting our scheduling requirements.”

While Lab scientists have positive indications from their early access systems, de Supinski said until Sierra is on the floor and running stockpile stewardship program applications, which could take up to two years, they won’t be certain how powerful the machine will be or how well it will work for them.

Sierra will feature two IBM Power 9 processors and 4 NVIDIA Volta GPUs per node. The Power 9s will provide a large amount of memory bandwidth from the chips to Sierra’s DDR4 main memory, and the Lab’s workload will benefit from the use of second-generation NVLINK, forming a high-speed connection between the CPUs and GPUs.

As Livermore’s first extreme-scale CPU/GPU system, Sierra has presented challenges to Lab computer scientists in porting codes, identifying what data to make available on GPUs and moving data between the GPUs and CPUs to optimize the machine’s capability. Through the Sierra Center of Excellence, Livermore Lab code developers and computer scientists have been collaborating with on-site IBM and NIVIDIA employees to port applications.

Feature Image: Sierra, LLNL

The post Installation of Sierra Supercomputer Steams Along at LLNL appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

Thus week’s SC17 keynote – Life, the Universe and Computing: The Story of the SKA Telescope – was a powerful pitch for the potential of Big Science projects that also showcased the foundational role of high performance computing in modern science. It was also visually stunning as images of stars and galaxies and tiny telescopes and giant telescopes streamed across the high definition screen extended the length of Colorado Convention Center ballroom’s stage. One was reminded of astronomer Carl Sagan narrating the Cosmos TV series.

SKA, you may know, is the Square Kilometre Array project being run by an international consortium and intended to build the largest radio telescope in the world; it be 50 times more powerful than any other radio telescope today. The largest today is  ALMA (Atacama Large Millimeter/submillimeter Array) located in Chile and has 66 dishes.

SKA will be sited in two locations, South Africa, and Australia. The two keynoters Philip Diamond, Director General of SKA, and Rosie Bolton, SKA Regional Centre Project Scientist and Project Scientist for the international engineering consortium designing the high performance computers, took turns outlining radio astronomy history and SKA ambition to build on that. Theirs was a swiftly-moving talk, both entertaining and informative. The visuals flashing adding to the impact.

Their core message: This massive new telescope (I guess you could say two telescopes) will open a new window on astrophysical phenomena and create a mountain of data for scientists to work on for years. SKA, say Diamond and Bolton, will help clarify the early evolution of the universe, be able to detect gravitational waves by their effect on pulsars, shed light on dark matter, produce insight around cosmic magnetism, create detailed, accurate 3D maps of galaxies, and much more. It could even play a SETI like role in the search for extraterrestrial intelligence.

“When fully deployed, SKA will be able to detect TV signals, if they exist, from the nearest tens maybe 100 stars and will be able to detect the airport radars across the entire galaxy,” said Diamond, in response to a question. SKA is creating a new government organization to run the observatory, “something like CERN or the European Space Agency, and [we] are now very close to having this process finalized,” said Diamond.

Indeed this is exciting stuff. It is also incredibly computationally intensive. Think about an army of dish arrays and antennas, capturing signals 24×7, moving them over high speed networks to one of two digital “signal processing facilities”, one for each location, and then on to two ‘science data processors” centers (think big computers). And let’s not forget data must be made available to scientists around the world.

Consider just a few data points, shown below, that were flashed across stage during the keynote presentation. The context will become clearer later.

It’s a grand vision and there’s still a long way to go. SKA, like all Big Science projects, won’t happen overnight. SKA was first conceived in 90s at the International Union of Radio Science (URSI) which established the Large Telescope Working Group to begin a worldwide effort to develop the scientific goals and technical specifications for a next generation radio observatory. The idea arose to create a “hydrogen array” able to detect H radiofrequency emission (~1420 MHz). A square kilometer was required to have a large enough collection area to see back into the early universe. In 2011 those efforts consolidated in a not-for-project company that now has ten member countries (link to brief history of SKA). The U.S. which did participate in early SKA efforts chose not to join the consortium at the time.

Although first conceived as a hydrogen array, Diamond emphasized, “With a telescope of that size you can study many things. Even in its early stages SKA will be able to map galaxies early in the universe evolution. When full deployed it will conduct fullest galaxy mapping in 3D encompassing up to one million individual galaxies and cover 12.5 billon years of cosmic history.”

A two-phase deployment is planned. “We’re heading full steam towards critical design reviews next year,” said Diamond. Full construction starts in two years with construction of the first phase expected to begin in 2019. So far €200 have been committed for design along with “a large fraction” of the €640 required for first phase construction. Clearly there are technology and funding hurdles ahead. Diamond quipped if the U.S. were to join SKA and pony up, say $2 billion, they would ‘fix’ the spelling of kilometre to kilometer.

There will actually be two telescopes, one in South Africa about 600 km north of Cape Town and another one roughly 800 km north of Perth in western Australia. They are being located in remote regions to reduce radiofrequency interference from human activities.

“In South Africa we are going to be building close to 200 dishes, 15 meters in diameter, and the dishes will be spread over 150 km. They [will operate] over a frequency range of 350 MHz to 14 GHz. In Australia we will build 512 clusters, each of 256 antennas. That means a total of over 130,000 2-meter tall antennas, spread over 65 km. these low frequency antennas will be tapered with periodic dipoles and will cover the frequency range 50 to 350MHz. It is this array that will be the time machine that observes hydrogen all the way back to the dawn of the universe.”

Pretty cool stuff. Converting those signals is a mammoth task. SKA plans two different types of processing center for each location. “The radio waves induce voltages in the receivers that capture them and modern technology allows us to digitize them to high precision than ever before. From there optical fibers transmit the digital data from the telescopes to what we call central processing facilities or (CPFs). There’s one for each telescope,” said Bolton.

Using a variety of technologies including “some exciting FPGA, CPU-GU, and hybrids”, CPFs are where the signals are combined. Great care must be taken to first synchronize the data so it enters the processing chain exactly when it should to account for the fact the radio waves from space reached one antenna before reaching another. “We need to correct that phase offset down to the nanosecond,” said Bolton.

Once that’s done a Fourier transform is applied to the data. “It decomposes essentially a function of time into the frequencies that make it up; it moves us into the frequency domain. We do this with such precision that the SKA will be able to process 65000 different radio frequencies simultaneously,” said Diamond

Once the signals have been separated in frequencies they processed one of two ways. “We can either stack the signals together of various antenna in what we call a time domain data. Each stacking operation corresponds to a different direction in the sky. We’ll be able to look at 2000 such directions simultaneously. This time domain processing analysis detects repeating objects such as pulsars or one off events like gamma ray explosions. If we do find an event, we are planning to store the raw voltage signals at the antennae for a few minutes so we can go back in time and investigate them to see what happened,” said Bolton.

This time domain data can be used by researchers to measure pulsar – which are a bit like cosmic lighthouses – signal arrival times accurately and detect the drift if there is one as a gravitational wave passes through.

“We can also use these radio signals to make images of the sky. To do that we take the signals from each pair if antennas, each baseline, and effectively multiply them together generating data objects we call visibilities. Imagine it will be done for 200 dishes and 512 groups of antennas, that’s 150,000 baselines ad 65000 different frequencies. That makes up to 10B different data streams. Doing this is a data intensive process that requires around 50 petaflops of dedicated digital signal processing.

Signals are processed inside these central processing facilities in a way that depends on the science that “we want to do with them. Once processed the data are then sent via more fiber optic cables to the Science Data Processors or SDPs. Two of these “great supercomputers” are planned, one in Cape Town for the dish array and one in Perth for low frequency antennas.

“We have two flavors of data within the science processor. In the time domain we’ll do panning for astrophysical gold, searching over 1.5M candidate objects every ten minutes sniffing out the real astrophysical phenomena such as pulsar signals or flashes of radio light,” said Diamond. The expectation is for a 10,000 to 1 negative-to-positive events. Machine learning will play a key role in finding the “gold”.

Making sense of the 10 billion incoming visibility data streams poses the greatest computational burden, emphasized Bolton: “This is really hard because inside the visibilities (data objects) of the sky and antenna responses are all jumbled. We need to do another massive Fourier transform to get from the visibility space that depends on the antenna separations to sky planes. Ultimately we need to develop self-consistent models not only of the sky that generated the signals but also how each antenna was behaving and even how the atmosphere was changing during the data gathering.

“We can’t do that in one fell swoop. Instead we’ll have several iterations trying to find the calibration parameters and source positions of brightnesses.” With each iteration bit by bit, fainter and fainter signal emerge from the noise. “Every time we do another iteration we apply different calibration techniques and we improve a lot of them but we can’t be sure when this process is going to converge so it is going to be difficult,” said Bolton.

A typical SKA map, she said, will probably contain hundreds of thousands of radio array sources. The incoming images are about 10 petabytes in size. Output 3D images are 5000 pixels on each axis and 1 petabyte in size.

Distributing this data to scientists for analysis is another huge challenge. The plan is to distribute data via fiber to SKA regional centers. “This another real game changer that the SKA, CERN, and a few other facilities are bringing about. Scientists will use the computing power of the SKA regional centers to analyze these data products,” said Diamond.

The keynote was a wowing, multimedia presentation, and warmly received by attendees. It bears repeating that many issues remain and schedules have slipped slightly, but it is still a stellar example of Big Science, requiring massively coordinated international efforts, and underpinned with enormous computing resources. Such collaboration is well aligned with SC17’s theme – HPC Connects.

Link to video recording of the presentation: https://www.youtube.com/watch?time_continue=2522&v=VceKNiRxDBc

The post SC17 Keynote – HPC Powers SKA Efforts to Peer Deep into the Cosmos appeared first on HPCwire.

via Government – HPCwire

[mthrynn]

OpenACC, the directive-based parallel programming model used mostly for porting codes to GPUs for use on heterogeneous systems, came to SC17 touting impressive speedups of climate codes as well as its latest release, version 2.6, which adds deep copy functionality but otherwise minor enhancements. While perhaps not startlingly newsworthy the progress reported demonstrates OpenACC’s steady and impactful penetration of the HPC landscape.

Three of the top five HPC applications[i] (ANSYS Fluent, Gaussian, and VASP) now support and use OpenACC for host-based parallelism as well as for accelerator based parallelism, and on the order of 85 applications have been “accelerated” at hackathons since 2014. The work on climate codes – conducted over a few years – includes recent successful efforts by a 2017 Gordon Bell finalist to speed up the widely used Community Atmosphere Model (CAM) from The National Center for Atmospheric Research (NCAR) on China’s Sunway TaihuLight Supercomputer.

“OpenACC allowed us to scale the CAM-SE to over 1.8 million Sunway cores with a simulation speed of over 3 simulated years per day,” said Haohuan Fu, deputy director of the National Supercomputing Center in Wuxi NAD in OpenACC’s SC17 announcement. “Without OpenACC, the team would have spent years coding for the complexity of the components in the TaihuLight Supercomputer.”

It seems increasingly clear that OpenACC is finding its place in the HPC toolbox. The spotlighting of work done with climate and weather forecasting codes is one example.

Code speedup in climate research is important because of the time scales tackled. Stan Posey, program manager, earth system modeling solution development at NVIDIA, told HPCwire, “Climate research might model decades of climate behavior and so you want to achieve as many simulated years per day as you can. I believe at the Department of Energy they have gotten to about 1.5 simulated years per day which is about the best I have seen until this announcement.”

He noted the work on Sunway “is not related to GPUs; it’s just strictly the Chinese indigenous system but they used OpenACC because they don’t have the kinds of tools and compilers and so forth. It’s such a new design,” according to Posey.

It’s worth pointing out that most climate models have two distinct parts, according to Posey, “One half to solve what’s called the dynamics and one half to solve the physics. The dynamics covers the transport or the CFD that’s required in these models. Then we have the physics which handles processes such as turbulence and so on.” The two tasks have somewhat different computational requirements.

“The [physics] have a higher opportunity for performance gains because they operate not necessarily embarrassingly parallel but they operate in individual vertical columns so you can process many of these at once, which is what GPUs really like. Meanwhile the dynamics part of the model is about 50 percent of the total profile and that’s nearly always going to be limited to the 2X-3X range improvement,” he said. The latter limits potential performance gains for the overall model. Speedups are achieved by attacking amenable portions of the code.

OpenACC has been proving its effectiveness in climate models. NCAR, for example, developed an atmospheric model called Model for Prediction Across Scales (MPAS). In collaboration with the University of Wyoming, and the Korea Institute of Science and Technology Information (KISTI), the team evaluated various platforms and programming models, and implemented the MPAS with OpenACC directives to take advantage of fine grain parallel processing on GPUs. Scientists at NCAR and University of Wyoming developed OpenACC code for the MPAS dynamical core and scientists at KISTI developed OpenACC code for the MPAS physics in a project that will implement the full MPAS model on GPUs.

“Our team has been investigating OpenACC as a pathway to higher performance, and with performance portability for the MPAS global atmospheric model,” said Rich Loft, director of technology development at the NCAR Computational Information and Systems Laboratory in the release. “Using this approach for the MPAS dynamical core, we have achieved speedups on a single P100 GPU equivalent to nearly 3 CPU nodes on our new Cheyenne supercomputer.”

Other climate and weather forecasting work cited by OpenACC included:

COSMO. Scientists at MeteoSwiss, ETH Zurich, and the Swiss National Supercomputing Center (CSCS) have developed the physics of the COSMO regional atmospheric model in OpenACC to deploy on GPUs for use in operational numerical weather prediction, and climate research.

IFS. Collaboration with ECMWF in the ESCAPE Project has demonstrated speedups of more than an order of magnitude when optimizing existing GPU code for the spectral transform operations in the IFS Details were presented at the ESCAPE Workshop during Sep 2017.

NICAM. Computational scientists at RIKEN have achieved GPU factors speedup over CPU-only, for the NICAM-DC (dynamical core) project using OpenACC on more than 1000 GPU nodes of the TSUBAME 2.5 supercomputer at the Tokyo Institute of Technology.

The paper “Parallelization and Performance of the NIM Weather Model on CPU, GPU, and MIC Processors,” published in the Oct 2017 edition of BAMS, describes the OpenACC and GPU developments by scientists at the NOAA Earth System Research Laboratory to achieve performance portability with the same FORTRAN code compiled across various types of HPC platforms.

OpenACC also presented favorable benchmarks for speeding up ANSYS Fluent, Guassian, and VASP (see slide below) with significant speedups.

Talking about the VASP numbers, Guido Juckeland, head of the computational science group at Helmholtz-Center Dresden-Rossendorf (HZDR), Germany and OpenACC Secretary, said “The interesting thing here is the OpenACC port covers more VASP routines than the CUDA version and their latest OpenACC version actually outperforms their CUDA version which might be surprising. Their (VASP) CUDA code only parallelizes small kernels or small sections within the models. With OpenACC they were actually able to parallelize whose modules so that whole work is pushed over to the accelerator and this give you the overall speedup.”

“On the right side of the slider are the CAAR codes (Center for Accelerated Application Readiness) for CORAL systems, and five of those also use OpenACC to use the accelerator but also to express parallelism potentially for the CPU side.” CARR is a collaborative effort of application development teams and staff from the OLCF Scientific Computing group, CAAR is focused on redesigning, porting, and optimizing application codes for Summit’s hybrid CPU–GPU architecture.

There wasn’t much change in membership. Michigan State University has completed the process and Juckeland said another organization would soon complete the process. He also emphasized OpenACC’s commitment supporting many platforms. Arm is still not supported. AMD was a late addition to the SC17 briefing slide.

[i] Top five HPC applications according to Intersect360 Research: 1 Gromacs, 2 ANSYS Fluent, 3 Gaussian, 4 VASP, and 5 NAMD

The post OpenACC Shines in Global Climate/Weather Codes appeared first on HPCwire.

via Government – HPCwire