Today
at AWS re:Invent, Amazon Web Services, Inc. (AWS) announced 13 new machine learning
capabilities and services, across all layers in the machine learning
stack, to help put machine learning in the hands of even more
developers. AWS introduced new Amazon SageMaker features making it
easier for developers to build, train, and deploy machine learning
models - including low cost, automatic data labeling and reinforcement
learning (RL). AWS revealed new services, framework enhancements, and a
custom chip to speed up machine learning training and inference, while
reducing cost. AWS announced new artificial intelligence (AI) services
that can extract text from virtually any document, read medical
information, and provide customized personalization, recommendations,
and forecasts using the same technology used by Amazon.com. And, last
but certainly not least, AWS will help developers get rolling with
machine learning with AWS DeepRacer, a new 1/18th scale autonomous model race car for developers, driven by reinforcement learning.
These
announcements continue the drum beat of machine learning innovation
from AWS, which has launched more than 200 significant machine learning
capabilities in the past 12 months. Customers using these new services
and capabilities include Adobe, BMW, Cathay Pacific, Dow Jones, Expedia,
Formula 1, GE Healthcare, HERE, Intuit, Johnson & Johnson, Kia
Motors, Lionbridge, Major League Baseball, NASA JPL, Politico.eu,
Ryanair, Shell, Tinder, United Nations, Vonage, the World Bank, and
Zillow. To learn more about AWS's new machine learning services, visit: https://aws.amazon.com/machine-learning.
"We
want to help all of our customers embrace machine learning, no matter
their size, budget, experience, or skill level," said Swami
Sivasubramanian, Vice President, Amazon Machine Learning. "Today's
announcements remove significant barriers to the successful adoption of
machine learning, by reducing the cost of machine learning training and
inference, introducing new SageMaker capabilities that make it easier
for developers to build, train, and deploy machine learning models in
the cloud and at the edge, and delivering new AI services based on our
years of experience at Amazon."
New infrastructure, a custom machine learning chip, and framework improvements for faster training and low-cost inference
Most
machine learning models are trained by an algorithm that finds patterns
in large amounts of data. The model can then make predictions on new
data in a process called ‘inference'. Developers use machine learning
frameworks to define these algorithms, train models, and infer
predictions. Frameworks (such as TensorFlow, Apache MXNet, and PyTorch)
allow developers to design and train sophisticated models, often using
multiple GPUs to reduce training times. Most developers use more than
one of these frameworks in their day-to-day work. Today, AWS announced
significant improvements for developers building with all of these
popular frameworks, by improving performance and reducing cost for both
training and inference.
- New Amazon Elastic Compute Cloud (EC2) GPU instances (available next week):
With eight NVIDIA V100 GPUs, 32GB GPU memory, fast NVMe storage, 96
Intel "Skylake" vCPUs, and 100Gbps networking, the new P3dn.24xl
instances are the most powerful machine learning training processors
available in the cloud, allowing developers to train models with more
data in less time.
- AWS-Optimized TensorFlow framework (generally available today):
When training with large amounts of data, developers who choose to use
TensorFlow have found that it's challenging to scale TensorFlow across
many GPUs, which often results in low utilization of these GPUs and
longer training times for large training jobs. AWS worked on this
problem and has innovated on how to make TensorFlow scale across GPUs.
By improving the way in which TensorFlow distributes training tasks
across those GPUs, the new AWS-Optimized TensorFlow achieves close to
linear scalability when training multiple types of neural networks (90
percent efficiency across 256 GPUs, compared to the prior norm of 65
percent). Using the new AWS-Optimized TensorFlow and P3dn instances,
developers can now train the popular ResNet-50 model in only 14 minutes,
the fastest time recorded, and 50 percent faster than the previous best
time. And, these optimizations are generally applicable not just for
computer vision models but also for a broader set of deep learning
models.
- Amazon Elastic Inference (generally available today): While
training rightfully receives a lot of attention, inference actually
accounts for the majority of the cost and complexity for running machine
learning in production (for every dollar spent on training, nine are
spent on inference). Amazon Elastic Inference allows developers to
dramatically decrease inference costs with up to 75 percent savings when
compared to the cost of using a dedicated GPU instance. Instead of
running on a whole Amazon EC2 P2 or P3 instance with relatively low
utilization, developers can run on a smaller, general-purpose Amazon EC2
instance and provision just the right amount of GPU performance from
Amazon Elastic Inference. Starting at just 1 TFLOP, developers can
elastically increase or decrease the amount of inference performance,
and only pay for what they use. Elastic Inference supports all popular
frameworks, and is integrated with Amazon SageMaker and the Amazon EC2
Deep Learning Amazon Machine Image (AMI). And, developers can start
using Amazon Elastic Inference without making any changes to their
existing models.
- AWS Inferentia (available in 2019): For
larger workloads that consume entire GPUs or require lower latency, AWS
announced a high performance machine learning inference chip, custom
designed by AWS. AWS Inferentia provides hundreds of teraflops per chip
and thousands of teraflops per Amazon EC2 instance for multiple
frameworks (including TensorFlow, Apache MXNet, and PyTorch), and
multiple data types (including INT-8 and mixed precision FP-16 and
bfloat16).
Autodesk
is a leader in 3D design, engineering, and entertainment software who
uses deep learning models for use cases ranging from exploring thousands
of potential design alternatives, semantically searching designs and
optimizing the engineering construction process to applying better UV
mapping in photo realistic renderings. "Running efficient inference is
one of the biggest challenges in machine learning today," said Peter
Jones, Head of AI Engineering for Autodesk Research. "Amazon Elastic
Inference is the first capability of its kind we've found to help us
eliminate excess costs that we incur today from idle GPU capacity. We
estimate it will save us 75 percent in costs compared to running GPUs."
EagleView,
a property data analytics company, helps lower property-damage losses
from natural disasters by decreasing the time it takes to assess damage
so that homeowners can decide next steps much faster. Using aerial,
drone, and satellite images, EagleView runs deep learning models on AWS
to make quicker, more accurate assessments of property damage within 24
hours of a natural disaster. "Matching the accuracy of human adjusters
in property assessments requires us to process massive amounts of data
in the form of ultra-high resolution images covering the entire
multi-dimensional space (spatial, spectral, and temporal) of a
disaster-affected region," explains Shay Strong, Director of Data
Science and Machine Learning at EagleView. "Amazon Elastic Inference
opens new doors that enables us to explore running workflows more cost
effectively at scale."
New
Amazon SageMaker capabilities make it easier to build, train, and
deploy machine learning; developers get hands on with AWS DeepRacer, a
1/18th scale autonomous race car driven by reinforcement learning
Amazon
SageMaker is a fully managed service that removes the heavy lifting and
guesswork from each step of the machine learning process. Amazon
SageMaker makes it easier for developers to build, train, tune, and
deploy machine learning models. Today, AWS announced a number of new
capabilities for Amazon SageMaker.
- Amazon SageMaker Ground Truth (generally available today): The journey to build machine learning models requires developers to prepare their datasets for training their ML models. Before
developers can select their algorithms, build their models, and deploy
them to make predictions, human annotators manually review thousands of
examples and add the labels required to train machine learning models.
This process is time consuming and expensive. Amazon SageMaker Ground
Truth makes it much easier for developers to label their data using
human annotators through Mechanical Turk, third party vendors, or their
own employees. Amazon SageMaker Ground Truth learns from these
annotations in real time and can automatically apply labels to much of
the remaining dataset, reducing the need for human review. Amazon
SageMaker Ground Truth creates highly accurate training data sets, saves
time and complexity, and reduces costs by up to up to 70 percent when
compared to human annotation.
- AWS Marketplace for Machine Learning (generally available today): Machine
learning is moving quickly, with new models and algorithms from
academia and industry appearing virtually every week. Amazon SageMaker
includes some of the most popular models and algorithms built-in, but to
make sure developers continue to have access to the broadest set of
capabilities, the new AWS Marketplace for Machine Learning includes over
150 algorithms and models (with more coming every day) that can be
deployed directly to Amazon SageMaker. Developers can start using these
immediately from SageMaker. Adding a listing to the Marketplace is
completely self-service for developers who want to sell through AWS
Marketplace.
- Amazon SageMaker RL (generally available today): In
machine learning circles, there is a lot of buzz about reinforcement
learning because it's an exciting technology with a ton of potential.
Reinforcement learning trains models, without large amounts of training
data, and it's broadly useful when the reward function of a desired
outcome is known but the path to achieving it is not and requires a lot
of iteration to discover. Healthcare treatments, optimizing
manufacturing supply chains, and solving gaming challenges are a few of
the areas that reinforcement learning can help address. However,
reinforcement learning has a steep learning curve and many moving parts,
which effectively puts it out of the reach of all but the most
well-funded and technical organizations. Amazon SageMaker RL, the
cloud's first managed reinforcement learning service, allows any
developer to build, train, and deploy with reinforcement learning
through managed reinforcement learning algorithms, support for multiple
frameworks (including Intel Coach and Ray RL), multiple simulation
environments (including SimuLink and MatLab), and integration with AWS
RoboMaker, AWS's new robotics service, which provides a simulation
platform that integrates well with SageMaker RL.
- AWS DeepRacer (available for pre-order today): In just a few lines of code, developers can start learning about reinforcement learning with AWS DeepRacer, a 1/18th scale
fully autonomous race car. The car (with all-wheel drive, monster truck
tires, an HD video camera, and on-board compute) is driven using
reinforcement learning models trained using Amazon SageMaker. Developers
can put their skills to the test and race their cars and models against
other developers for prizes and glory in the DeepRacer League, the
world's first global autonomous racing league, open to everyone.
- Amazon SageMaker Neo (generally available today): The
new deep learning model compiler lets customers train models once, and
run them anywhere with up to 2X improvement in performance. Applications
running on connected devices at the edge are particularly sensitive to
performance of machine learning models. They require low latency
decisions, and are often deployed across a broad number of different
hardware platforms. Amazon SageMaker Neo compiles models for specific
hardware platforms, optimizing their performance automatically, allowing
them to run at up to twice the performance, without any loss in
accuracy. As a result, developers no longer need to spend time hand
tuning their trained models for each and every hardware platform (saving
time and expense). SageMaker Neo supports hardware platforms from
NVIDIA, Intel, Xilinx, Cadence, and Arm, and popular frameworks such as
TensorFlow, Apache MXNet, and PyTorch. AWS will also make Neo available
as an open source project.
Tyson
Foods is one of the world's largest food companies and a recognized
leader in protein. "We are building a computer vision system for our
chicken processing facilities and we need highly accurate labeled
training datasets to train these systems," said Chad Wahlquist, Director
of Emerging Technology for Tyson Foods. "When we first tried to setup
our own labeling solution, it required a large amount of compute and a
Frankenstein of open source solutions - even before creating the user
interface for data labeling. With Amazon SageMaker Ground Truth, we were
able to use the readymade template for bounding boxes and got a
labeling job running in just a few clicks, quickly and easily. Amazon
SageMaker Ground Truth also enables us to securely bring our own workers
to label the data, which is an essential requirement for our business.
We are looking forward to using Amazon SageMaker Ground Truth across our
business."
Dubbed
"America's Un-carrier," T-Mobile is a leading wireless services,
products, and service innovation provider. "The AI at T-Mobile team is
integrating AI and machine learning into the systems at our customer
care centers, enabling our team of experts to serve customers with
greater speed and accuracy through Natural Language Understanding models
that show them relevant, contextual customer information in real-time,"
said Matthew Davis, Vice President of IT Development for T-Mobile.
"Labeling data has been foundational to creating high performing models,
but is also a monotonous task for our data scientists and software
engineers. Amazon SageMaker Ground Truth makes the data labeling process
easy, efficient, and accessible, freeing up time for them to focus on
what they love - building products that deliver the best experiences for
our customers and care representatives."
Chick-fil-A,
Inc. is a family owned and privately held restaurant company that is
known for its original chicken sandwich and which serves freshly
prepared food in more than 2,300 restaurants in 47 states and
Washington, DC. "Food safety is of critical importance in our business.
Our early efforts with computer vision and machine learning show promise
in improving operations," said Jay Duff, Principal Team Lead for
Chick-fil-A. "Amazon SageMaker and GroundTruth helped us speed up the
development of new models and evaluations by making it easier to label
and verify new training sets, re-train models, and then iterate on more
complex data. Additionally, the workforce management features gave us
faster turnaround on manual tasks while reducing administrative toil."
Arm
technology is at the heart of a computing and connectivity revolution
that is transforming the way people live and businesses operate. "Arm's
vision of a trillion connected devices by 2035 is driven by the
additional consumer value derived from innovations like machine
learning," said Jem Davies, fellow, General Manager and Vice President
for the Machine Learning Group at Arm. "The combination of Amazon
SageMaker Neo and the Arm NN SDK will help developers optimize machine
learning models to run efficiently on a wide variety of connected edge
devices."
Cadence
enables electronic systems and semiconductor companies to create the
innovative end products that are transforming the way people live, work,
and play. Cadence software, hardware and semiconductor IP are used by
customers to deliver products to market faster. "Cadence(r) Tensilica(r)
processors are optimized for on-device machine learning applications
spanning from autonomous driving cars to speech processing to robotics,"
said Babu Mandava, Senior Vice President and General Manager of the IP
Group at Cadence Design Systems. "Amazon SageMaker Neo simplifies the
deployment of optimized models from cloud to the edge. We are excited to
be driving a seamless integration of Amazon SageMaker Neo with our
Tensilica processor family and development environment to help
developers optimize machine learning models for Tensilica-powered edge
devices."
GE
Healthcare is a leading provider of medical imaging, monitoring,
biomanufacturing, and cell and gene therapy technologies that enables
precision health in diagnostics, therapeutics and monitoring through
intelligent devices, data analytics, applications and services. "GE
Healthcare is transforming healthcare by empowering providers to deliver
better outcomes," said Keith Bigelow, Senior Vice President of Edison
Portfolio Strategy, GE Healthcare. "We train computer vision models with
Amazon SageMaker that are then deployed in our MRI and X-Ray devices.
By applying reinforcement learning techniques, we are able to reduce the
size of our trained models while achieving the right balance between
network compression and model accuracy. Amazon SageMaker RL enabled us
to get from idea to implementation in less than four weeks by removing
the complexities of running reinforcement learning workloads."
"Reinforcement
Learning is enabling innovation in machine learning and robotics," said
Brad Porter, Vice President and Distinguished Engineer of Amazon
Robotics. "We're excited Amazon SageMaker is making it easier to try
reinforcement learning techniques with real-world applications, and
we're already experimenting with ways to use it for robotic
applications. For instance, earlier this year we showed a robot that was
able to play beer pong using some of these techniques and we're excited
to continue to explore these opportunities in partnership collaboration
with AWS."
New AI services bring intelligence to all apps, no machine learning experience required
Many
developers want to be able to add intelligent features to their
applications without requiring any machine learning experience. Building
on existing computer vision, speech, language, and chatbot services,
AWS announced a significant expansion of AI services.
- Amazon Textract (available in preview today): Many
companies today extract data from documents and forms through manual
data entry which is slow and expensive, or using simple optical
character recognition (OCR) software, which is often inaccurate and
typically produces output that requires extensive post-processing to put
the extracted content in a format that is usable by a developer's
application. Amazon Textract uses machine learning to instantly read
virtually any type of document to accurately extract text and data
without the need for any manual review or custom code. Amazon Textract
allows developers to quickly automate document workflows, processing
millions of document pages in a few hours.
- Amazon Comprehend Medical (generally available today): Building
the next generation of medical applications requires being able to
understand and analyze the information that is often trapped in
free-form, unstructured medical text, such as hospital admission notes
or patient medical histories. Comprehend Medical is a highly accurate
natural language processing service for medical text, which uses machine
learning to extract disease conditions, medications, and treatment
outcomes from patient notes, clinical trial reports, and other
electronic health records. Comprehend Medical requires no machine
learning expertise, no complicated rules to write, no models to train,
and it is continuously improving. You pay only for what you use and
there are no minimum fees or upfront commitments.
- Amazon Personalize (available in preview today): Based
on the same technology that powers Amazon.com, Amazon Personalize is a
real-time recommendation and personalization service. Amazon
pioneered the use of machine learning for recommendation and
personalization over twenty years ago. Experience has shown that there
is no master algorithm for personalization. Each use case, from videos,
music, products, news articles, has its own specificities, which require
a unique mix of data, algorithms, and optimizations. Amazon Personalize
provides this experience to customers in a fully managed service, which
will build, train, and deploy custom, private personalization and
recommendation models for virtually any use case. Amazon Personalize can
make recommendations, personalize search results, and segment customers
for direct and personalized marketing through email or push
notifications.
- Amazon Forecast (available in preview today): Just like personalization, forecasting
has traditionally been a bit of a dark art, where customers try to
predict future trends in supply chain, inventory levels, and product
demand, based on historical data. And just like Amazon Personalize,
Amazon Forecast is based on technology that has been developed by
Amazon.com and used for a lot of critical forecasting. Forecasting is
hard to do well because there are often so many inter-related factors
(such as pricing, events, and even the weather). Missing the mark with a
forecast can have a significant impact, such as being unable to meet
customer demand or significantly over-spending. Amazon Forecast creates
accurate time-series forecasts. Using historical data and related causal
data, Amazon Forecast will automatically train, tune, and deploy
custom, private machine learning forecasting models, so that customers
can be more confident that they'll provide the right customer experience
while optimizing their spend.
Cox
Automotive is a subsidiary of Cox enterprises which encompasses all of
Cox's global automotive businesses including Kelley Blue Book, Xtime,
Autotrader.com, and Manheim. "At Cox Automotive, we are looking to
transform how the world buys, sells, and trade cars. To further
modernize our automotive solutions, we will be leveraging Amazon
Textract to accelerate how quickly cars can be transacted," said Bryan
Landerman, Chief Technology Officer at Cox Automotive. "With Amazon
Textract, we can automatically capture and validate data from documents
and forms, such as loan applications or vehicle titles, so decisions can
be made more quickly. This will reduce customer effort and further
streamline the process for everyone involved from the manufacturer to
the buyer."
Alfresco
is a leading enterprise open source provider of process automation,
content management, and information governance software. "At Alfresco,
we want to make document processing and content management as simple as
possible for our customers. Since a document management system is only
as good as its input, it is critical that we have the foundational tools
that can automatically and accurately extract key information from
digitized documents," said John Newton, CTO and Founder at Alfresco.
"Previously, we built custom solutions on top of OCR technology in order
to extract data of interest, which required intensive manual training.
This process consumed valuable time and resources, but it was work that
had to be done. With Amazon Textract, we can now automatically extract
not just the text in a document and table information, but real insights
that allow us to automate data entry and facilitate faster business
decisions. Amazon Textract is enabling us to provide greater data
integrity, security compliance, and the ability to launch business
processes faster than ever. And most importantly, all of this helps us
better assist our customers in their digital transformation journey."
Beth
Israel Deaconess Medical Center (BIDMC) is a patient care, teaching,
and research affiliate of Harvard Medical School and consistently ranks
as a national leader among independent hospitals in National Institutes
of Health funding. BIDMC is the official hospital of the Boston Red Sox.
"At BIDMC, we have over 490 surgical beds that are always occupied. We
strive to quickly and successfully perform surgical procedures so our
patients can be treated in time. But a lot of procedures were being
canceled and delayed because the patients' completed History and
Physical (H&P) form required before surgery was difficult to locate
in the Electronic Health Records (EHR)," said Venkat Jegadeesan, Senior
Enterprise Architect of Beth Israel Deaconess Medical Center. "To solve
this, we started to use Amazon Comprehend Medical to make data in our
EHR systems to be easily searchable using key medical text. Our teams
are now able to identify the H&P's quickly with the right prompts
for clinical staff. As a result, we can save a lot of valuable time and
help prevent delays or potential cancellations which can be inconvenient
for patients and their families."
At
Fred Hutchinson Cancer Research Center, home to three Nobel laureates,
interdisciplinary teams of world-renowned scientists seek new and
innovative ways to prevent, diagnose and treat cancer, HIV/AIDS and
other life-threatening diseases. "Curing cancer is, inherently, an issue
of time," said Matthew Trunnell, Chief Information Officer, Fred
Hutchinson Cancer Research Center. "For cancer patients and the
researchers dedicated to curing them, time is the limiting resource. The
process of developing clinical trials and connecting them with the
right patients requires research teams to sift through and label
mountains of unstructured medical record data. Amazon Comprehend Medical
will reduce this time burden from hours to seconds. This is a vital
step toward getting researchers rapid access to the information they
need when they need it so they can find actionable insights to advance
lifesaving therapies for patients."
PricewaterhouseCoopers
(PwC) is a network of firms in 158 countries with over 250,000 people
who are committed to delivering quality in assurance, advisory, and tax
services. "Amazon Comprehend Medical provides us the ability to realize
better results with less overhead. By using Comprehend Medical with our
customers we are able to focus more on building smarter applications and
less on annotating, training and re-training models," said Matt
Rich, Healthcare AI Lead for PwC. "For our customers the ability to
perform a very manual task accurately at scale allows us to create more
impactful solutions. For example, one of our pharmaceutical clients is
using Comprehend Medical on a limited sample size to help extract
information that allows them to identify medically relevant events. In
preliminary findings, we are seeing a significantly faster throughput
than in the past."
Domino's
Pizza Enterprises Ltd. is the largest pizza chain in Australia and
their vision is to be the leader in the Internet of Food in every
neighborhood. "The customer is at the heart of everything we do at
Domino's and we are working relentlessly to improve and enhance their
experience with the brand," said Mallika Krishnamurthy, Global Head,
Strategy & Insights, Domino's Pizza Enterprises. "Using Amazon
Personalize in conjunction with Amazon Pinpoint, we are able to achieve
personalization at scale across our entire customer base, which was
previously impossible. Amazon Personalize enables us to apply context
about individual customers and their circumstances, and deliver
customized communications such as special deals and offers through our
digital channels."
Mercado
Libre is a leading online commerce and payments platform in Latin
America. "We have been predicting demand for over 50,000 different
products using Amazon Forecast's state-of-the-art deep learning
algorithms, that we can use right out of the box," said Adrian Quilis,
Director of Business Intelligence for Mercado Libre. "Amazon Forecast
takes care of all the heavy lifting of setting up pipelines, re-training
schedules, and re-generating forecasts, so we can experiment with
hundreds of models very easily."