Building systems that automatically adjust to workloads and data

Cloud and systems
Building systems that automatically adjust to workloads and data
Tim Kraska, who joined Amazon this summer to build the new Learned Systems research group, explains the power of “instance optimization”.
分享到微信
x
As an associate professor of electrical engineering and computer science at MIT, Tim Kraska cofounded a company called Instancio, which focused on instance-optimized database systems, or systems that can automatically adapt to new workloads with minimal human involvement.
Tim Kraska, an associate professor of electrical engineering and computer science at MIT and director of applied science for Amazon Web Services.
Earlier this year, Amazon acquired Instancio’s technology and hired Kraska and his team to continue developing it. Currently, Kraska is on leave from MIT, and as director of applied science for Amazon Web Services (AWS), he is helping establish Amazon’s new Learned Systems Group (LSG), which will focus on integrating machine learning (ML) into system design. The group’s first project is to bring instance optimization to AWS’s data warehousing service, Amazon Redshift. Kraska spoke with Amazon Science about the value of instance optimization and the attraction of doing research in an industrial setting.
Q. 
What is instance optimization?
A. 
If you develop a system from scratch for a particular use case, you are able to get orders of magnitude better performance, as you can tailor every system component to that use case. However, in most cases you don't want to do that, because it's a huge effort. In the case of databases, the saying is that it normally takes at least seven years to get the system so that it's usable and stable.
The idea of instance optimization is that, rather than build one system per use case, we build a system that self-adjusts — instance-optimizes itself — to a particular scenario to get as close as possible to a hand-tuned solution.
Q. 
How does it do that?
A. 
There are different ways to achieve the self-adjustment. With any system, you have a bunch of knobs and a bunch of design choices. If you take Redshift, you can tune the buffer size; you can create materialized views ; you can create different types of sort orders. And database administrators can adjust these knobs and make design choices, based on their workloads, to get better performance.
Related content
Amazon Redshift: Ten years of continuous reinvention
Two authors of Amazon Redshift research paper that will be presented at leading international forum for database researchers reflect on how far the first petabyte scale cloud data warehouse has advanced since it was announced ten years ago.
The first form of self-adjustment is to make those decisions automatically. You have, let's say, a machine learning model that observes the workload and figures out how to adjust these knobs and what materialized views and sort keys to create. RedShift already does this, for example, with a feature called Automated Materialized Views , which accelerates query performance.
The next step is that in some cases it's possible to replace components through novel techniques that allow either more customization or tuning in ways that weren’t previously possible.
To give you an example, in the case of data layouts, current systems mainly support partitioning data by one attribute, which could be a composite key. The reason is that the developers of these systems always thought that someone has to eventually make these design choices manually. Thus, in the past, the tendency was to reduce the number of tuning parameters as much as possible.
Related content
How to reduce communication overhead for database queries by up to 97%
Amazon researchers describe new method for distributing database tables across servers.
This, of course, changes the moment you have automatic tuning techniques using machine learning, which can explore the space much more efficiently. And now maybe the opposite is true: providing more degrees of freedom and more knobs is a good thing, as they offer more potential for customization and, thus, better performance.
The third self-adjustment method is where you deeply embed machine learning models into a component of the system to give you much better performance than is currently possible.
Every database, for example, has a query optimizer that takes a SQL query and optimizes it to an execution plan, which describes how to actually run that query. This query optimizer is a complex piece of software, which requires very carefully tuned heuristics and cost models to figure out how best to do this translation. The state of the art now is that you treat this as a deep-learning problem. So we talk at that stage about learned components.
A comparison of two different approaches to learning to detect query patterns, using graph convolution networks (top) and tree convolution networks (bottom). From “ LSched: A workload-aware learned query scheduler for analytical database systems ”.
The ultimate goal is to build a system out of learned components and to have everything tuned in a holistic way. There's a model monitoring the workload, watching the system, and making the right adjustments — potentially in ways no human is able to.
Q. 
Is it true that you developed an improved sorting algorithm? I thought that sorting was pretty much a solved problem.
A. 
That's right. It's still surprising. The way it works is, you learn a model over the distribution of the data — the cumulative distribution function, or CDF, which tells you where an item falls into the probability mass. Let's assume that in an e-commerce database, you have a table with orders, each order has a date, and you want to sort the table by date. Now you can build the CDF over the date attribute, and then you can ask a question like “How many orders happened before January 1st, 2021?”, and it spits out the probability.
The nice thing about that is that, essentially, the CDF function allows you to ask, “Given an order date, where in the sorted order does it fit?” Assuming the model is perfect, it suddenly allows you to do sorting in O(n). [I.e., the sorting time is proportional to the number of items being sorted, n, not n2, nlogn, or the like .]
Recursively applying the cumulative distribution function (CDF) to sort items in an array in O(n) time. From “ The case for a learned sorting algorithm ”.
Radix sort is also O(n), but it can be memory intensive, as the efficiency depends on the domain size — how many unique values there could possibly be. If your domain is one to a million, it might still be easily do-able in memory. If it's one to a billion, it already gets a little bit harder. If it's one to — pick your favorite power of ten — it eventually becomes impossible to do it in one pass.
The model-based approach tries to overcome that in a clever way. You know roughly where items land, so you can place them into their approximate position and use insertion sort to correct for model errors. It’s a trick we used for indexes, but it turns out that you can use the same thing for sorting.
Q. 
For you, what was the appeal of doing research in the industrial setting?
A. 
One of the reasons we are so attracted to doing this deal with Amazon is access to real-world data. Instance optimization is all about self-adjusting to the workload and the data. And it's extremely hard to test it in academia.
There are a few benchmark datasets, but internally, they often use random-number generators to create the data and to determine when and what types of queries are issued against the system.
We fundamentally have to rethink how we build systems. ... Whenever a developer has to make a trade-off between two techniques or defines a constant, the developer should think about if this constant or trade-off shouldn’t be automatically tuned.
Tim Kraska
Because of this randomness, first of all, there are no interesting usage patterns — say, when are the dashboarding queries running, versus the batch jobs for loading the data. All that is gone. Even worse, the data itself doesn’t contain any interesting patterns, which either makes it too hard, because everything is random, or too easy, because everything is random.
For example, when we tested our learned query optimizer on a very common data-warehousing benchmark, we found that we barely got any improvements, whereas for real-world workloads, we saw big improvements.
We dug in a little bit, and it turns out that for common benchmarks, like TPC-H, every single database vendor makes sure that the query plans are close to perfect. They manually overfit the system to the benchmark. And this translates in no way to any real-world customer. No customer really runs queries exactly like the benchmark. Nobody does.
Working with Redshift’s amazing development team and having access to real workloads provides a huge advantage here. It allows us not only to evaluate if our previous techniques actually work in practice, but it also helps us to focus on developing new techniques, which actually make a big difference to users by providing better performance or improved ease of use.
Q. 
So the collaboration with the Redshift team is going well?
A. 
It has been great and, in many ways, exceeded our expectations. When we joined, we certainly had some anxiety about how we would be working with the Redshift team, how much we would still be able to publish, and so on. For example, I know many researchers in industry labs who struggle to get access to data or have actual impact on the product.
Related content
Speeding database queries by rewriting redundancies
Amazon Athena reduces query execution time by 14% by eliminating redundant operations.
None of these turned out to be a real concern. Not only did we define our own research agenda, but we are also already deeply involved with many exciting projects and have a whole list of exciting things we want to publish about.
As far as I can tell, we are the first dedicated ML-for-systems research group at any large tech company. This provides a unique option not only for grad students and postdocs but for professionals who are looking for the right mix of research and real-world impact.
Q. 
Do you still collaborate with MIT?
A. 
Yes, and it is very much encouraged. Amazon recently created a Science Hub at MIT , and as part of the hub, AWS is also sponsoring DSAIL, a lab focused on ML-for-systems research. This allows us to work very closely with researchers at MIT.
Q. 
Some of the techniques you’ve discussed, such as sorting, have a wide range of uses. Will the Learned Systems Group work with groups other than Redshift?
A. 
We decided to focus on Redshift first as we had already a lot of experience with instance optimization for analytical systems, but we’ve already started to talk to other teams and eventually plan to apply the ideas more broadly.
I believe that we fundamentally have to rethink how we build systems and system components. For example, whenever a developer has to make a trade-off between two techniques or defines a constant, the developer should think about if this constant or trade-off shouldn’t be automatically tuned. In many cases, the developer would probably approach the design of the component completely differently if she knows that the component is expected to self-adjust to the workload and data.
Related content
Amazon EBS addresses the challenge of the CAP Theorem at scale
Optimizing placement of configuration data ensures that it’s available and consistent during “network partitions”.
This is true not only for data management systems but across the entire software stack. For example, there has been work on improving network packet classification using learned indexes, spark scheduling algorithms using reinforcement learning, and video compression using deep-learning techniques to provide a better experience when bandwidth is limited. All these techniques will eventually impact the customer experience in the form of performance, reduced cost, or ease of use.
For good reason, we already see a lot of adaptation of ML to improve systems at Amazon. Redshift, for example, offers multiple ML-based features — like Automated Materialized Views or automatic workload management . With the Learned Systems Group, we hope to accelerate that trend, with fully instance-optimized systems that self-adjust to workloads and data in ways no traditional system can. And that will provide better performance, cost, and ease of use for AWS customers.
Research areas
Larry Hardesty is the editor of the Amazon Science blog. Previously, he was a senior editor at MIT Technology Review and the computer science writer at the MIT News Office.
Related content
Staff writer
January 18, 2022
Amazon DynamoDB was introduced 10 years ago today; one of its key contributors reflects on its origins, and discusses the 'never-ending journey' to make DynamoDB more secure, more available and more performant.
Staff writer
May 18, 2022
Two authors of Amazon Redshift research paper that will be presented at leading international forum for database researchers reflect on how far the first petabyte scale cloud data warehouse has advanced since it was announced ten years ago.
Applied Scientist, AWS Proactive Security Detective Tooling
US, VA, Arlington
Job summaryAmazon Web Services is looking for world class scientists to join the Proactive Security group within AWS Security. You will join a team of scientists developing advanced techniques to assess and enhance the security of AWS systems. This group will be visible across AWS teams and to Amazon security leadership as a source of innovative techniques to keep AWS secure. You will help expand our ability to automatically fix security issues by synthesizing patches or automatically applying safe changes to running systems.This role can be based in Seattle, Portland, NYC, Cupertino, Washington DC, Boston, or another office with a significant Amazon Security presence. Each day, hundreds of thousands of developers make billions of transactions worldwide on our cloud. They harness the power of Amazon Web Services (AWS) to enable innovative applications, websites, and businesses. The AWS Security team owns security for all of these services offered by AWS, including EC2, S3, Lambda, and more than 150 others. Our Proactive Security team works with builders across AWS to ensure that AWS products are secure. We dive deep into security technologies such as new authentication systems, hardware security components, cryptography, system hardening, and massive-scale audit analysis. We use techniques drawn from research in Automated Reasoning, Program Analysis, Program Synthesis, and Machine Learning to automatically identify security issues or provide assurance that security requirements are met.You will identify, extend, and prototype advanced techniques that can be applied to solve hard problems in security. In some cases, your team will invent new approaches. In others cases, your team will identify and adapt existing approaches. You will work with security experts and software developers to integrate these techniques into AWS Security’s mechanisms for reducing security risk. AWS will look to your team to find the techniques that will radically improve the security of AWS.While you may not be a security expert yet, in this role you will develop a broad and deep understanding of cloud security and security automation. Our team puts a high value on work-life balance. Striking a healthy balance between your personal and professional life is crucial to your happiness and success here, which is why we aren’t focused on how many hours you spend at work or online. Instead, we’re happy to offer a flexible schedule so you can have a more productive and well-balanced life—both in and outside of work. Our team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. We care about your career growth and strive to assign projects based on what will help each team member take the next step in their career. Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and we host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust.Key job responsibilities* Rapidly design, prototype and test advanced security analysis and repair techniques in a high-ambiguity environment, making use of both quantitative and business judgment.* Collaborate with software engineers and security engineers to integrate successful experiments into large scale production services.* Report results in a scientifically rigorous way.* Interact with security engineers and related domain experts to dive deep into the types of challenges that need innovative solutions.* Track relevant research in other Amazon science teams and in the broader research community.
Applied Scientist I, Alexa Sensitive Content Intelligence (ASCI)
IN, KA, Bangalore
Job summaryAlexa is the voice activated digital assistant powering devices like Amazon Echo, Echo Dot, Echo Show,and Fire TV, which are at the forefront of this latest technology wave.To preserve our customers’ experience and trust, the Alexa Sensitive Content Intelligence (ASCI) team creates policies and builds services and tools through Machine Learning techniques to detect and mitigate sensitive content across Alexa.An Applied Scientist I will be working with a team of exceptional scientists to develop novel algorithms and modeling techniques to advance the state of the art in NLP related tasks.You will work in a hybrid, fast-paced organization where scientists, engineers, and product managers work together to build customer facing experiences.You will collaborate with other scientists to raise the bar of scientific research in Amazon.Your work will directly impact our customers in the form of products and services that make use of speech,language, and computer vision technologies.Key job responsibilities-You'll contribute to the science solution design, run experiments, research new algorithms, and find new ways of optimizing customer experience-Besides theoretical analysis and innovation, you will work closely with talented engineers and ML scientists to put your algorithms and models into practice-Your work will directly impact the trust customers place in Alexa, globallyA day in the lifeYou will be working with a group of talented scientists on researching algorithm and running experiments to test scientific proposal/solutions to improve our sensitive contents detection and mitigation.This will involve collaboration with partner teams including engineering,PMs,data annotators,and other scientists to discuss data quality, policy, and model development.You will mentor other scientists,review and guide their work,help develop roadmaps for the team.You work closely with partner teams across Alexa to deliver platform features that require cross-team leadership.About the teamThe mission of the Alexa Sensitive Content Intelligence (ASCI) team is to (1) minimize negative surprises to customers caused by sensitive content(2) detect and prevent potential brand-damaging interactions(3) build customer trust through appropriate interactions on sensitive topics.The term “sensitive content” includes within its scope a wide range of categories of content such as offensive content (e.g. hate speech, racist speech), profanity, content that is suitable only for certain age groups, politically polarizing content, and religiously polarizing content.
Senior Applied Scientist, Alexa Sensitive Content Intelligence (ASCI)
IN, KA, Bangalore
Job summaryAlexa is the voice activated digital assistant powering devices like Amazon Echo, Echo Dot, Echo Show, and Fire TV, which are at the forefront of this latest technology wave.To preserve our customers’ experience and trust, the Alexa Sensitive Content Intelligence (ASCI) team creates policies and builds services and tools through Machine Learning techniques to detect and mitigate sensitive content across Alexa.We are looking for an experienced Senior Applied Scientist to build industry-leading technologies in attribute extraction and sensitive content detection across all languages and countries.A Senior Applied Scientist will be a tech lead for a team of exceptional scientists to develop novel algorithms and modeling techniques to advance the state of the art in NLP related tasks.You will work in a hybrid, fast-paced organization where scientists,engineers,and product managers work together to build customer facing experiences.You will collaborate with and mentor other scientists to raise the bar of scientific research in Amazon.Your work will directly impact our customers in the form of products and services that make use of speech, language, and computer vision technologies.We are looking for a leader with strong technical experiences a passion for building scientific driven solutions in a fast-paced environment.You should have good understanding of NLP models (e.g. LSTM, transformer based models) and where to apply them in different business cases.You leverage your exceptional technical expertise, a sound understanding of the fundamentals of Computer Science, and practical experience of building large-scale distributed systems to creating reliable, scalable, and high-performance products. In addition to technical depth, you must possess exceptional communication skills and understand how to influence key stakeholders.You will be joining a select group of people making history producing one of the most highly rated products in Amazon's history, so if you are looking for a challenging and innovative role where you can solve important problems while growing as a leader, this may be the place for you.Key job responsibilities-You'll lead the science solution design, run experiments, research new algorithms, and find new ways of optimizing customer experience-You set examples for the team on good science practice and standards-Besides theoretical analysis and innovation, you will work closely with talented engineers and ML scientists to put your algorithms and models into practice-Your work will directly impact the trust customers place in Alexa, globally-You contribute directly to our growth by hiring smart and motivated Scientists to establish teams that can deliver swiftly and predictably,adjusting in an agile fashion to deliver what our customers needA day in the lifeYou will be working with a group of talented scientists on researching algorithm and running experiments to test scientific proposal/solutions to improve our sensitive contents detection and mitigation. This will involve collaboration with partner teams including engineering, PMs, data annotators, and other scientists to discuss data quality, policy, and model development. You will mentor other scientists, review and guide their work, help develop roadmaps for the team. You work closely with partner teams across Alexa to deliver platform features that require cross-team leadership.About the teamThe mission of the Alexa Sensitive Content Intelligence (ASCI) team is to (1) minimize negative surprises to customers caused by sensitive content(2) detect and prevent potential brand-damaging interactions(3) build customer trust through appropriate interactions on sensitive topicsThe term “sensitive content” includes within its scope a wide range of categories of content such as offensive content (e.g., hate speech, racist speech), profanity, content that is suitable only for certain age groups, politically polarizing content, and religiously polarizing content. The term “content” refers to any material that is exposed to customers by Alexa (including both 1P and 3P experiences) and includes text, speech, audio, and video.
Amazon Robotics - Applied Scientist II Co-op - Spring 2023, Navigation Technologies
US, MA, Westborough
Job summaryAre you inspired by invention? Is problem solving through teamwork in your DNA? Do you like the idea of seeing how your work impacts the bigger picture? Answer yes to any of these and you’ll fit right in here at Amazon Robotics. We are a smart team of doers who work passionately to apply cutting edge advances in robotics and software to solve real-world challenges that will transform our customers’ experiences. We invent new improvements every day. We are Amazon Robotics and we will give you the tools and support you need to invent with us in ways that are rewarding, fulfilling, and fun. Amazon Robotics empowers a smarter, faster, more consistent customer experience through automation. Amazon Robotics automates fulfillment center operations using various methods of robotic technology including autonomous mobile robots, sophisticated control software, language perception, power management, computer vision, depth sensing, machine learning, object recognition, and semantic understanding of commands. Amazon Robotics has a dedicated focus on research and development to continuously explore new opportunities to extend its product lines into new areas. Amazon Robotics internship/co-op/full-time opportunities will be based out of the Greater Boston Area in our two state-of-the-art facilities in Westborough, MA and North Reading, MA. Both campuses provide a unique opportunity to have direct access to robotics testing labs and manufacturing facilities.Key job responsibilitiesAs an Applied Scientist intern, you will work on projects related to autonomous coordinated path planning and problems in robotic field mapping. You will research the latest Academic literature in coordinated planning, brain storm different approaches with Applied and Research Scientists to solve real-world problems, and translate business and functional requirements into quick prototypes or proofs of concept.As an Applied Scientist intern, you will work from concept through to execution. This role will give you the opportunity to build tools and support services needed to analyze data, dive deep to resolve root cause of systems errors and changes, and present findings to business partners to drive improvements.About the teamThe Navigation Technologies team at Amazon Robotics is seeking interns with a passion for robotic research to work on cutting edge algorithms for robotics. Our team works on challenging and high-impact projects, including robotic field mapping, Multi-Agent coordinated planning of large fleets of robots, and complex navigation data services like obstacle detection, congestion detection and route estimations for large and complex Amazon Robotic sites. We are seeking internship candidates with backgrounds in computer vision, machine learning, path planning, simulation, discrete optimization, and robotics. You will be joining Research Scientists, Applied Scientists and software engineers working on challenging problems.
Applied Scientist II, Alexa Sensitive Content Intelligence (ASCI)
IN, KA, Bangalore
Job summaryAlexa is the voice activated digital assistant powering devices like Amazon Echo, Echo Dot, Echo Show,and Fire TV, which are at the forefront of this latest technology wave.To preserve our customers’ experience and trust, the Alexa Sensitive Content Intelligence (ASCI) team creates policies and builds services and tools through Machine Learning techniques to detect and mitigate sensitive content across Alexa.An Applied Scientist will be working with a team of exceptional scientists to develop novel algorithms and modeling techniques to advance the state of the art in NLP related tasks.You will work in a hybrid, fast-paced organization where scientists, engineers, and product managers work together to build customer facing experiences.You will collaborate with other scientists to raise the bar of scientific research in Amazon.Your work will directly impact our customers in the form of products and services that make use of speech,language, and computer vision technologiesKey job responsibilitiesKey job responsibilities-You'll contribute to the science solution design, run experiments, research new algorithms, and find new ways of optimizing customer experience-Besides theoretical analysis and innovation, you will work closely with talented engineers and ML scientists to put your algorithms and models into practice-Your work will directly impact the trust customers place in Alexa, globallyA day in the lifeYou will be working with a group of talented scientists on researching algorithm and running experiments to test scientific proposal/solutions to improve our sensitive contents detection and mitigation.This will involve collaboration with partner teams including engineering,PMs,data annotators,and other scientists to discuss data quality, policy, and model development.You will mentor other scientists,review and guide their work,help develop roadmaps for the team.You work closely with partner teams across Alexa to deliver platform features that require cross-team leadership.About the teamThe mission of the Alexa Sensitive Content Intelligence (ASCI) team is to (1) minimize negative surprises to customers caused by sensitive content(2) detect and prevent potential brand-damaging interactions(3) build customer trust through appropriate interactions on sensitive topics.The term “sensitive content” includes within its scope a wide range of categories of content such as offensive content (e.g. hate speech, racist speech), profanity, content that is suitable only for certain age groups, politically polarizing content, and religiously polarizing content.
Applied Science Manager, Alexa AI, Alexa AI
US, WA, Bellevue
Job summaryThe Alexa Artificial Intelligence (AI) organization has a mission to invent AI capabilities to complete customer interactions naturally and without any friction, anytime, anywhere. In Alexa AI, we build world-class conversational reasoning that enables Alexa to anticipate and accomplish customers’ tasks through context-aware, multi-turn, and multi-skill experiences. Key job responsibilitiesWe are looking for a passionate, talented, and resourceful Applied Science Manager in the field of Natural Language Processing (NLP) or Dialogue Management, to lead the science efforts in developing novel algorithms and modeling techniques, and advance the state of the art in spoken language understanding. The ideal candidate will also:Build a strong and coherent team with particular focus on automated ranking and arbitration, sciences, and innovationServe as a technical lead on demanding and cross-team projects, and effectively collaborating with multiple cross-organizational teamsApply technical influence on partner teams, increasing their productivity by sharing your deep knowledge
Senior Applied Scientist, Code Science and Intelligence
US, WA, Seattle
Job summaryThe AWS Code Science and Intelligence team supports building innovative developer-focused products, frameworks, SDKs and services. Your role will be to help bring this new product to life as one of the founding members of this team.As an applied scientist on our team, your role is to leverage your strong background in Computer Science and Machine Learning to help build our model development and assessment pipeline, harness and explain rich data at AWS scale, and provide automated insights to improve machine learning solution that will impact millions of developers every day. This role requires a pragmatic technical leader comfortable with ambiguity, capable of summarizing complex data and models through clear visual and written explanations. The ideal candidate will have experience with machine learning models and information retrieval system. We are particularly interested in experience applying natural language processing, deep learning, and reinforcement learning at scale. Additionally, we are seeking candidates with strong rigor in applied sciences and engineering, creativity, curiosity, and great judgment.Inclusion Here at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences. Amazon’s culture of inclusion is reinforced within our 14 Leadership Principles, which remind team members to seek diverse perspectives, learn and be curious, and earn trust. Work/Life Balance Our team puts a high value on work-live balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives. Mentorship & Career Growth Our team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future.About Us AWS has the most services and more features within those services, than any other cloud provider–from infrastructure technologies like compute, storage, and databases–to emerging technologies, such as machine learning and artificial intelligence, data lakes and analytics, and Internet of Things. Whether its Identity features such as access management and sign on, cryptography, console, builder & developer tools, and even projects like automating all of our contractual billing systems, AWS Platform is always innovating with the customer in mind. The AWS Platform team sustains over 750 million transactions per second. Learn more about Amazon on our Day 1 Blog: https://blog.aboutamazon.com/
Sr. Manager, Economics, AWS Marketing Science
US, Virtual
Job summaryAmazon Web Services (AWS) is building a world-class marketing organization, and we are looking for a Senior Manager, Economics to join Marketing Data: Science & Engineering (D:SE) organization to lead AWS Measurement, Valuation and experimentation programs and initiatives across AWS Marketing, and own mechanisms to raise the science and measurement in AWS Marketing. As a leader of the Marketing Data: Science & Engineering (D:SE) team you will lead a team economists, scientists and engineers, and partner with other science, economics, product and business teams across AWS Marketing to build the next generation marketing measurement, valuation and machine learning capabilities directly leading to improvements in our key performance metrics.This role can sit anywhere in the United States, remotely. Key job responsibilitiesLead a team across technical job families (economists, scientists, engineers)Shape the thinking of measurement and valuation of the AWS Marketing organization with senior leaders across AWS and AmazonDrive the adoption of measurement, attribution and valuation products across AWS Marketing at senior level, and impact AWS Marketing investment allocation decisionsOwn a portfolio of science-based products (e.g. multi-touch attribution, downstream impact and life-time value) and scale them broadly to support stakeholders worldwideManage a research agenda that balances short term deliverables with measurable business impact with long term, high risk / high reward projectsProvide technical and scientific guidance to team membersPartner with academics to advance research in causal modelingAbout the teamWe refuse to accept constraints, internal or external, and have a strong bias for action. We love data and believe that we can use it to deliver epic experiences for our millions of prospect customers. We work across all areas of AWS Marketing including core marketing data solutions, insights and reporting, targeting and personalization, measurement, and the operational systems to support each of these areas. As a multi-functional team of experts, we deliver scaled solutions that are used globally across AWS Marketing.
Senior Research Scientist for AWS Worldwide Specialist Organization (WWSO), AWS Finance, Analytics and Science Team
CA, BC, Vancouver
Job summaryThis role can be based in Seattle, WA, Herndon, VA, Jersey City, NJ, Culver City, CA, or San Francisco, CA,.Join the Amazon Web Services (AWS) Finance, Analytics, and Science Team! AWS is one of Amazon’s fastest growing businesses, serving millions of customers in more than 190 countries. We are reshaping the way global enterprises use information technology and we are powering the next generation of global business leaders. We are looking for entrepreneurial, analytical, creative, flexible leaders to help us redefine the information technology industry. If you want to join a fast-paced, innovative team that is making history, this is the place for you.As a Research Scientist, you will be expected to work directly with senior management on key business problems. Scientists at Amazon use statistical and ML models to address complex problems, design and test hypotheses, and contribute to the design of automated systems. We are looking for a results-driven scientist to join our team in Seattle. The right candidate will work closely with other scientists, business leaders, business intelligence engineers, and data engineers.The key strategic objectives for this role include:· Work with the AWS Worldwide Specialist Organization (WWSO) to provide data- and model-driven guidance and recommendations on business questions posed by leadership.· Identify opportunities to measure and improve productivity with the end goal of driving growth.· Conduct, direct, and coordinate all phases of data science projects, demonstrating skill in defining key research questions, working with multiple data sources, evaluating competing models, and interpreting and communicating results.
Research Scientist, Sales Insights, Analytics, Data Engineering & Science (SIADS)
US, WA, Seattle
Job summaryAmazon is seeking an experienced, self-directed Research scientist to support the research and analytical needs of Amazon Web Services' Sales teams. This is a unique opportunity to invent new ways of leveraging our large, complex data streams to automate sales efforts and to accelerate our customers' journey to the cloud. This is a high-visibility role with significant impact potential.About you:You, as the right candidate, are adept at executing every stage of the machine learning development life cycle in a business setting; from initial requirements gathering to through final model deployment, including adoption measurement and improvement. You will be working with large volumes of structured and unstructured data spread across multiple databases and can design and implement data pipelines to clean and merge these data for research and modeling.Beyond mathematical understanding, you have a deep intuition for machine learning algorithms that allows you to translate business problems into the right machine learning, data science, and/or statistical solutions. You’re able to pick up and grasp new research and identify applications or extensions within the team. You’re talented at communicating your results clearly to business owners in concise, non-technical language. What you will do• Work with a team of analytics & insights leads, scientists and engineers to define business problems.• Research, develop, and deliver machine learning & statistical solutions in close partnership with end users, other science and engineering teams, and business stakeholders.• Use AWS services like SageMaker to deploy scalable ML models in the cloud.• Examples of projects include modeling usage of AWS services to optimize sales planning, recommending sales plays based on historical patterns, and building a sales-facing alert system using anomaly detection.Inclusive Team CultureHere at AWS, we embrace our differences. We are committed to furthering our culture of inclusion. We have ten employee-led affinity groups, reaching 40,000 employees in over 190 chapters globally. We have innovative benefit offerings, and host annual and ongoing learning experiences, including our Conversations on Race and Ethnicity (CORE) and AmazeCon (gender diversity) conferences.Work/Life BalanceOur team puts a high value on work-life balance. It isn’t about how many hours you spend at home or at work; it’s about the flow you establish that brings energy to both parts of your life. We believe striking the right balance between your personal and professional life is critical to life-long happiness and fulfillment. We offer flexibility in working hours and encourage you to find your own balance between your work and personal lives.Mentorship & Career GrowthOur team is dedicated to supporting new members. We have a broad mix of experience levels and tenures, and we’re building an environment that celebrates knowledge sharing and mentorship. Our senior members enjoy one-on-one mentoring and thorough, but kind, code reviews. We care about your career growth and strive to assign projects based on what will help each team member develop into a better-rounded engineer and enable them to take on more complex tasks in the future. About the teamAmazon is seeking an experienced, self-directed research scientist to support the research and analytical needs of Amazon Web Services' Sales teams. This is a unique opportunity to invent new ways of leveraging our large, complex data streams to automate sales efforts and to accelerate our customers' journey to the cloud. This is a high-visibility role with significant impact potential.About AWS:Amazon Web Services (AWS) provides companies of all sizes with an infrastructure web services platform in the cloud (“cloud computing”). With AWS you can requisition compute power, storage, and many other services – gaining access to a suite of elastic IT infrastructure services as your business demands them. AWS is the leading platform for designing and developing applications for the cloud and is growing rapidly with hundreds of thousands of companies in over 190 countries on the platform.