How Masters uses watsonx to manage the AI ​​lifecycle

At Masters®, time-honored tradition meets cutting-edge technology. Through more than 25 years of partnership, IBM has helped Augusta National Golf Club capture, analyze, distribute and use data to bring fans closer to the action, resulting in the AI-powered Masters digital experience and mobile app. Now fans can more fully enjoy the performances of the world’s best golfers at the most prestigious tournaments in sports, whether they are standing on the fairway or watching from home.

In a continuous design thinking process, the IBM consulting team and the club work together to improve the fan experience each year. New features for 2024 include Hole Insights, statistics and predictions for every shot for every player on every hole. Extended AI-generated narration (including Spanish) to over 20,000 highlight clips.

Masters has long relied on IBM to manage data, applications and workloads across on-premises servers and multiple clouds, but this year saw significant advancements. The entire AI lifecycle is managed on the AI ​​and data platform IBM® watsonx™.

data collection

The IBM watsonx platform includes watsonx.data, a purpose-built data store built on an open lakehouse architecture. This allows Masters to scale analytics and AI wherever their data resides, through open formats and integrations with existing databases and tools.

“The Masters’ data lake leverages eight years of data that reflects how the course has changed over time, while using only shot data captured with current ball tracking technology,” said Aaron Baughman, IBM Fellow, AI and Hybrid Cloud. .” Leader of IBM. “Hole distances and pin locations vary from round to round and year to year. These factors are important when preparing data.”

The historical sources that watsonx.data accesses consist of relational, object, and document databases, including IBM® Db2®, IBM® Cloudant, IBM Cloud® Object Storage, and PostgreSQL.

Finally, watsonx.data is pulled from a live feed. “We will get a variety of feeds from the system, including scoring, ball tracking, pin locations, player pairings and scheduling,” says Baughman. “We also take the video, add commentary, and insert it into the clip.”

Watsonx.data allows organizations to optimize their workloads for a variety of purposes. For Masters, “Consumer-facing data access is fronted by a CDN that caches resources to prevent traffic from reaching the origin server, while AI workflows call data directly from the origin to ensure it is as up-to-date as possible. ” says Baughman.

Prepare and annotate data

IBM watsonx.data helps organizations put data to work, curate and prepare data for use in AI models and applications. Masters uses watsonx.data to organize and structure data related to a tournament (course, round, hole), which can then be populated with real-time data as the tournament progresses. “We also have player factors, ball tracking information, and scoring information,” Baughman said. “If you can organize your data around that structure, you can efficiently query, retrieve, and use downstream information, such as AI narration.”

Watsonx.data uses machine learning (ML) applications to simulate data representing ball location predictions. “Using the data we have prepared, we can calculate the probability of a birdie or eagle in a particular category. You can also look across the fairway for contrasting statistics,” says Baughman.

AI model development and evaluation

IBM® watsonx.ai™ components from watsonx enable enterprise users to build AI applications faster, with less data, whether using generative AI or traditional ML.

“For the Masters, we use 290 traditional AI models to predict where the golf ball will land,” says Baughman. “Once the ball passes one of the predefined distance thresholds for the hole, it switches to the next model and eventually reaches the green. Additionally, there are four pin positions: front left, front right, rear left, and rear right, so a total of about 16 models are possible per hole. “It would be a huge challenge for humans to manage these models, so we use watsonx’s autoAI feature to help us build the right models and choose the best projections.”

Watsonx.ai also helped the digital team build a generative AI model for text generation as the basis for voiceover. This allows you to use watsonx.governance to evaluate output quality using metrics like ROUGE, METEOR, and embarrassment score, while using HAP guardrails to remove hateful, abusive, or profanity content.

“The tools in watsonx.governance are really helpful,” says Baughman. “We can track the model versions we use, promote them to validation, and finally deploy them to production once we are confident that all metrics have passed our quality estimates. Since this is a near-real-time system, we also measure response time. Watsonx.governance allows us to effectively manage and deploy all of these models.”

Train and test models

The Masters digital team used watsonx.ai to automate the creation of ML models used in Hole Insights based on eight years of data. For AI narration, we used a pre-trained large language model (LLM) with billions of parameters.

“We used a few learnings to help guide the model,” Baughman said. “Instead of fine-tuning the model through tournaments, we fine-tune the input statistics that go into the model. “This is a compromise that delivers the results we need while minimizing risk.”

Watsonx.governance also provides several LLMs that are used to validate data in the underlying model, for example to remove HAP content. “We have a lot of guardrails right down to regular expressions,” says Baughman. “Watsonx gives us the confidence that we can identify and mitigate HAP content in real time before it is published.”

Model Deployment and Management

After tuning and testing your ML or generative AI model, watsonx.ai provides a variety of ways to deploy it into production and evaluate your model within your deployment space. You can also evaluate model fairness, quality, and drift.

“We used Python scripts from watsonx to deploy our ML models to Watson Machine Learning, a set of Machine Learning REST APIs running on the IBM Cloud,” says Baughman. “We also have a container that loads the model in memory, so we run the model locally, so there is absolutely no network latency. We have both strategies. Typically, we run it in memory first and then, if problems arise, use the model deployed in Watson Machine Learning.”

The team uses a model deployed within watsonx.ai (where generation parameters can be managed) to deploy the LLM used for AI narration, and secondly, a model deployed to Watson Machine Learning via watsonx.governance. So we took a different approach. .

Model management and maintenance

Watsonx.governance provides automated monitoring of deployed ML and generative AI models and promotes transparent and explainable results. Users can set their risk tolerance and set alerts on various indicators.

“Watsonx.governance alerts you if your model fails in any dimension and allows you to easily fix it,” says Baughman. “You can also run experiments as needed, create AI use cases, and verify that it works as expected.” One such experiment is that after a round is over, the team has some evidence for that round that they can add to the model and re-validate, enabling continuous improvement and improved results.

88^Day The Masters Tournament will be held April 11-14 at Augusta National Golf Club in Augusta, Georgia. To see IBM technology in action, visit Masters.com or the Masters app on your mobile device, available in the Apple App Store and Google Play Store.

Learn how watsonx can help you manage your entire AI lifecycle.