1  Real-Time ML: What and Why

This chapter covers

• Lorem ipsum dolor sit amet, consectetur adipiscing elit
• Ut enim ad minim veniam, quis nostrud exercitation
• Duis aute irure dolor in reprehenderit in voluptate
• Excepteur sint occaecat cupidatat non proident, sunt culpa
• Quis nostrud exercitation ullamco laboris nisi ut aliquip

---

Machine Learning

Machine learning (ML) is a collection of techniques to learn statistical patterns from data. This book assumes basic knowledge of ML.

1.1 What Is Real-Time ML

Real-time ML refers to cases where one performs inference with an ML model and uses the predictions immediately as part of come decision process.

Figure 1.1: legend-here

Let’s jump straight into examples:

• Credit Card Fraud Detection

  Detecting whether a given credit card operation is a fraud incident (e.g. leaked number, stolen card) is a classic use case of real-time ML.

Figure 1.2: legend-here

• Loan Origination

  The main component in deciding whether to extend a loan to a customer is the likelihood the loan will be paid back. The default risk is often calculated by an ML model, in real-time.

  

  Figure 1.3: legend-here

• Customer Service Chat Routing

  Although many use Generative AI for support, customers still often need to speak to humans. Routing the call or chat to the correct team reduces the waiting time and increases customer satisfaction.

  

  Figure 1.4: legend-here

These 3 examples display the basic pattern common to all ML-enabled real-time systems: a payload is built and passed on to a previously trained model, which outputs a prediction immediately (in the order of dozens or hundreds of milliseconds), which is then used to make some kind of decision.

But wait, aren’t all ML-enabled systems like this?

Not really. Most uses of ML are executed in batch, not in real-time

Table 1.1: Batch ML vs Real-time ML

	What is Scored	Latency
Batch ML	A dataset containing many rows	Minutes or Hours
Real-Time ML	A single instance	Milliseconds

1.2 Anatomy of a Real-Time ML System: Credit Card Fraud Detection

Let’s look in detail at the same main components of an ML-enabled real-time system.

1.3 What Real-Time ML Makes Possible

Modern organizations are mostly or even fully digital enterprises; there’s often no brick-and-mortar interface, only an web or a website.

This means that systems mediate every interaction and collect data from it. So one can apply RTML to almost all business flows.

1.4 How Real-Time ML Differs from Traditional Systems

1.5 The Game Changes After the First Few Models in Production

1.6 Summary