How to use Live Text in macOS Monterey

One of macOS Monterey’s coolest new features in Live Text. As Apple says, “it uses on-device machine learning to detect text in photos, including phone numbers, websites, addresses, and tracking numbers, so users can copy and paste, make a phone call, open a website, and easily find more information.” 

Think of it as a kind of OCR (optical character recognition) on steroids. Also think of it as an M.x processor equipped Mac feature — cause it is. Sorry, users of Intel-based Macs.

Along the same lines is Visual Lookup. It also uses machine learning to help users discover and learn about animals, art, landmarks, plants, and more in photos. These features work across macOS, including in apps like Photos, Messages, and Safari.

First, let’s look at Live Text

The feature works with Safari, Photos, Screenshots, and Quick Look (and perhaps more have gained compatibility since I wrote this). If you’re viewing an image that includes text in such apps, move your cursor over the text. The text selection tool appears. 

Now you can select the text, copy it and then paste it into a document. You can also “grab” the text in the an image, then drag ‘n drop it in another app.

You can double-click to select the word or triple-click to select the whole sentence. Or drag the cursor over the entire paragraph of text to make your selection.

Once the text is selected, you can right-click or Control-click to bring up the context menu. This offers access to options such as Look Up, Translate, Search with Google, Copy and Look Up.

Live Text can “recognize” and differentiate between text such as phone numbers, addresses, web links, formatted text, etc. and let you apply actions to the text. For example, with Live Text you can:

° Tap on a website link to launch Safari.

° Click on an address to add it to Contacts. Or open it in Maps.

° Click on a phone number to launch a FaceTime call.

° Add it in a Quick Note.

An interesting aside about the phone number feature: Live Text offers a “Large Type” option that displays the number in a big overlay. This helps folks with vision problems with accessing the macOS Accessibility settings. (It also helps folks with aging eyes like Yours Truly).

Let’s look at one example of using Live Text. My favorite way of using it is with Safari. 

When you’re using the web browser and are viewing a photo or image with text in it, hover your mouse pointer over the text. Select the text to copy it. Since this is a new feature, it may not with all web pages. 

You can also — among other options — choose to share the text or have your Mac read it to you as shown by the image above.