At WWDC 2024 Apple has showcased a number of software developments across all of their OS. They also spent a considerable amount of time talking about Apple Intelligence, which we’ll get to in a while.
For those who have been wondering: We have confirmation that Apple did indeed sign a partnership with OpenAI. We’ll talk about that later.
For now, let’s go through what was discussed.
VisionOS 2
VisionOS 2 aims to improve the Apple Vision Pro, including features like Spatial Photos, and the ability to bring out a screen by looking at a MacBook. Apple released APIs for developers to create more spatial content, as a way to create more interest in the Vision Pro.
Canon – yes Canon, the camera company – will be releasing a new lean to capture spatial content. That's surprising!
We might be getting spatial movies someday.
iOS 18
Here’s something most of you readers out there are excited to hear more about.
Home screen improvement. App icons have their own dark mode looks, and you can change their colors. iOS will recommend colors to match your wallpaper.
Control Center is now customizable and offers a wide range of controls across Apple apps. Once again, Apple is extending Control Center for developers, so that users can customize their favorite apps.
Apple introduces a lock app feature, allowing you to lend your phone with peace of mind. No more snooping around your friend’s Photos for anything naughty. Though, weirdly, I was quite sure Apple had a similar feature before. Unless it’s a third-party app? Apple has also introduced a feature that allows you to hide apps, so no one can just access them.
Pairing is being pushed to become simpler. Apple has released APIs for developers to swiftly connect your devices together without broadcasting them out in the open. Think about how AirPods can just pair with an iPhone so simply. Apple is seeking third-party developers for that functionality. They claim it’s for privacy, so no one can locate your device through Bluetooth.
Messages get text effects, where you can have fun with friends sending messages. Effects such as large texts, bouncing texts, exploding texts, it’s all there.
Messages now support Satellite SMS for iPhones 14 and up, and also introduces RCS support – finally?
Apple is also proud to announce an update to Mail by adding Categories. Like “Primary” where you can find all emails from the people you know. Not actually a new thing; but Apple has done the extra step here by adding other categories like Transactions, where all your email receipts end up, which is cool.
Apple Wallet introduces Tap to Cash: simply tap your iPhone to a friend's to send money. This is a feature I hope we can have more frequently in the country. Sending payments with a tap of the phone sounds really convenient and easier than scanning QR codes.
Photos gets a major redesign, which includes a filter, to help you search through your photo library easier. With Apple Intelligence, you can pull up images by describing them, which is as insane as it sounds.
AirPod Pro gestures and Apple TV
While wearing AirPods, you can simply nod or shake your head to accept or reject a call. Noise-cancelling functionality has been added to get rid of background noise.
For Apple TV, while watching a show, you can swipe up with the remote to see who the actor is on screen. Also, the song playing in the movie can be added to your playlist.
WatchOS 11
Training Load provides data to users allowing them to adjust their workouts avoiding overexertion. Vitals, on the other hand, shows various user metrics for a more comprehensive health analysis.
Health apps will now be able to provide information for pregnant users.
Depending on your location, new weather-related widgets may appear in your Apple Watch's Smart Stack.
New widgets may appear on the Watch’s smart stack now and depending on the conditions of your area, such as weather.
iPadOS
As in iOS, iPadOS also gets Control center customization, and the Photos feature. You can even have remote control access between iPhone and iPads.
iPad is getting the Calculator app that you can open with the Pencil.
Then there’s Math Notes, which is interesting. You can use the Apple Pencil to write math problems and Math Notes will solve it. You can even include Algebra and other complex math problems.
Notes get Smart Script, which smoothes out your handwriting. Through machine learning, you can paste text-based documents and have them converted to a style resembling your handwriting.
MacOS Sequoia
The name of the new MacOS, Sequoia, is powered with Apple Intelligence as well. Building on top of Keychain is Passwords, which stores all your passwords across your apps on all your devices.
It also comes with an iPhone mirroring, so you can access your iPhone from your Mac. You can send files wirelessly this way as well.
Apple Intelligence, Personal Intelligence
Apple starts owning the word “AI” by labeling it “Apple Intelligence.” And throughout the presentation, Apple has been using the words privacy and personal. They also explain how it must be integrated into your needs.
One of Apple Intelligence's features include notification prioritization, so you don’t get swamped with thousands of notifs. You will also get AI-powered writing tools and image generation, creating a more integrated creative experience.
Apple says you own your data. And all of Apple Intelligence use remains on-device. However, for some more complex tasks, Apple may tap into what they call Private Cloud Compute. This allows Apple Intelligence to conduct more complex actions, while offering privacy and security. Apple will only use Private Cloud Compute if it needs more power.
With Apple Intelligence, Apple is reintroducing Siri.
Siri has a new and colorful look. Every time you use Siri, a certain glow appears on your screen. Now you can talk to Siri and have it do some tasks, such as adding notes to your Calendar and searching for other information. Siri is also capable of maintaining conversation context, like, if you’re asking about a location, later you can tell Siri to add it to your notes, and Siri will remember what you were talking about.
On top of this, Apple is leveraging Siri's new features by letting it be the new manual or troubleshooter. Siri contains vast, if not all, information of all-things Apple, and you can just ask Siri for anything about Apple products.
Siri also has on-screen awareness, so if someone sends you information, you can tell Siri to store that information. If an address appears on the screen, you can tell Siri to save it, for instance. Siri also now has in-app actions, so you can tell Siri to copy/paste from your Notes, or you can have images sent to your Notes with ease.
As part of Apple Intelligence, Apple has introduced Rewrite, Proofread, Summary, and Smart Reply, along with Image Wand and Image Playground.
If you’re not confident or in a rush to draft something quick enough, Rewrite will be able to do it for you. You can choose from Professional or Concise, setting the tone of your message. Proofread, as the name suggests, proofreads your stuff.
Summary is used when your message is way too long. Summary will create a TL;DR on top of your message.
And Smart Reply will use the information on the screen to generate a response for you. For instance, you receive an email asking if you want to attend a party, how many friends you will be bringing and around what time you will be arriving. Smart Reply will generate a message while asking you those exact questions, so all you need to do is tap on the answers and let your device finish the letter for you.
Other AI features include genmoji! It sounds exactly as what it’s called. If you’re too lazy to pick one of the thousands of available emoji, you can generate your own emoji.
Image Playground, Apple’s image generation tool. Apple is using examples of sending friends digital cards through messages when using this tool, turning images of your friends into an animated form. It’s all for funsies.
Image Wand is used in Notes, where you can use your Apple Pencil to circle a sketch to generate an image based on that sketch. Even if you circle on a blank page, the device will gather context within Notes to generate the image.
OpenAI partnership
Apple says there could be other reasons why you may want to use other AI for other tasks. And, of course, there is that minor inconvenience where you have to switch tabs and all before you can use them. For Apple, Apple Intelligence is integrated with ChatGPT. If you ask Siri a question, Siri may suggest using ChatGPT to get more information. Siri will always ask for your permission first before accessing ChatGPT. Apple insists that no data is sent to OpenAI. While using Apple Intelligence, no one needs to pay for the ChatGPT experience.
Apple hinted at future updates that could bring more features. Still, what Apple has shown of its AI already looks quite impressive and fun. At least Apple is giving its users the choice to tap into ChatGPT when desired.
And so that wraps up our Apple WWDC coverage!