Apple finally jumped into the ai game with Apple Intelligence, a new feature announced at WWDC 2024. The company promises a powerful, personal ai assistant that integrates into iPhones, iPads, and Macs. Here's a quick breakdown of the features key features.
General description
- Siri gets a major update with natural conversations, text input, and on-device intelligence.
- A powerful writing assistant rewrites, corrects and summarizes text in various applications.
- Say goodbye to email clutter with ai-powered priority messages, digests, and smart replies.
- Smarter notifications prioritize what's important and reduce distractions with focus modes.
- Notes and Phone apps get ai-powered transcription and call summaries.
- Enhance your creativity with Image Playground to design fun images and custom Genmoji.
- Easily search and edit photos using natural language and a new cleanup tool.
- Apple prioritizes user privacy with on-device processing and secure cloud computing.
- Apple is integrating ChatGPT access directly into iOS 18, iPadOS 18, and macOS Sequoia.
Siri with a new look
Apple Intelligence brings a significant update to Siri. Gone are the days of simple voice commands. Siri can now understand and respond to complex requests, even generating images based on their descriptions. Activating Siri now comes with a new visual style: a bright border that wraps around the screen. Here's how the new Siri can help you:
- More natural conversations: Siri can follow your train of thought, understand if you stumble, and remember context between requests.
- Write to Siri: You can choose to communicate with Siri by voice or text, whichever you prefer.
- Device Guide: Siri can answer your questions about how to use various features on iPhone, iPad, and Mac.
- Screen awareness: Siri can understand and act based on information displayed on your screen, such as adding an address from a message to a contact card.
- Take actions across all apps: Siri can perform hundreds of new actions within and across apps, such as searching for book recommendations mentioned in emails or messages.
- Personalized intelligence: Siri can access information on your device to provide personalized responses, such as playing a podcast recommended by a friend or looking up flight details for an upcoming trip.
All-new writing assistant
Apple Intelligence goes beyond simple voice commands. It features a robust writing assistant with “Writing Tools” built directly into several Apple apps, including Mail, Notes, Pages, and even third-party apps. This allows users to:
- Write again: Craft emails, cover letters, and resumes delicately. Apple Intelligence suggests different versions of your writing, adjusting the tone depending on the audience and task.
- Correct: Ensure grammatical accuracy, proper word choice, and sentence structure. It even proposes edits with explanations, allowing users to learn and improve their writing skills.
- Resume: Quickly capture key points from long texts. Choose to summarize the text as a paragraph, bullet points, a table, or a list.
Email management with Apple Intelligence
Apple Intelligence simplifies email management in Mail with several features:
- Priority messages: A dedicated section at the top of the inbox highlights urgent emails, like dinner invitations or boarding passes.
- Email summaries: Preview summaries instead of just the first few lines, allowing users to quickly decide which emails should be opened.
- Smart answer: Generate quick responses and identify questions in emails to ensure a complete response.
Smarter notifications
- Priority notifications: Displays the most important notifications at the top of the stack, saving users time by prioritizing what's most important.
- Notification summaries: Scan long or stacked notifications to see key details right on the lock screen, such as when a group chat is particularly active.
- Reduce interruptions: A new focus mode that shows only notifications that require immediate attention, such as a daycare pick-up reminder.
<h2 class="wp-block-heading" id="h-ai-powered-transcription-and-summarization”>ai-powered transcription and summary
The Notes and Phone apps now offer audio recording, transcription, and summarization:
- Call recordings: Participants are automatically notified when a call is recorded and summaries are generated to remind them of key points after the call ends.
Playground image
- Create fun images: Apple Intelligence powers Image Playground, allowing users to design unique images in seconds. Choose from Animation, Illustration, or Sketch styles.
- Built-in Accessibility: Image Playground is available in Messages and as a standalone app for maximum convenience. Experiment with various concepts and styles without limitations.
- Custom suggestions: In Messages, generate fun images relevant to your conversations. For example, if you're planning a hiking trip with friends, Image Playground suggests concepts related to your destination and activity.
- Improved notes: Integrate Image Playground with the Notes app using Image Wand in the Apple Pencil tools palette. Transform sketches into charming images or use the context of surrounding notes to create an image.
- Third Party Integration: Image Playground extends its reach to other apps through a dedicated API, allowing developers to integrate it into their apps.
Genmoji: Express yourself in a whole new way
Take emojis one step further with Genmoji. Simply type a description and a custom Genmoji will appear, along with additional options. Create Genmoji based on your photos or those of your friends and family. Use Genmoji online within messages or share them as stickers or reactions in a Tapback.
Improved photo searching and editing
- Natural language search: Search photos and videos effortlessly using natural language. Find specific moments describing the content, such as “Maya skateboarding in a tie-dyed shirt” or “Katie with stickers on her face.”
- Search within Videos: Locate specific moments in video clips, allowing you to go directly to the corresponding section.
- Cleaning tool: Identify and remove unwanted background objects in photos accurately, ensuring the subject remains unchanged.
- ai-Assisted Memories: Create a compelling narrative from your photos and videos. Simply provide a description and Apple Intelligence will select the best content, create a story based on identified themes, and organize them into a movie with its own narrative arc. Apple Music even suggests songs to complement your memories. All photo and video processing is done on the device, ensuring privacy.
Keep privacy at the forefront
Apple is emphasizing privacy with Apple Intelligence. The company says that user data will be processed on the device whenever possible and that any cloud processing will be done using Apple's secure “Private Cloud Compute” system. Users will also have full control over who can access their data.
ChatGPT + Apple: a powerful partnership
Apple is integrating ChatGPT access directly into iOS 18, iPadOS 18, and macOS Sequoia, allowing users to use its experience, including image and document understanding capabilities, without switching tools. Siri can now leverage ChatGPT's extensive knowledge to get more accurate and detailed responses. Users are asked for permission before submitting any questions, documents or photos to ChatGPT, ensuring privacy and control.
For example, an iPhone 15 Pro user can ask Siri for help planning a five-course meal with specific ingredients. Siri will confirm whether ChatGPT should be used and, once approved, provide detailed suggestions, such as a tomato and basil bruschetta recipe, presented seamlessly through Siri. This integration improves the user experience by offering practical and immediate solutions directly within the Apple ecosystem.
Apple Intelligence Availability
- Apple Intelligence will be available as a free beta this fall in US English.
- Additional features, software platforms and languages will be introduced over the next year.
- Initially available on iPhone 15 Pro, iPhone 15 Pro Max, iPad, and Mac devices with M1 chip or later.
- Requires Siri and device language to be set to US English for now.
Final note
So what do I think of Apple entering the world of ai? Let me know below in the comments section!
It's definitely interesting to see Apple enter the ai race. Apple Intelligence has the potential to be a powerful tool that can make our lives easier. It will be interesting to see how Apple Intelligence develops and how it compares to other ai assistants on the market.
