Newsreel KW 03/2024 and Podcast #041: 🤖 Google AI doctor 🤓 Vision Pro experience reports 📱 Apple No. 1 smartphone seller

Newsreel Podcast

We say goodbye to you with this podcast and are happy to tell you our reasons. Then we talk about the Vision Pro reviews, why the keyboard in visionOS is junk, how you can let friends play with the Vision Pro and why the Apple headset might come to Germany before the summer.

We also have a report about an AI doctor from Google up our sleeve and we are pleased that Apple will finally send a serious opponent for ChatGPT into the ring with its multimodal language model Ferret. That and much more in this episode.

For people who don't like to listen and prefer to read, we have prepared the individual topics in bullet points further down on this page.

At this point you will also find all the show notes, sources and comments that we have put together for the individual topics below.

This episode as a YouTube video with Lynne and Jens

Would you like to see us podcasting again? Then here is the current episode for you on YouTube:

As always, we look forward to your comments and thumbs up on YouTube. 😊

Our Apple podcast has finally made it to Apple Podcasts.

Note: The summaries here were created using ChatGPT, but in our podcast we talk without artificial intelligence. So if you would like background information and good entertainment about the news, then please listen to us!

Apple filed a patent for a VR headset back in 2008 that looks very similar to the Vision Pro
https://www.macrumors.com/2008/11/07/apple-researching-virtual-reality-headsets/

  • Apple was already researching personal virtual reality headsets in 2008 (February 2008 patent application).
  • Patent describes “personal viewing system” for cinema experience.
  • Device adjusts image based on user's head movements.
  • funny comments on Macrumors 16 years ago:
    • Just looking at that thing gives me a headache!
    • The technology has so much potential, but I wonder why they limit it to just the cinematic experience. You could also make programs and games with it.
    • Interesting and unusual. I don't expect that to happen any time soon. Apple knows people will feel silly wearing something like this.
    • Addendum: People also felt silly when they started wearing Bluetooth headphones.


Analyst Kuo expects the Vision Pro to launch internationally before WWDC 2024
https://www.macrumors.com/2024/01/16/vision-pro-to-launch-internationally-before-wwdc/

  • Main content: Apple plans to launch the Vision Pro headset internationally before WWDC 2024
  • WWDC is always at the beginning of June
  • Analyst: Ming Chi Kuo
  • Goal: Promote a “global development system” for visionOS
  • Reasons for initial US exclusivity: Limited availability, smooth sales process, software adjustments for other countries
  • WWDC has been held at the beginning of June every year since 2007
  • Possible next markets: UK, Canada, China
  • US launch: 80.000 units at launch, expects quick sell-out and long delivery times
  • US pre-order date: January 19th, launch February 2nd
  • Vision Pro features: Dual 4K displays, gesture tracking, M2 chip, price: $3.499


The first user reports about the Apple VisionPro are arriving
https://www.theverge.com/24040075/apple-vision-pro-hands-on-virtual-reality

  • various testers received 30 minute demos
  • Victoria Song (wearables and healthcare reporter at The Verge), Jamie Camil (Mexican actor and singer)
  • Impressions: Similar to other VR headsets, but with typical Apple design
  • Quote: I know what I saw, but I'm still trying to figure out where the headset fits in real life.
  • Experience report: looks like other VR headsets, only Apple-like, it ruins your hairstyle
  • Problems distinguishing between single and double taps / single tap needs to be light so it doesn't get recognized as a double tap
  • Forgot to wink because the displays were so impressive
  • all impressive experiences and all similar to the Quest, but sharper, faster and more awesome
  • Motion sickness when watching 3D videos with the camera being carried around
  • strange feeling when you wear the headset but don't know exactly what it shows to others on the outside: eyes, shimmer, blue mist around the eyes, camera shutter
  • At the end: There were wrinkles on my forehead and a slight headache due to high concentration, but that quickly disappeared
  • Features: iPhone face scanner for light seal, eye tracking setup, visionOS with app launcher
  • Special features: Digital Crown for realigning the home screen, button for spatial photos and videos
  • Performance: Fast and precise eye tracking, ability to enter text using eye movements and finger gestures
  • Screen: Two 4K screens, 23 million pixels
  • Virtual Environment: Higher resolution than Meta Quest, M2 based computer
  • Social aspects: Display on the front of the headset shows the wearer's eyes
  • Usage experience: Impressive, but with initial adjustment difficulties
  • Comparison: Sony's PSVR 2 launches on February 22, 2024 for $549

Virtual keyboard in the Vision Pro is a crutch
https://appletechnikblog.com/de/2024/01/15/vision-pro-virtuelles-keyboard-von-visionos-in-version-1-ist-eine-katastrophe/

  • Main point of criticism: The virtual keyboard works poorly for productive work
  • Criticism from Mark Gurman: The keyboard in version 1.0 is inadequate for efficient typing
  • Gurman recommends using a Bluetooth keyboard
  • Risk: Without improving the keyboard, the Vision Pro could remain just an expensive gadget for gaming and 3D home cinema
  • Apple emphasizes the use of the headset for “spatial computing” and productive work
  • visionOS offers visual highlighting of potential next buttons and Siri support

Facial scan with Face ID is a prerequisite for pre-ordering the Vision Pro
https://www.engadget.com/apple-vision-pro-pre-orders-require-a-faceid-scan-with-an-iphone-or-ipad-171544636.html

  • Necessity: Face scan with Face ID on iPhone or iPad for accurate headset fit
  • Availability: On apple.com, in the Apple Store app and in Apple Stores
  • Condition: Latest version of Apple Store Face Scan app
  • Option for glasses wearers: Adding optical inserts in the headset for an additional charge, provider Zeiss
  • Required for glasses inserts: Current, official glasses prescription when pre-ordering
  • Availability for in-store demos: From launch date
  • First presentation: Apple’s WWDC 2023 Keynote
  • Features: Sharp 3D visuals, VR screen and speaker system, use as a social and gaming tool, space for developers to create apps

Vision Pro could eventually be usable with a pen
https://www.imore.com/vision-pro/this-patent-could-have-vision-pro-owners-writing-and-drawing-in-thin-air-with-a-futuristic-apple-pencil

  • Topic: New patent for Apple Pencil for use with Apple Vision Pro
  • confirmed on January 11.01.2024, XNUMX
  • Apple Pencil function: Allows you to write and draw in 3D space
  • Mini scroll wheel and haptic feedback possible
  • Uses: Creation, gaming, working with Mac apps in 3D
  • Patent Application: US Patent Application 20240012496
  • Relevance: Focus on artists and notes in 3D
  • Note: Not every Apple patent becomes an available product

Vision Pro also allows guest profiles for your friends
https://www.theverge.com/2024/1/16/24039995/apple-vision-pro-guest-mode-friends

  • The Apple Vision Pro will offer a “guest user” option, allowing friends and family members to try out the device without Optical ID having to use.
  • Apple hasn't explicitly mentioned whether guests can bypass the Optic ID requirement, which unlocks the device through iris scanning.
  • Guests can access certain app content, such as large photo attachments in messages, while the Settings app is open.
  • While a guest is using the headset, Apple ID settings, Optic ID, password, Apple Pay, Persona, and health data are restricted.
  • Apple has only now confirmed this feature, although evidence of a “guest mode” was found in the visionOS software development kit last year.
  • Apple also confirmed rumors that there will be a travel mode to stabilize the visualization when used on a plane.

These streaming services are coming to the Apple Vision Pro at launch
https://www.apfellike.com/2024/01/apple-verkuendet-diese-streaming-apps-sind-zum-start-auf-der-vision-pro-vertreten/

  • Apple has announced details of the streaming services that will be available when the Vision Pro, a pair of smart glasses, is launched.
  • The initial offering is primarily aimed at the US market.
  • Available streaming platforms include ESPN, MLB, PGA Tour, Max, Discovery+, Amazon Prime Video, Paramount+, Peacock, Pluto TV, Tubi, Fubo, Crunchyroll, Red Bull TV, IMAX, TikTok and MUBI. → no Netflix
  • The CEO of Disney is quoted as saying that they are working on new forms of entertainment and information.
  • Additional media content can be accessed via the Vision Pro's web browser.
  • The Vision Pro supports both 2D and 3D content and has a battery life of around two and a half hours when using a 2D video interface.
  • The Vision Pro will launch in the US on February 2nd, with pre-orders available starting next Friday.

Apple disables blood oxygen measurement in Apple Watch due to patent dispute
https://9to5mac.com/2024/01/15/apple-watch-blood-oxygen-feature-remove-ban/

  • US Customs has ruled: Apple Watch Series 9 and Ultra 2 do not infringe Masimo patents if they do not include pulse oximetry (blood oxygen measurement).
  • Apple plans to remove the pulse oximetry feature in newly sold devices in the US.
  • Update: For now, Series 9 and Ultra 2 with blood oxygen monitoring remain available while US Court of Appeals decides on stay pending ITC appeal.
  • The decision only affects sales in the USA; existing Apple Watches with pulse oximetry function are not affected.
  • Details of Masimo and Apple's actions are limited because Apple maintains confidentiality.
  • US appeals court is still deciding whether ban on Apple Watch sales will be stayed throughout the appeal process.

Apple must allow payments for in-app buyers outside of the App Store
https://www.bloomberg.com/news/articles/2024-01-16/supreme-court-rejects-apple-s-request-for-epic-app-store-review
https://www.sir-apfelot.de/us-app-store-app-und-in-app-kaeufe-nun-auch-extern-moeglich-mit-vielen-huerden-57118/

  • US Supreme Court rejects Apple's appeal in antitrust case against Epic Games
  • Apple plans to allow external payment options in the US App Store
  • Developers can link to an external website for in-app purchases to bypass Apple's payment system
  • Apple's current commission: 15% or 30%; Planned commission for external payments: 12% or 27%
  • Background: Ruling from the 9th US Circuit Court of Appeals confirms that Apple did not violate federal antitrust laws
  • Financial aspects: In-app spending is estimated at $182 billion in 2024 and $207 billion in 2025
  • Epic Games Reaction: CEO Tim Sweeney Criticizes Apple's Planned 27% Fee on Web Transactions, Calls Customer Warning a 'Scare Screen'
  • Global pressure on Apple: EU competition authorities have two antitrust cases pending against Apple
  • Procedure for developers: StoreKit External Purchase Link Entitlement application, integrate entitlement profile into Xcode, check external link and the website behind it, warning message for users, 15 days after the end of the month a report on the previous month's income must be sent to Apple - with reservation Test

EU Competition Commissioner reminds Tim Cook of the sideloading requirement and the deadline for the changeover
https://www.macrumors.com/2024/01/15/app-store-to-be-split-in-two/

  • Apple is preparing to split the App Store in two in the coming weeks to comply with European Union (EU) requirements that will force Apple to allow sideloading of apps.
  • These changes are in response to the upcoming entry into force of the EU's Digital Markets Act (DMA).
  • Apple has until March 7 to implement the changes, giving the company just over seven weeks.
  • Last week, Apple CEO Tim Cook met with EU Competition Commissioner Margrethe Vestager at Apple Park.
  • Vestager reminded Cook of Apple's upcoming commitment to allow users to install third-party app stores and sideload apps under the DMA.
  • Apple will also be required to allow developers to promote their offerings outside of the App Store and use third-party payment systems.
  • The DMA is expected to force Apple to make significant changes to how the App Store, FaceTime and Siri work in Europe.

Apple allows developers to process in-app payments outside of the App Store - with almost the same fees to Apple
https://9to5mac.com/2024/01/16/apple-revises-us-app-store-rules-to-let-developers-link-to-outside-payment-methods-but-it-will-still-charge-a-commission/

  • Apple has updated its App Store guidelines to allow developers to link to alternative payment methods, but only on the condition that the app also offers purchases through Apple's own in-app purchase system.
  • Developers can request permission to include links or buttons in their apps that direct users to out-of-app purchasing mechanisms.
  • These links to alternative payment platforms may only be displayed on a single page of the app that the end user visits and not as an intermediary image, modal or pop-up.
  • Apple continues to charge a commission for purchases made through alternative payment platforms. This commission is 12% for developers who are members of the App Store Small Business Program and 27% for other apps.
  • The commission applies to purchases made within seven days of tapping an external purchase link.
  • Developers must provide a periodic statement of qualifying out-of-app purchases and pay the applicable commission to Apple. Apple also has the right to audit developers' accounting records.
  • The links must go directly to the developer's website, without redirects or intermediate links, and must not contain any additional parameters in the URL.
  • Apple provides templates for developers to inform customers about alternative in-app payment systems, such as “For special offers, visit [x]” or “Buy for [$x.xx] at [x].”
  • These policy changes apply only to the US App Store. In all other stores, apps and their metadata cannot contain buttons, external links, or other calls to action that direct customers to alternative purchasing mechanisms.


In 2023, Apple will sell more smartphones than Samsung for the first time and will be number 1 worldwide
https://arstechnica.com/gadgets/2024/01/apple-hits-all-time-high-smartphone-market-share-takes-1-spot-for-2023/

  • Apple will reach a “historic high” in smartphone market share in 2023, taking first place for the first time.
  • Apple's market share is 20,1% for 2023, a growth of 3,7% compared to 2022.
  • Samsung is now in second place with a loss of 13,6% in 2023, giving it a market share of 19,4%.
  • The Chinese company Xiaomi recorded a decline of 4,7% and achieved a market share of 12,5%.
  • Oppo, the parent company of OnePlus, loses 9,9% and takes fourth place with 8,8% market share.
  • Transsion, a company operating in emerging markets such as Africa, achieved 30% growth from 2022 to 2023 and ranked fifth with 8,1% market share.
  • Apple's best-selling phone in 2023 was the iPhone 14 Pro Max at $1.100, followed by the iPhone 14 Pro at $1.000 and the iPhone 14 at $800.
  • The average selling price (ASP) for an Android smartphone in Q2023 250 was $949, while the average iPhone price was $XNUMX.
  • In 2020, Apple was in fourth place behind Samsung, Huawei and Xiaomi.
  • Apple's success is attributed in part to the increasing trend toward premium devices, which now make up over 20% of the market, supported by aggressive trade-in offers and interest-free financing plans.

Beeper Mini users complain because Apple would lock their Macs from iMessage
https://9to5google.com/2024/01/16/beeper-mac-ban-reports/

  • Beeper users report that Apple is blocking their Macs from iMessage.
  • Beeper Mini, launched in December, offered access to iMessage on Android but was quickly stopped by Apple.
  • Beeper users on Reddit are reporting issues with Macs being locked out of iMessage.
  • Some users will no longer be able to use iMessage, both in the Beeper app and in Apple's own iMessage app.
  • iPhones do not appear to be affected by the ban.
  • One user reported that Apple Support told him his account had been flagged for sending too many messages.
  • Apple is believed to be locking down the hardware IDs of Macs used to register the app with iMessage.
  • Beeper has not yet been able to comment.
  • Apple argued that Beeper Mini's method of accessing iMessage poses security and privacy risks.
  • Beeper Mini was also recently removed from the Play Store.

⠀

Google's new AI doctor is right more often than a real doctor
https://aimodels.substack.com/p/googles-new-llm-doctor-is-right-way
https://arxiv.org/pdf/2312.00164.pdf

  • Main point: A differential diagnosis AI system developed by Google
  • Differential diagnosis: A medical process in which doctors weigh various possible diagnoses to identify the specific cause of a patient's symptoms.
  • The following should be taken into account: symptoms, medical history and clinical examination results of the patient
  • Study: An LLM optimized for diagnostic reasoning was introduced and assessed in its ability to independently create a DDx or assist clinicians in doing so.
  • Method: 20 clinicians evaluated 302 challenging, real-world medical cases from the New England Journal of Medicine (NEJM) case reports.
  • Comparison: Clinicians were randomized to work with either search engines and standard medical resources or additional LLM support.
  • Results:
    • LLM alone achieved a top-10 accuracy of 59,1% versus 33,6% for unassisted clinicians.
    • Clinicians with LLM assistance demonstrated higher DDx quality (top 10 accuracy 51,7%) compared to clinicians without LLM assistance (36,1%) and clinicians with search assistance (44,4%).
    • LLM assisted clinicians in creating more comprehensive differential diagnosis lists.
    • Top 10 Accuracy: Clinicians and LLMs were able to provide a list of possible diagnoses and the Top 10 Accuracy tells how often the correct diagnosis was in the first 10 suggestions

Google Chrome's Incognito Mode isn't as incognito as some thought
https://arstechnica.com/tech-policy/2024/01/chrome-updates-incognito-warning-to-admit-google-tracks-users-in-private-mode/

  • Google updates Chrome's Incognito Mode warning.
  • New warning says Google and other websites continue to collect data, even in semi-private mode.
  • The change comes as part of a settlement in a class action lawsuit against Google.
  • New warning already available in Chrome Canary, the developer version.
  • Older version of Incognito Mode said: “You browse privately”, new version says: “You browse more privately”.
  • New warning emphasizes that Incognito Mode does not prevent websites and Google from collecting data.
  • The change has not yet been implemented in the stable versions of Chrome.
  • Google plans to add the warning to the stable Chrome channel, but the timing is still unclear.
  • The lawsuit began in June 2020 and the settlement was announced at the end of December 2023.
  • Additional information: Firefox in the private window: “Private window: Firefox empties the entered search terms and visited websites when all private windows have been closed. That doesn’t make you anonymous.”


Google is laying off hundreds of employees from its advertising sales team – first major wave of layoffs due to AI
https://arstechnica.com/google/2024/01/google-lays-off-hundreds-more-as-ad-division-switches-to-ai-powered-sales/

  • Google has laid off “hundreds” of employees from its ad sales department.
  • The layoffs mainly affect the “Large Customer Sales” team, which looks after large advertising customers.
  • The reason for the layoffs is the introduction of AI-driven sales tools in Google Ads.
  • AI tools, such as a natural language chatbot and a system that independently creates promotional materials, replace human labor.
  • The AI-driven advertising system is part of the “Performance Max” product, which optimizes based on CTR and conversion rate.
  • It's one of the first waves of AI-related job cuts in the industry.
  • There have been several waves of layoffs at Google recently in various departments.

Elon Musik wants more control over Tesla and gives an ultimatum
https://arstechnica.com/cars/2024/01/elon-musk-gives-tesla-ultimatum-another-12-of-shares-or-no-ai-robotics/

  • Elon Musk gives Tesla an ultimatum for an additional 12% of the company's shares.
  • Without these shares, he does not want to focus on AI and robotics at Tesla.
  • Musk currently owns 13% of Tesla and would like to get 25%.
  • Tesla generates 95% of its revenue from automobile sales.
  • Musk hints at developing his AI products outside of Tesla if he doesn't get the shares.
  • He proposes a similar share structure to Facebook, but this would be illegal after the IPO.
  • Tesla shares have lost 25% of their value in the last six months.
  • Musk is also CEO of SpaceX and runs his own social network.
  • A court case in Delaware is currently dealing with Musk's $56 billion compensation package from Tesla.
  • Voice structure on Facebook:
    • Class A shares carry one vote per share
    • Class B shares have 10 votes per share - only Mark Zuckerberg and a few key figures in the company have this type of shares.
    • These dual-class shares are common in Silicon Valley to give founders control of their company, but at the same time they concentrate power in the hands of a few people and limit the rights of shareholders.

eBay pays $3 million in cyberstalking fine against couple
https://www.justice.gov/usao-ma/pr/ebay-inc-pay-3-million-connection-corporate-cyberstalking-campaign-targeting

  • eBay Inc. will pay $3 million in penalties for a cyberstalking campaign against a Massachusetts couple in August 2019.
  • The campaign consisted of harassment and intimidation in response to the couple's reporting on eBay.
  • Anonymous and disturbing deliveries to the victims' home, including a book about surviving the death of a spouse, a bloody pig mask, a dead pig fetus, a funeral wreath and live insects.
  • Sending private Twitter messages and public tweets criticizing the content of the victims' newsletter and threatening to visit the victims in Natick.
  • Traveling to Natick to monitor the victims and install a GPS tracking device on their car.
  • Classified ads that invited strangers to sexual encounters in the victim's house.
  • eBay has been charged with stalking, tampering with witnesses and obstruction of justice and has accepted a deferred criminal trial.
  • The campaign included anonymous threatening deliveries, Twitter messages and surveillance of the couple.
  • Seven former eBay employees were sentenced, including Jim Baugh, to 57 months in prison.
  • eBay must also hire an independent corporate compliance monitor for three years and improve its compliance program.

Apple patent suggests underwater iPhone
https://www.imore.com/iphone/apple-just-patented-an-iphone-you-can-use-underwater-unique-new-interface-could-power-a-simplified-version-of-ios

  • Apple has filed a patent for an iPhone with an “underwater user interface” (April 2021, published in early 2024).
  • The patent covers a simplified version of iOS that can be used underwater.
  • The Apple Watch Ultra 2 can be used as a dive computer up to 40 meters deep, but iPhones, including the iPhone 15, are currently only water-resistant, not waterproof.
  • The patent describes that the iPhone can detect when it is placed underwater and adjust the user interface accordingly.
  • The patent is 78 pages long and covers almost every aspect of iOS.
  • According to Apple, iOS is inefficient for use underwater and takes longer to operate than necessary and therefore requires too much energy.
  • Navigating through apps can be done using the volume buttons and selecting them using the lock button.
  • For specific apps, like the camera app, the volume buttons could be used instead of gestures for zoom controls.
  • This technology is not expected to appear in the iPhone 16 Pro Max this September, but Apple is considering future iPhones with simplified underwater functionality.

Apple makes its Ferret language model open source
https://ai.plainenglish.io/apple-enters-the-ai-race-with-open-source-ferret-llm-192db92b3535

  • Apple Open Source Ferret LLM: Apple has released and open-sourced Ferret, a new major language model.
  • Innovation in generative AI: Ferret is Apple's first big step into AI research and development.
  • Hybrid architecture: Ferret combines dense and sparse transformer blocks, providing accuracy and efficiency.
  • training data: Ferret was trained on 60GB of Wikipedia articles, 20GB of App Store reviews, 5GB of Apple developer documentation, and Siri requests.
  • High precision: Ferret achieves 87% accuracy in the SuperGLUE benchmark. (GPT-3 → 71,8% / GPT-4 → 89,5%)
  • The SuperGLUE Benchmark is an advanced test set that evaluates the language understanding capabilities of AI models, paying particular attention to aspects such as logic, knowledge understanding and context interpretation.
  • Efficiency: Ferret requires less computing power compared to similar models.
  • Fields of application: Ferret is suitable for text generation, answering questions, summarizing, translating and classifying texts.
  • Integration with Apple products: Possible integration of Ferret with Siri and other Apple services.
  • ethics and responsibility: Apple values ​​ethical development and responsible use of AI.

Preprint on Apple's multimodal language model Ferret and comparisons with GPT-4V
https://arxiv.org/pdf/2310.07704.pdf

  • Preprint means: published by authors but not yet cross-checked by other scientists or published in scientific newspapers
  • Ferret: A new, advanced Multimodal Large Language Model (MLLM).
  • Purpose: Understanding and localizing (grounding) spatial information in images.
  • versatility: Can refer to different image regions such as points, boxes or free shapes.
  • Hybrid representation: Combines precise coordinates and continuous image features.
  • Open vocabulary skills: Can respond to a variety of objects and situations in images.
  • Training with GRIT dataset: 1,1 million multimodal dialogs to improve the model.
  • Versatile applications: Suitable for tasks such as object classification, image description, and conversations that require image references.
  • Ferret Bench: A new benchmark that tests Ferret's abilities in various scenarios.
  • Improved accuracy: Surpasses other models in the accuracy of locating and describing image content.
  • Reduction of hallucinations: Reduces the problem of incorrect or fabricated answers.
  • Ferret:
    • Specializes in the precise referencing and location of objects in images.
    • Effective in handling various region shapes (dots, boxes, free shapes).
    • Strengths in multimodal conversations that affect specific image regions.
    • Reduces the risk of hallucinations in responses.
    • Very good at performing complex tasks requiring accurate image analysis and specific regional references.
  • GPT-4V:
    • Strengths in general question-answer areas and commonsense knowledge.
    • Effective in answering questions that go beyond the pure image content.
    • Can provide detailed and in-depth answers to questions about specific image regions.
    • Less precise than Ferret when locating small or complex image regions.
Did you like the article and did the instructions on the blog help you? Then I would be happy if you the blog via a Steady Membership would support.

Post a comment

Your e-mail address will not be published. Required fields are marked with * marked

In the Sir Apfelot Blog you will find advice, instructions and reviews on Apple products such as the iPhone, iPad, Apple Watch, AirPods, iMac, Mac Pro, Mac Mini and Mac Studio.