Not that I have any answers. But I do have lots of questions. And there are interesting tidbits of information out there, as we contemplate the inability of Apple Computer’s Siri to make any...
I think the problem is that LLMs are less than meets the eye. Apple got punked. LLMs are very good at impressing the uninformed and that includes upper management. As Cory Doctorow noted, they don't have to be good enough to do your job, they just have to be good enough to convince your boss that they can do your job.
It's very easy to imagine a v1 that was tuned to work on one battery of tests to the point where the engineering management involved was convinced that just another six to twelve months of tuning would get something working more generally. Unfortunately, programming LLMs is not like writing code. There are lots of people with a vague sense of how far along software is and how close to meeting goals. They're usually wrong in detail, but often close enough to estimate ship dates. The joke is that the best way to improve such skill is to multiply your first estimate by your age.
LLMs are another matter. These systems are not transparent and not robust. Remember when the big thing was tricking computer vision systems by putting small stickers on stop signs to convince them that a firetruck was blocking the way. LLMs are all too similar in operation, so v1 was a lot farther from release than anyone at Apple thought. They found out the hard way as so many others will.
Unlike traditional software, where you can design, build, and test modules so that they work and can be integrated into the OS, LLM/LRMs are very difficult to control. Think of them as children that you are trying to train to do various tasks. They learn some, then forget while doing something else, and then different tasks that seem similar confuse task completion, etc. How do you ensure reliability on an AI that has to be packed on an iPhone? We see the so-so reliability of huge LLM/LRM models, so it is easy to understand why a Siri-controlled system might prove unreliable.
It may be that traditional software engineering methods and development don't work when shifting to an "educational/teaching" method. Do we need a pedagogy for AIs?
A day late but here are my thoughts about what Apple is undertaking in trying to get Siri up to snuff and why it’s a hard problem to solve:
1) Apple doesn’t just want a reliable LLM interface with their products, such as what Microsoft is doing with copilot. They also want a reliable voice interface and that’s a whole other set of challenges that no leading AI company has solved yet.
2) Apple wants their AI enhanced Siri to work *offline* and that’s a significant challenge. Perhaps even more difficult than they were expecting.
2a) Using any generative AI at full capability right now requires an entire farm of servers with thousands of nvidia chips networked in just the right way with full access to reams of training data. That’s a lot of overhead to make every little query and comment work.
2b) You can run simplified AI models locally on your PC but it requires you to have an expensive nvidia GPU, often a part that consumes more than 300wats under load and has a physical footprint measured in tens of inches/centimeters. On a single chip with a smaller set of data, capabilities are extremely… modest.
2b1) I mention nvidia chips because nobody has an alternative to their chips now or in the immediate future. Apple’s chip designs are amazing but they were not developed for the work of generative AI. Running a local LLM on your iPhone just isn’t possible with apple’s current chip architecture and they can’t simply copy nvidia because, yes, patents but also because nvidia is not efficient enough for mobile, and because Apple doesn’t want to be dependent on anyone else.
3) the networking required to synchronize an LLM’s operations across a server farm and a local device with different chips and different capabilities and different latencies and data lost to cellular or WiFi conditions is a nontrivial challenge but Apple’s vision for AI Siri requires solving that too.
4) we haven’t even gotten to the software running Siri and all the little agentive interfaces into every app and program on your iPhone or your Mac.
So, Apple basically needs to be the first company to solve several different but related hard problems in order for Siri to function in the way they envision. They need an in-house AI chipset that is efficient enough for mobile because Siri needs to be able to do some “thinking” on your local device, potentially offline. That chipset, when online, needs to function well with the large server farms, something nobody else is trying to do at the moment afaik. There’s a reason all of the compute is being done in server farms and not distributed across everyone’s individual devices! Oh, and they’ve got to get a class-leasing voice interface off the ground so Siri properly understands people of varying accents, languages, and blood alcohol content.
I have no idea why anyone at Apple thought they would solve all of this in a matter of 12-18 months! I suppose they underestimated the challenges involved.
This whole thread, starting with the main post, has been enlightening. I'm an old-school programmer, skilled enough in 20th-century technology, but definitely ignorant of AI, which I studied briefly when "neural networks" were a fad on its way out because they just didn't perform. Wonderful, what can be done when cost-effectiveness improves by 7+ orders of magnitude!
But modern AI is a mystery, and how to manage software development now is an unexplored mystery. I appreciate the expertise I've seen here, not least in DeLong's essay.
Apple's software is no longer as far ahead of the industry as it once was, and their quality control problems with both iOS and MacOS have been documented for at least five years, so perhaps their development group is just failing with "AI".
Apple's fundamental research is still very good however. Their CPU group continues to leapfrog every competitor, and they may have finally broken Broadcom's monopoly on cellular modems. All that hardware requires extremely good firmware to reach its potential, and Apple has clearly invested in that and research to support it.
So maybe - just maybe - Apple's research group dug deep into the LLMs currently being touted as this half generation's "Artificial Intelligence" - and found nothing there. That the whole thing is is a fraud built on an illusion built on stolen intellectual property of others. Marketing won't listen and keeps pushing "Apple AI" but there is no there to be had...
Sorry, I do not want my driver’s license floating around in computer space for anyone to access.
I don’t use Siri. Really turned me off that I am supposed to say “Hey Siri” to turn on that feature. If I would not say that to a person, I would not say it to a computer either.
I don’t think it is a poor product just because it can’t recall every minute detail about my personal life.
I experienced the restaurant feature of Siri, and it did not work very well.
Have experienced another restaurant seeking program and it did not work well either.
I thought it was expecting way too much. Way too much of the programmers if this detail they are supposed to furnish. Way too much of restaurant managers if they are supposed to update all this information every day.
I think it is a mistake to program features like this for certain people who need this in their lives.
Just upgraded to Alexa Plus, and I'll never use Siri again until Apple gets this most basic feature right. An integrated assistant, a la Apple 1986 demoware, is too advanced for Apple to achieve now. Just give us a good verbal search assistant for now.
I am not particularly happy with the privacy on Alex, and Alex+ seems to require less privacy.
If we can run an LLM on a phone, I see no reason why Alex/+ shouldn't run locally, and only have access to your AMZN account when required.
Alexa has become increasingly intrusive, offering reorder "suggestions", new services, etc., principally to encourage transactions. That might benefit AMZN, but, IMO, it is getting potentially creepy, and at some point may blurt out something very embarrassing in company.
[Siri was never of much use to me as it had difficulty with my British accent. ]
Firms with their own LLM: Open AI, Google, Anthropic, Meta, XAI, Amazon, Bloomberg, DeepSeek, Baidu, Tencent, Huawei, Foxconn. Some are better than others, and some are for internal use only. The length of the list shows that LLM's aren't rocket science (we can't currently produce a Saturn V). Regardless, Apple is oddly missing. At the least, they should incrementally improve Siri.
Apple is the tech equivalent of Louis Vuitton or Chanel. Their brand makes them incredibly valuable. But Louis Vitton and Chanel don't have to worry about new technology.
I think the problem is that LLMs are less than meets the eye. Apple got punked. LLMs are very good at impressing the uninformed and that includes upper management. As Cory Doctorow noted, they don't have to be good enough to do your job, they just have to be good enough to convince your boss that they can do your job.
It's very easy to imagine a v1 that was tuned to work on one battery of tests to the point where the engineering management involved was convinced that just another six to twelve months of tuning would get something working more generally. Unfortunately, programming LLMs is not like writing code. There are lots of people with a vague sense of how far along software is and how close to meeting goals. They're usually wrong in detail, but often close enough to estimate ship dates. The joke is that the best way to improve such skill is to multiply your first estimate by your age.
LLMs are another matter. These systems are not transparent and not robust. Remember when the big thing was tricking computer vision systems by putting small stickers on stop signs to convince them that a firetruck was blocking the way. LLMs are all too similar in operation, so v1 was a lot farther from release than anyone at Apple thought. They found out the hard way as so many others will.
Unlike traditional software, where you can design, build, and test modules so that they work and can be integrated into the OS, LLM/LRMs are very difficult to control. Think of them as children that you are trying to train to do various tasks. They learn some, then forget while doing something else, and then different tasks that seem similar confuse task completion, etc. How do you ensure reliability on an AI that has to be packed on an iPhone? We see the so-so reliability of huge LLM/LRM models, so it is easy to understand why a Siri-controlled system might prove unreliable.
It may be that traditional software engineering methods and development don't work when shifting to an "educational/teaching" method. Do we need a pedagogy for AIs?
A day late but here are my thoughts about what Apple is undertaking in trying to get Siri up to snuff and why it’s a hard problem to solve:
1) Apple doesn’t just want a reliable LLM interface with their products, such as what Microsoft is doing with copilot. They also want a reliable voice interface and that’s a whole other set of challenges that no leading AI company has solved yet.
2) Apple wants their AI enhanced Siri to work *offline* and that’s a significant challenge. Perhaps even more difficult than they were expecting.
2a) Using any generative AI at full capability right now requires an entire farm of servers with thousands of nvidia chips networked in just the right way with full access to reams of training data. That’s a lot of overhead to make every little query and comment work.
2b) You can run simplified AI models locally on your PC but it requires you to have an expensive nvidia GPU, often a part that consumes more than 300wats under load and has a physical footprint measured in tens of inches/centimeters. On a single chip with a smaller set of data, capabilities are extremely… modest.
2b1) I mention nvidia chips because nobody has an alternative to their chips now or in the immediate future. Apple’s chip designs are amazing but they were not developed for the work of generative AI. Running a local LLM on your iPhone just isn’t possible with apple’s current chip architecture and they can’t simply copy nvidia because, yes, patents but also because nvidia is not efficient enough for mobile, and because Apple doesn’t want to be dependent on anyone else.
3) the networking required to synchronize an LLM’s operations across a server farm and a local device with different chips and different capabilities and different latencies and data lost to cellular or WiFi conditions is a nontrivial challenge but Apple’s vision for AI Siri requires solving that too.
4) we haven’t even gotten to the software running Siri and all the little agentive interfaces into every app and program on your iPhone or your Mac.
So, Apple basically needs to be the first company to solve several different but related hard problems in order for Siri to function in the way they envision. They need an in-house AI chipset that is efficient enough for mobile because Siri needs to be able to do some “thinking” on your local device, potentially offline. That chipset, when online, needs to function well with the large server farms, something nobody else is trying to do at the moment afaik. There’s a reason all of the compute is being done in server farms and not distributed across everyone’s individual devices! Oh, and they’ve got to get a class-leasing voice interface off the ground so Siri properly understands people of varying accents, languages, and blood alcohol content.
I have no idea why anyone at Apple thought they would solve all of this in a matter of 12-18 months! I suppose they underestimated the challenges involved.
This whole thread, starting with the main post, has been enlightening. I'm an old-school programmer, skilled enough in 20th-century technology, but definitely ignorant of AI, which I studied briefly when "neural networks" were a fad on its way out because they just didn't perform. Wonderful, what can be done when cost-effectiveness improves by 7+ orders of magnitude!
But modern AI is a mystery, and how to manage software development now is an unexplored mystery. I appreciate the expertise I've seen here, not least in DeLong's essay.
Apple's software is no longer as far ahead of the industry as it once was, and their quality control problems with both iOS and MacOS have been documented for at least five years, so perhaps their development group is just failing with "AI".
Apple's fundamental research is still very good however. Their CPU group continues to leapfrog every competitor, and they may have finally broken Broadcom's monopoly on cellular modems. All that hardware requires extremely good firmware to reach its potential, and Apple has clearly invested in that and research to support it.
So maybe - just maybe - Apple's research group dug deep into the LLMs currently being touted as this half generation's "Artificial Intelligence" - and found nothing there. That the whole thing is is a fraud built on an illusion built on stolen intellectual property of others. Marketing won't listen and keeps pushing "Apple AI" but there is no there to be had...
Sorry, I do not want my driver’s license floating around in computer space for anyone to access.
I don’t use Siri. Really turned me off that I am supposed to say “Hey Siri” to turn on that feature. If I would not say that to a person, I would not say it to a computer either.
I don’t think it is a poor product just because it can’t recall every minute detail about my personal life.
I experienced the restaurant feature of Siri, and it did not work very well.
Have experienced another restaurant seeking program and it did not work well either.
I thought it was expecting way too much. Way too much of the programmers if this detail they are supposed to furnish. Way too much of restaurant managers if they are supposed to update all this information every day.
I think it is a mistake to program features like this for certain people who need this in their lives.
Just upgraded to Alexa Plus, and I'll never use Siri again until Apple gets this most basic feature right. An integrated assistant, a la Apple 1986 demoware, is too advanced for Apple to achieve now. Just give us a good verbal search assistant for now.
I am not particularly happy with the privacy on Alex, and Alex+ seems to require less privacy.
If we can run an LLM on a phone, I see no reason why Alex/+ shouldn't run locally, and only have access to your AMZN account when required.
Alexa has become increasingly intrusive, offering reorder "suggestions", new services, etc., principally to encourage transactions. That might benefit AMZN, but, IMO, it is getting potentially creepy, and at some point may blurt out something very embarrassing in company.
[Siri was never of much use to me as it had difficulty with my British accent. ]
Gad, you'd expect that an LLM thingie would immediately recognize an accent and adapt to it!
(This started out to be sarcasm, but I suspect it's a valid criticism of the state of the whole technology.)
Firms with their own LLM: Open AI, Google, Anthropic, Meta, XAI, Amazon, Bloomberg, DeepSeek, Baidu, Tencent, Huawei, Foxconn. Some are better than others, and some are for internal use only. The length of the list shows that LLM's aren't rocket science (we can't currently produce a Saturn V). Regardless, Apple is oddly missing. At the least, they should incrementally improve Siri.
Apple is the tech equivalent of Louis Vuitton or Chanel. Their brand makes them incredibly valuable. But Louis Vitton and Chanel don't have to worry about new technology.