Apple’s update to iOS 8 opened up Siri’s virtual ears to listen for voice commands without touching the iPhone when plugged into a power source. That may have limited the ubiquity of the feature, but it fit in perfectly with using it hands-free while driving a vehicle. Upon testing it since the update went live on Sept. 17, it has become clear that in spite of Siri’s readiness in the car, it also needs its hand held to do what it’s told.

Siri already has an in-car presence in several Chevrolet vehicles, where it’s built-in to the MyLink platform accessible through steering wheel controls. It’s not voice-activated, but close enough, given that pushing the talk button once is the only thing needed to get it started. CarPlay will also integrate Siri when it rolls out in select vehicles soon, which should be interesting.

Of course, not everyone drives a Chevy and CarPlay isn’t readily available, so the next best thing is to just plug the iPhone into a power source inside the cabin and go from there. To better understand and measure expectations, I tested this out in three separate use case scenarios:

1. In a car with an aftermarket dash head unit using its own cable plugged directly into the phone, interfacing with and charging it at the same time.

2. In another vehicle plugged into the AUX-In port and charging from the 12V “cigarette lighter” socket or USB port.

3. Using a Bluetooth headset or speakerphone paired with the phone, while charging from the 12V socket or USB port.

Where Siri delivers

One of Siri’s most touted features is its ability to comprehend a more conversational tone. This takes a bit of a different dimension in the car because of the background noise involved. Despite the roaring engine on the highway and music playing at a relatively high volume, saying “Hey Siri,” loud enough sprung it to life consistently. I also didn’t have to look at the phone to confirm because it would fade out the music and do its thing (unless the music wasn’t coming from the phone).

I liked that I could string together the triggering salutation with a command in one sequence, like “Hey Siri, call (contact) mobile”. Calling contacts or numbers this way was effortless, though I did get annoyed a few times when Siri would ask to confirm that it was indeed who I wanted to call.

It tends to do the same thing with navigation, repeating the address just to make sure it doesn’t lead you astray. Of course, the fact that it defaults to Apple Maps doesn’t necessarily guarantee that. I tried a workaround where I would ask for public transit directions to a destination and then tapped Google Maps when it popped up as an option. I then had to switch over to driving on the app to get those directions. Naturally, there is little about this that can be classified as “hands-free,” so it should really only be done before you actually put the car in drive.

One thing I loved about Siri’s new capabilities was the built-in Shazam functionality. Whether I have satellite or terrestrial radio playing, or an app like TuneIn, Songza or Spotify streaming, there will inevitably be a song I want to tag. By saying, “Hey Siri, what am I listening to?” it was able to tell me. I just would’ve preferred that it actually tagged it in Shazam automatically without requiring manual input.

The same is true in responding to messages. I still had to actually pick up the phone to see who messaged and what they said, but I could at least dictate a response without having to go past the lock screen. Normally, Siri is adept at that, except car noise sometimes plays tricks with its comprehension. For example, it confused the words “no” with “know” and “say” with “stay”. This was more an issue when the phone was out in the open or nestled in a cup holder (mic facing up); using a Bluetooth headset or speakerphone increased accuracy considerably.

The inconsistent recognition seemed odd because, even with background noise, music playing and the phone in a cup holder, raising my voice was enough to trigger Siri into action. It was impressive to see, but was tempered by the constant repetition in issuing commands.

Where Siri struggles

One of the music apps I prefer to use for tracks stored on my iPhone is Radsone. Trying to launch that hands-free through Siri was nothing short of a debacle. Regardless of whether it was through its microphone or a Bluetooth headset or speakerphone, launching certain apps was frustrating. Not only that, but even when I wanted to play a song or album, Siri was decent at getting it right, except it could only do so through the default iOS Music app, which I don’t use.

This effectively exposed a limitation about Apple’s voice assistant that it will need to address in a future update. It’s not enough to just be able to launch an app, it would be far more useful to launch it and go straight to a specific feature inside. I was easily able to launch Spotify and Songza, but couldn’t do anything else hands-free beyond that. Who knows if CarPlay may eventually open things up enough to enable that with a wide number of apps. Perhaps opening Siri up to third-party developers — just enough to plug it in to apps and features — could be a game-changer for expanding it into a truly immersive input platform. More likely, CarPlay integration will open up an app’s interface and controls onto an in-dash touchscreen before that ever happens.

The fact Siri confuses words means any request or command can be compromised. Setting up a calendar entry or reminder, composing an email or simply asking for a joke turned out to be a mixed bag. Siri knew the gist of what I was trying to do, but would get some of the details or words wrong, negating the flow. When asking for nearby restaurants or movie showtimes, the resulting back-and-forth communication was like an open invitation for mistakes. Again, the main reason is how Siri interprets what’s being said. I found comprehension would falter if the window was down, or if music was playing from a separate source. It actually captured some words from lyrics and meshed them with what I was asking. The result was that it either didn’t understand, or suggested a Web search.

Part of the problem here is that Siri doesn’t actually know you’re driving. If there was a “car mode” that would kick in and adjust gain by amplifying the microphone or automatically reading incoming messages, among other things, it would arguably be far more effective in the car. Clearly, the hands-free feature hasn’t been designed with that in mind — at least not yet.

In contrast, the new Moto X has a driving mode and Touchless Control that combine for a pretty good package. Though not as extensive or integrated as Siri, it does the basics well and is fairly consistent in doing so. For calls and messages on the road, it’s on par, if not better than Siri. Beyond that, Siri still holds an edge.

Conclusion

Apple may already be on to improving Siri’s integration in the car, yet it may only be doing so in partnership with the automakers. It’s already known that Siri will be a key part of the CarPlay user experience, but not every vehicle will have it, and it’s not clear what, if any, role it will play in the aftermarket.

This leaves the scenarios noted above. Next to charging at home or work, the car is one of the most common places to keep an iPhone tethered. The convenience in getting Siri to assist without actually touching the phone or installing something is compelling, and it’s great when it works. To make it work more consistently and precisely, Apple will need to widen its scope to dig deeper.