For these comparisons we shall discuss the acting prowess of the TOS and AOS triumvirates.
For TOS my ranking goes (top down) Spock, McCoy, Kirk.
First off, William Shatner as Captain Kirk. Now, I know Shatner takes a lot of flak for his overacting, and I don't want to just hate on him. Shatner has some really good aspects as Kirk - his command presence is consistently impressive, he's got great interactions with Spock, his acting in TWOK is brilliant (I mean that no in the warp core chamber? Too incredible?) - but the overacting is real.
To be fair to Shatner, he was acting in a time where overacting was more common. People didn't have the kind of access to high quality television back then that we do now. Many people only had small, black and white, grainy TVs, and since they were worried about radiation from those televisions, they often sat many feet away from those TVs. In those circumstances it was important to be more exaggerated, like in stage acting.
However.
Spock exists, and his brilliant subtlety is evidence enough that Shatner's delightful ham was not always necessary. It is, of course, a gift to mankind, and far be it from me to regret his delightful corniness, but, as they say, someone has to come in last.
Number two of TOS is DeForest Kelly as Dr McCoy, for the simple reason that the man is incapable of giving a bad performance. Taking into account the entire series and the movies De actually consistently has a better performance than anyone else in the show, including Spock. This is because Spock had a rocky start while they were figuring his character out, but De hit the ground running and he did not slow down.
I legitimately cannot think of a single bad performance from this man. I mean, watch that scene from Spock's Brain. "His brain… is gone!" De didn't have to go that hard with lines that ridiculous, but he did. For us. Star Trek V The Final Frontier? He kills it there too. He's got a great presence as a doctor, he's completely natural; he just kills it with an all-around solid performance.
Of course, the number one spot can only go to Leonard Nimoy as Mr Spock. Yes, he had a rough start. No one knew who Spock was supposed to be when the series was starting, and boy does it show in the earliest episodes, but he catches up very quickly.
There are so many stories about how Nimoy specifically contributed to the character of Spock: he invented the ta'al based off a Jewish blessing. When the script called for him to knock someone out with a haymaker he decided Spock would have a more elegant way of dealing with physical problems and invented the Vulcan neck pinch. In the Naked Time, Spock was originally supposed to break down because someone drew a mustache on his face, and Nimoy argued that Spock's character deserved more respect, leading to Spock's incredible breakdown speech "when I feel friendship for you I feel ashamed".
And let's be honest - Nimoy was out here killing it in every aspect of the acting department. Never has one man's eyebrow said so much. The amount of inflection he could manage by barely changing his tone is incredible. He would barely move his face, but he could out-emote Shatner. For example in Amok Time listen to Spock talk during his breakdown with Chaple and conversation with Kirk where he requests leave. You can feel the tension and emotion bursting at the seams, but even then he still has such tight control over himself that the tension is through the roof.
He's a great physical actor as well, especially when he's being intimidating. Again, in Amok Time when he's just standing menacingly as he loses control to Pon Farr… you will forget that this is a normal human man wearing pointy ears with funny eyebrows. He goes full predator, and it's especially good because he's still restraining himself. The ultimate expression of this is in Mirror Mirror, because MU Spock is always in predator mode. But that mind-meld with McCoy… acting. Perfection.
But he was also great when he was being friendly - when he could sass McCoy or banter with Jim. Even though he didn't emote openly you could always feel how much affection he had for Jim, or how much sass he could work into his perfectly logical statements.
Why am I still talking? It's Leonard Nimoy. You know how great he is. Time for AOS!
My AOS ranking (top down) is a little different, and it goes McCoy, Jim, Spock.
No hate to Zachery Quinto, but we all knew Leonard Nimoy was lightning in a bottle, and no one will ever be that good as Spock again. There are a few things Quinto does that I like, of course. When he and Jim are being debriefed by Pike after their disaster of a mission when Spock ratted Jim out for breaking the prime directive, Spock has a great interaction with Pike.
It's not so much what he says as it is… well, how he says it. He's too fast. As soon as Pike stops talking, Spock starts, just a beat before is socially acceptable. There's no time to think in there. The only explanation is that Spock has extrapolated what Pike is going to say and found his response to it before Pike even finishes his sentence, and Spock is just waiting to get to use his snappy comeback. I like that.
But overall… he wavers from either too emotional to too shut down. Part of this is the script - JJ Abrams was interested in angsty violent protagonists who did cool things like 'be angry' and 'beat people up' and we got some serious character assassination thanks to that. But part of this is just the acting. He never feels like the old Spock, but you can only be so mad at an actor for not living up to near-perfection.
Second place is Chris Pine as Jim Kirk. Note that I am specifically talking about the acting, not the writing. This guy had a hard job. The writers needed to bridge the pop culture perception of Kirk, the womanizing hotshot, with the cannon Kirk, a responsible but unpredictable captain.
Now, this iteration doesn't really feel like the old Kirk until Beyond, but his backstory is so different that this is alright. He does an admirable job acting out the transition required for his character, and by the time we hit Beyond, he's honestly really good.
In a vacuum, Pine is good in this role, and compared to the original series… he gets there eventually, and I believe the character he is portrayed as leading up to the final movie.
Finally, last but not least, in fact the greatest of them all, is Karl Urban as Dr McCoy. Urban is a revelation as McCoy, and it is as though DeForest Kelly returned from the grave to grace us with this breathtaking performance.
I actually consider everything about Urban's McCoy borderline cannon for TOS McCoy unless it is directly contradictory to TOS cannon. This is solely because of how incredible Urban is in this role. When Nimoy saw Urban's performance he cried because it reminded him so much of his late friend the late De Kelly.
The lack of screentime Urban gets in these films is criminal. He is by far the best character in all three movies, he's extremely likable, he feels natural as all get-out, his banter and griping are on point… any critiques I have with his character are really relegated to the writing department, not his acting. Make Star Trek VI and let McCoy be a proper main character this time, you cowards!