נקמה

Just to wrap this up, the Sefer HaChinuch says something very interesting about the prohibition of taking revenge. If asked, I think most of us would categorize this as a מצות בין אדם לחבירו. Yet, he explains that the reason for the mitzvah is because taking revenge reflects a lack of אמונה. If I truly believed, I would realize that anything this person did to me, I deserved - because Hashem made it happen and there is no reason to be angry at the other person. (The implications of this approach on the question of how free will coexists with divine providence is beyond the scope of this post).

Therefore, vengeance itself is not inherently a bad midda. By strict justice, or on a בין אדם לחבירו level, there is no reason not to respond in kind to someone the way they treated you. It is only the issue of אמונה that tells us not to take נקמה. When that is out of the picture - either because God is the actor, or we are taking the revenge on His command - vengeance is appropriate.

More on נקמה and the כהן גדול

Some answers, from Ma'ayanot Alum (and Why-a'anot regular) Aviva Novick:

I remember learning in 9th grade halacha (yay Rabbi Besser's class!) that a talmid chacham is allowed to take nekama because he is not acting on his own behalf - he is taking revenge on behalf of Hashem and Torah. So based on the fact that Hashem does seem somewhat ok with nekama, I think at it's core it is like all middot: it's good but only in moderation and when it's used appropriately. The Rambam says that when someone is too extreme in a certain middah, they should aim to go to the other extreme so that they will end up in the middle. Humans by nature are have a desire to take revenge - they are on the "too much nekama" extreme. Perhapes we only have the mitzvah of "lo tikom" because we need to go to the other extreme. Hashem on the other hand knows when it's good to use nekama and when it's not, and by Him commanding Bnei Yisrael to take revenge, He's showing them that this is the correct time to use nekama.

I think there's a lot of truth in this. I will B"N come back to this soon, ,but interested parties should see the Sefer Hachinuch on this mitzvah.

I heard a nice answer about the midrash with the kohen gadol's mom's cookies from a scholar in residence at shomrei torah in Fair Lawn. A few snacks are not enough for a rotzeach b'shogeg to stop davening for the death of the kohen gadol. but in order for tefila to "work," it has to be completely sincere. the mother of the kohen gadol is hoping that by doing something nice for the rotzeach, he will not be able to daven for the death of her son whole-heartedly - he will still want her son to die, just not as strongly.
His other answer which is not as "nice" was derived from a close reading of the midrash. the midrash does not say the mother of the "kohen gadol," it says the "mothers of the kohanim." the p'shat way of interpreting this is that the various mothers of the kohanim gidolim over time would have this custom, and it's in plural because over time there have been many kohanim gidolim. but sometimes there were multiple kohaim gidolim at a time - like if the kohen gadol became tamey and they needed to appoint a replacement, when the original kohen gadol was tahor again, both kohanim were kohanim gidolim in a way. a rotzeach b'shogeg would be free upon the death of either kohan gadol. So each mother made cookies for the killers in the arei miklat to try to get them to daven for the death of the other kohan gadol and not her son.

Interesting, I also like the first one better, (though I like mine best). אילו ואילו (ואילו?) דברי אלוקים חיים.