EMBench.com provides a comprehensive destination to understand and discuss the manifestation of "empathy" in generative AIs along with associated benchmarks.
The word "empathy" is quoted because I am not making a claim the generative AI's actually have emotions and are empathetic. The focus should be on the word manifestation. Generative AIs have definitively shown they are capable of manifesting the ability to:
identify emotions in content,
pass tests that require a nuanced "understanding" of emotional needs,
generate content that appears empathetic in nature.
As with the words "empathy" and "understanding" above, I will attempt to remember to quote words in all my posts where is seems like the generative AI does something but I am trying to avoid a debate about the word.
A number of academic papers have been published and formal frameworks proposed for assessing the EQ (emotional/empathy quotient) and in dialog empathetic capability of generative AI. Some of these are easy to find and others are not. Most provide useful insights. However, many are somewhat unidimensional and others use tests designed in such a way that the generative AIs actually get a leg up.
Each week it is my intent to provide a blog post that does one of:
pointing to a new development or direction in "generative empathy"
reviews a paper, test, testing framework, or benchmark.
addresses a topic relevant to "generative empathy"
I am sure these articles will bring up ethical, cultural, and technical issues. It should be interesting!
Over time, I hope to provide a comprehensive table of benchmarks for open source and proprietary but publicly available models as well as models embedded in purpose built chat bots and robots. I have started with a short table that uses industry standard multiple-choice psychology tests. There are shortcomings in this approach I will discuss in my next post.
I am also be providing links to relevant resources. Despite the power of search and generative AI, I have found that consolidated lists still help. They may be incomplete, but they can certainly help improve the signal to noise ratio.
And finally, what I am really trying to support is the development of AI that does not behave like the man in this video, It's Not About The Nail.
So, sign-up for email delivery or visit regularly.