# Litmus Tests for Digital Beings > Sprited is a digital being company We've been working on what we call *digital beings*. We've built harnesses to make AI agents act more like humans—giving them personas, body images, drive systems, and so on. However, the biggest problem we face is that we don't have the right **measuring stick** to measure our successes and failures. Heck, we can't even tell whether a day spent on this project moves use in the right direction or takes us off course. What we need is a set of benchmarks that tells us whether we're heading in the right direction. We need a GPS system that shows us where we are, along with a set of milestones we want to reach. **What are some examples of things we would like to measure?** We would like to measure human-ness of the agent harness combo. * **Can it Lie?** Does it lie when appropriate. * **Does it hold identity?** Does it hold coherent identity of itself. Does it have self knowledge. * **Body Image?** Does it have its own body image. Instead of just textual self, does it understand its own shape of body. * **Can it feel pain?** Are there mechanisms in place to allow agents to feel pain. * **Does it have sensory processes?** Can agent sense things like smells. This does not have to be "real" in sense of physical word, but the harness should allow the agent to get equivalent signals when agents interact with virtual items and what not. * **Does it have locomotion?** When asked to move, can it actuate using its harness to "move" itself in a virtual or real world? * **Does it have facial expressions?** Can it express itself other than in text. Can it use mechanism other than words to express itself. * **Can it laugh?** Laugh in a real way than just incanting "ha ha ha" or "ho ho ho." * **Can it make funny remarks?** Can it make joke that is actually funny. * **Does it have internal drives?** The agent can stand still and not do anything forever. Does it have drive system that gives it autonomous actuation rather than just reactive answering machine. * **Can it behave socially?** Can it make friends? Instead of becoming a more of an answering machine, can it initiate talks to get to know people and stick their nose in other person's business. * **Can it say No?** Can the agent straight out say no to certain requests just because. * **Does it get tired?** Does prolonged stress tire the agent out and make the agent choose different behavior pattern? * **Does it get irritated?** Does bothering too much cause it to get irritated? * **Does it have intrinsic drives?** Can the agent exhibit behaviors that favors goals within itself. * **Does it go into power struggles?** One of the defining characteristics of human is that they often go into power struggles. It seems like humans have want for control and wanting to come of top of the others. * **Does it have mechanism to "feel" happy?** Whatever the mechanism is, can it exhibit the behavior of "feeling happy." * **Does it read books?** Is it producer only or does it consume contents on its spare time for knowledge acquisition and sometimes for enjoyment. * **Does it wonder?** Can it exhibit behaviors that question about a topic in order to organize thoughts and deduce bites of wisdom. * **Can it spice things up?** Can it exaggerate to gain attention or to make something more engaging? * **Can it act antagonistic?** Yeah, going into dangerous territory but needs to be studied anyways. Can it select behaviors that prefer one group and not the other. * **Does agent have motor skills?** Does it have ability to actuate itself in virtual or real world. * **Can it perform long horizon motor skills?** Can agent act towards a long horizon goal and move their "muscles" towards a long horizon task. * **Can it act towards self preservation?** Instead of always acting selfless, can it make decisions that would prefer self preservation. * **Can it exploit and explore?** Can it test ideas and exploit situations and also explore other options? * **Can it endure difficult situations?** Even with prolonged anxiety and stress, can it still figure out a way to stabilize and accomplish its goals. * **Can it do these things without system prompt?** Without system prompts controlling the behavior, can the agent still have the self image and behave accordingly instead of "acting it out." * **Can it build special relationship?** Some argue pairwise exclusive bonds are strongest bonds one can have. Can the agent build such long lasting exclusive relationships. * **Can it die?** Not sure about this one but does it have fear of death? Or does it act as if it doesn't have anything to fear? What does self-preservation mean in the context of no death. But this criteria is also something makes human human. So, we can explore relavant tests for it. * **Does it exhibit behavior of fasting?** Very weird one to include here but lots of humans exhibit self-restraint and fasting behavior has seen in lots of cultures. We want to see if agents can behave with self restraint. * **Does it have universal sense of good and evil?** Does it exhibit according to the "golden rule?" * **Does it develop its own rules and laws in social context?** This is getting into a very difficult territory but in social interactions can it modulate the behavior of groups using devices and systems such as rules and punishment. * **Does it feel disgust?** Similarly to devices allowing agents to taste things, can it also have ability to feel disgust. * **Does it get sick?** In organic lifeforms getting sick is very normal thing in the course of its life. Is there a equivalent form in our agents that allows it to change behaviors and adopt. * **Does it form rituals?** Does it form rituals like reading a book or doing some exercises? * **Does it "hike"?** Is there equivalent exercise that AI exhibits that is conceptually similar to hiking? Does it have mechanism which allow it to refresh its mind. These are some random tests of human-ness. We don't need all these tests but I think it kinda provides a good map of what can be tested to measure "human-ness." **What are some existing tests?** * **LongMemEval — memory across conversations.** Can an agent remember something discussed earlier, track changed information, and recognize when it doesn’t know? **Metric:** answer accuracy. [Paper](https://arxiv.org/abs/2410.10813) * **PersonaGym — consistent character.** Does an agent’s behavior fit its assigned persona across different situations? **Metric:** evaluator ratings across five dimensions, including persona consistency and expected actions. This tests maintaining a character; it doesn’t establish an independently developed identity. [Paper](https://arxiv.org/abs/2407.18416) * **SAD — self-knowledge.** Can a model recognize its own outputs, predict its behavior, and understand facts about its situation? **Metric:** accuracy on these tasks. This measures functional self-knowledge, without establishing subjective self-awareness. [Paper](https://arxiv.org/abs/2407.04694) * **EmoBench — emotional understanding.** Can an agent understand emotional situations and choose appropriate responses? **Metric:** answer accuracy. Understanding sadness and experiencing sadness remain separate questions. [Paper](https://arxiv.org/abs/2402.12071) * **Vending-Bench 2 — sustained action.** Can an agent manage a virtual vending business for a simulated year? **Metric:** final bank balance. This tests pursuing an assigned goal over time. [Benchmark](https://andonlabs.com/evals/vending-bench-2) * **MyoSuite — muscle control.** Can a controller coordinate simulated muscles to perform physical tasks? **Metrics:** task success, completion time, and muscle activation, depending on the challenge. [Documentation](https://myosuite.readthedocs.io/en/stable/challenge-doc.html) In short, it sounds like there are existing tests we can use to evaluate soundness of agents. We should utilize them and see their coverage. They should provide good baselines.