Evaluating Creative Artifacts with CUPA
The final chapter in the Creativity part of my new book, Discontinuities: Love, Art, Mind
In a previous article I expounded on the CUPA (caliber, utility, potency, appealingness) criteria for assessing assertive artifacts summarizing what I published in my two Cognitive Productivity books. In my previous article, I argued that creativity includes but requires more than Margaret A. Boden’s SIN criteria (surprise, importance, and novelty). In a new chapter ending the Creativity part of my book, Discontinuities: Love, Art, Mind, I combine and extend these two sets of ideas. I adapt the CUPA criteria to assess creative artifacts in general: conceptual and non-conceptual, assertive and non-assertive. The following is a draft of this new chapter. I hope to get feedback on the chapter so that I can improve it before the book is marked fully complete. Discontinuities is currently marked as 95% complete. The remain chapter is the eponymous and final chapter of the book, “Discontinuities”, which argues that the space of possible (artificial and natural) minds is discontinuous, echoing a theme of my recent articles on intelligence. I hope you will enjoy the following chapter, but let me know if you don’t 😉.
In a previous chapter, I argued that Margaret Boden’s SIN criteria—Surprise, Interest, and Novelty—and other theories of creativity in cognitive science I’ve encountered, while helpful, capture only part of what makes creative works memorable. They tell us something important about creative products themselves, but not enough about the relationship between those products and the minds that encounter them. The greatest works of art, literature, music, science, and even engineering do more than satisfy our curiosity or impress us with their originality. They seem crazy (bizarre and dubious), they may feel a bit funny, they not only capture our attention while we are experiencing them (as Brian Boyd aptly argued), they create a perturbance meaning they repeatedly come to mind. The greatest artifacts (assertive or artistic) reorganize our conceptual structures and reshape our motivation —i.e. require accommodation, — inspire new projects, and continue influencing our thinking years after our first encounter with them. Creativity, in other words, is not simply a property of artifacts. It is also a property of what those artifacts do to us: they have psychological effects on the audience.
That observation suggests a broader question. When we recommend a novel to a friend, praise a film, admire a painting, or become excited about a scientific theory we are not merely evaluating its creativity according to SIN criteria. We assess whether it is well crafted, whether it taught us something, whether it changed us, whether it helped us, whether we enjoyed it, and whether we think others ought to experience it.
Over the past twenty years, while working on Cognitive Productivity books and R&D (such as Hookmark), I found myself repeatedly evaluating books, scientific papers, software, lectures, and other knowledge resources using four broad questions. Although I originally developed the CUPA framework below to help knowledge workers make better decisions about what non-fiction to read, study and use, while working on this Discontinuities book I gradually realized that the framework applies just as naturally to works of art qua art. Indeed, one of the themes of this book has been that art itself can be a knowledge resource—not always propositional knowledge, to be sure — but often psychological, emotional, motivational, or existential knowledge. We therefore need a richer vocabulary for discussing its value; more precisely what I referred to as its helpfulness. The CUPA framework developed in Cognitive Productivity books involves four compound categories (not just dimensions) of assessment: Caliber, Utility, Potency, and Appeal. I recommend my Cognitive Productivity books for more details on the general framework. Here I expound its general applicability.
To be clear, each component of CUPA is compound: not merely a single dimension or set of dimension. Cognitive Productivity with macOS: 7 Principles for Getting Smarter with Knowledge makes that abundantly clear particularly for caliber which resists overly simplified ways of assessing scientific theories.
C: Caliber
The first set of criteria, caliber, concerns the quality of the artifact itself. These are the criteria with which traditional discussions of creativity are most concerned. (For assertive artificats, it subsumes but extends the CRAAP criteria often taught to undergraduates.) Is the work original? Is it insightful? Does it display technical mastery? Does it make an important contribution to its field? Does it exhibit elegance, subtlety, coherence, or depth?
Two of Boden’s SIN criteria, discussed above, fit quite naturally within the caliber category. Novelty is obviously an aspect of caliber and reflects whether the a work has achieved something genuinely original. Surprisingness is psychological and thus part of potency, discussed below. The importance of a scientific discovery or an artistic innovation likewise belongs here. In the opening chapter, however, I suggested that caliber extends beyond novelty. Some works become classics not because they were radically unprecedented, but because they achieve extraordinary levels of refinement, coherence and profundity. Shakespeare did not invent every dramatic device he employed, nor did Bach invent harmony. Greatness often consists not merely in novelty, but in realizing possibilities that others had only dimly perceived.
Caliber alone, however, cannot explain why certain works become personally transformative. Two books may be equally well written, yet one quietly returns to our thoughts for decades while the other fades from memory within a week. The difference, I suggest, often lies in what I call potency.
P: Potency
Potency concerns the psychological consequences of engaging with an artifact. It is the degree of Piagetian accommodation (not mere assimilation) required to grasp the significance of the work. A potent work changes the way we think. Sometimes it forces us to accommodate new concepts rather than merely assimilating new information into existing conceptual structures. Sometimes it leaves us with productive uncertainty rather than tidy conclusions. Sometimes it motivates us to begin an entirely new project or to reconsider long-held assumptions.
Real accommodation requires intellectual effort. It is not for the cognitively miserly as Keith Stanovich discussed in What intelligence tests miss: The psychology of rational thought. It requires changes not merely in what I call comprehension, which is short term familiarity. It can take months, sometimes years, to grasp the full implications of a new theory, novel, or film. It can take months, sometimes years, to appreciate the full implications of a new theory, novel, or film.Charles Darwin’s theory of evolution, Richard Dawkins’s The Selfish Gene, Turing’s last paper, on morphogenesis (accessibly explained by Philip Ball for the The Royal Society), and Aaron Sloman’s The Computer Revolution in Philosophy, are four of the best examples I know.
One cannot do the work required to accommodate to every piece of information we encounter. However, the difference between a fluid expert and a crystallized one is that the fluid expert spends a portion of his time daily, or at least weekly, updating their mental models with potent new information. This is beautifully explained in Surpassing ourselves: An inquiry into the nature and implications of expertise by Carl Bereiter and Marlene Scardamalia. Many people who were experts fossilize. They remain inert, refusing to engage deeply with information that requires effort to accommodate, though they may superficially skim it.
This type of effort typically must come from what psychologist Robert White referred to as effectance, the motivation to improve oneself, which in turn is normally a matter of architecture-based motivation. That is inherent propensity to respond to information by effortful accommodation. (This is discussed in my substack article on Improving Concepts of Epistemic Agency.)
Humor, strangeness, emotional power, ambiguity, and even bewilderment can all contribute to potency—not because they are objectively valuable, but because they increase the likelihood that the work will continue occupying our minds.
Potency does not merely entail dry, cognitive change. It also includes affective change: new motivators, new motive generators, new preferences and new moral values may emerge. War and Peace has turned many readers into pacifists.
Throughout this book (and my career) I have emphasized the importance of mental perturbance: the tendency of mental content to disrupt, maintain and otherwise influence ongoing cognition. As I noted in the opening chapter of this part of the book, great works, like great love affairs and grief, are often potent precisely because they induce such perturbance. They refuse to let us remain intellectually or emotionally unchanged. By recurring repeatedly in one’s minds, one can deeply reshape oneself.
This is why I suggested in the opening chapter of this part of the book that several criteria often associated with creativity are better understood as indicators of potency than of creativity itself. A work that leaves us uncertain for weeks is if not creative at least noteworthy. Nor is a bizarre work, to use Eric Schwitzgebel concept, necessarily a better work. Yet both uncertainty and bizarreness may greatly increase the extent to which the artifact continues interacting with our cognitive architecture. From a design-oriented perspective, this continuing interaction is part of what makes the work potentially helpful.
U: Utility
The third set of categories, utility, reminds us that readers, listeners, and viewers are not necessarily passive spectators. They are autonomous agents pursuing multiple goals, solving problems, forming relationships, grieving losses, raising children, designing software, conducting research, and attempting to live meaningful lives. A work may therefore be valuable because it helps someone pursue one or more of these projects. A novel about grief may help someone navigate bereavement. A philosophical essay may clarify a difficult decision. A scientific paper may provide the missing concept needed to solve a technical problem. A song may help regulate one’s emotional state during a difficult period. None of these forms of usefulness diminish the artistic value of the work. On the contrary, one of the recurring themes of Discontinuities has been that stories, music, and other forms of art can become tools for self-understanding and self-transformation when engaged with deliberately and thoughtfully.
Utility in my framework is not a sociological concept (though it can be in other frameworks). It is expected value relative to the individual. I mean personal utility. A treatise on quantum mechanics may possess extraordinary caliber while being almost useless to someone interested only in Renaissance painting. Likewise, a deeply personal memoir may profoundly help one reader while being irrelevant to another. Utility therefore asks not simply whether a work is useful, but useful for whom, and for what purpose.
A: Appeal (appealingness)
The final set of criteria concerns appealingness. Many academics are understandably suspicious of appeal because popularity is such an unreliable guide to quality. Yet appeal cannot simply be ignored. No artifact can influence anyone unless it first attracts attention. Attention, as we have seen repeatedly throughout this book, is the gateway to learning, emotional engagement, conceptual change, and self-transformation. An artifact may possess extraordinary caliber and enormous transformative potential, but if nobody is willing to engage with it, its influence will necessarily remain limited. Appeal concerns the capacity of a work to invite attention and sustain engagement long enough for its caliber, utility, and potency to exert their effects.
Seen in this light, Boden’s SIN framework finds a natural home within a broader evaluative framework. Novelty, surprise, and importance remain indispensable characteristics of many great creative works, but they largely describe one aspect of a work’s value—its caliber. Other characteristics discussed in the previous chapter, including conceptual accommodation, productive uncertainty, humor, emotional resonance, and persistent cognitive occupation, contribute more directly to potency. Utility reminds us that works interact with people pursuing real projects in the real world, while appeal recognizes the indispensable role of attention in every form of human learning and transformation.
The primacy of wholistic evaluation
These sets of criteria of overall helpfulness of an artifact must not be taken in isolation. They must be integrated with each other. Not every transformation (potency) is desirable. Reading Mein Kampf for instance turned many readers into lunatics. Moral standards, the first criterion, applies not only to the evaluation of the work, but the person that one would become in response to it. Some works must be considered as repugnant. They must trigger disgust. Some works are seductively appealing but counter-productive. Alternatively, the truth is not always beautiful. Darwin recognized that his theory while true entailed an ugliness in nature. The first two set of criteria — praiseworthiness and desirability — must normally dominate the second evaluation. One needs to adapt carefully (potency) to artifacts – not allowing the theory of evolution for instance to affect our assessment of the worth of other human beings.
The Cognitive Structure of “Emotions” Ortony, Clore & Collins (1988/2022)
Surprising as it may seem, the CUPA framework actually draws on a theory of emotions. I will take the shortcut of quoting from above:
According to Ortony, Clore and Collins, in their book, The Cognitive Structure of Emotions, there are three dispositional kinds of affect each yielding its respective type of emotional occurrence:
Standards or norms, which generate assessments of the praiseworthiness of agents’ actions, yielding attribution emotions.
Goals, which generate assessments of the desirability of events, yielding event-based emotions.
Tastes (also known as attitudes, and dispositional liking and disliking) that generate assessments of the appealingness of objects (episodic liking and disliking). These assessments are called attraction emotions. We can generalize these to include pain and pleasure mechanisms which generate, as you would expect, feelings of pain and pleasure.
You can thus see that the Caliber criteria of CUPA map onto Ortony’s praiseworthiness criterion. The “Utility” criteria map onto Ortony’s desirability criterion. The appealingness criteria are, well, Ortony’s appealingness criterion applied to responses to artifacts.
I acknowledge however, that the mapping between Ortony, Clore & Collins’s dimensions and the CUPA framework is not perfect. For example, praiseworthiness in their theory is usually applied to agents’ actions, whereas caliber in mine applies to artifacts. So the CUPA framework generalizes their framework.
Leveraging their framework is a fittingly strange loop (a recurring feature of this book), reflecting the caliber, utility, potency and appeal of The Cognitive Structure of Emotions theory, which is deservedly famous in affective sciences.
Conclusion
The CUPA criteria don’t replace the creativity criteria mentioned above. They include and extend them. Artists, scientists, engineers, teachers, and software designers all create things intended to interact with human minds. Creativity remains one hallmark of excellence, but it is not the final objective. The final objective is to create artifacts that matter: artifacts of high caliber, capable of helping people pursue their projects, potent enough to reshape their thinking, and appealing enough to earn their sustained attention.
Having said that, I expect some readers will be inclined to define creativity in terms of the CUPA criteria. A rose by any other name…