The phrase “How does it know that?” is flattering, that is until it is not. There comes a point when the user transitions from feeling amazed to feeling disturbed at some point during the use of the technology, more often than not during the third or fourth instance of some relevant suggestion. However, most people on the team cannot agree on where the exact moment happens. This is the issue.

The problem arises from the misconception that the line depends on the amount of data collected. A scholar in ethics Helen Nissenbaum has introduced the term “contextual integrity”, according to which it does not matter how much data the companies collect; what matters is the proper use of this data, more specifically using it in proper context. Your fitness app knowing your step count is fine. Your banking app knowing your step count would be weird, even though it’s the exact same data point.

What causes the creepy reaction is the fact of not having a clue as to the reason behind it. A recommendation that comes with no context can appear to be intrusive, despite the fact that it has an innocent purpose. If we have the same recommendation, but with just one sentence explaining it (something like: “you liked X), it seems to be useful, even when no changes are made to the algorithm used. Just the user level of understanding is what changed. That’s the idea: it is the inability to see that makes people feel uncomfortable.

Which flips explainability from an ethics checkbox into an actual UX feature, one worth spending design time on the same way you’d spend it on onboarding or navigation. Not a settings-page disclosure nobody reads. Something closer to a small, visible line near the result itself: this is why you’re seeing this. It costs almost nothing to build and it’s usually what separates “the app just gets me” from “wait, how does it know that.”

Signaling “we understand you” without signaling “we’re watching you” comes down to showing your work at the exact moment the personalization shows up, not three menus deep in a privacy policy nobody opens. If a user has to go dig to find out why something happened, the digging itself is the tell that something’s being hidden, even when nothing actually is.

Somewhere in the middle of this is Numira and YourStyle because both numerology and style recommendations only succeed when they are made to feel very personal to you. The design choice that ends up being more important than people may expect is linking every personalized output directly to the input that created it alongside the output. A numerology reading that refers to the actual birth date seems justified, while a reading that seems applicable to any person looks like either a generalized reading or an unsettling experience in which the app matches you with something invisible to you.

This is not about being cautious, since dialing down personalization altogether for the sake of avoiding the creepy experience ultimately leads to a worse, blander product that’s still opaque about the little it does use. Those applications that understand this phenomenon right are not those that are being cautious, but those that are ready to be specific and show their actual work at the same time.