What psychology has to do with the software I build

I am studying psychology and data science, and people assume the software company is the day job and the degree is the plan for later. The honest answer is that one week of a methods course has already changed something I had shipped.

Within-subject design

The rule, as the APA defines it, is that you measure a person against their own baseline rather than against everybody else's. It sounds obvious written down. It is not what I built.

My instinct on a personal AI product was to cluster users, find the pattern most of them shared, and build toward that pattern. That felt like product sense and I would have defended it if anyone had asked.

What I had actually done was dress up between-subject thinking in better clothes, and in doing it I flattened the one signal I care about most: the change in a single person over time.

A population average tells you what a typical user tends to do. It cannot tell you whether the particular person in front of you right now is drifting away from wherever they usually sit. A flat day on its own means nothing. Held against that same user's last three weeks it means quite a lot, and the difference between those two situations is the entire product.

The question I keep asking now is whether I am storing a profile or storing a trajectory. Most of what I had written stores profiles. Clinicians worked this out decades before I arrived, and it is sitting in my textbook next to a chart of one subject plotted against nobody but himself.

Base rates

The second thing that transfers immediately. A test that is ninety-nine per cent accurate for a condition affecting one person in ten thousand produces far more false positives than true ones, and the intuition that resists this is very strong.

Every alerting system I have built has this problem. An anomaly detector that is right ninety-five per cent of the time, run against a thousand events a day where three are real, produces about fifty false alarms for three true ones. People mute it in a week, and a muted monitor is worse than no monitor, because you believe you have one.

I hit exactly this with a test-result parser that misread a summary and reported failures that were not there. The lesson was not to improve the parser. It was that a monitor which cries wolf gets muted, so the cost of a false positive is not one wasted minute, it is the entire monitor.

Construct validity, which is the one engineers should steal

Psychology spends enormous effort on whether the thing you measured is the thing you meant. Not whether the measurement is precise. Whether it is of the right thing.

Software is full of measurements that are precise and about the wrong thing. Engagement time as a proxy for value. Test count as a proxy for coverage. Deploy frequency as a proxy for velocity. Every one is measurable to three decimal places and every one can move in the wrong direction while the thing it stands in for gets worse.

The habit worth stealing is the question asked before the measurement rather than after: if this number went up and the thing I actually care about got worse, would I be able to tell?

Why the combination, honestly

The plan is psychiatry, and the specific thing I want to work on is how much administrative load a clinician carries that could be carried by something else. Notes, documentation, the paperwork between a person and their patient.

That problem needs both halves. You cannot design it from the software side without knowing what the clinical work actually is, and you cannot design it from the clinical side without knowing what these systems reliably do and where they fail. Most attempts have one half.

I am one semester in and this is a claim about a plan rather than a result. But the transfer has already gone in the direction I did not expect: not the software improving the psychology, the psychology finding a real defect in software I had already shipped.

More on the degree structure that allows this combination: New College at Alabama and studying AI here.