Measuring urinary symptoms is harder than it looks

The standard questionnaire

The International Prostate Symptom Score (IPSS) is the questionnaire most urologists use. It began as the American Urological Association’s symptom index in 1992.² It has seven questions, on how often you need to go, getting up at night, a weak stream, difficulty starting, a stream that stops and starts, feeling you haven’t emptied your bladder, and urgency. Each asks about the past month and is scored from 0 to 5 (so the total runs from 0 to 35). A separate eighth question asks how you would feel if your symptoms stayed as they are.³ Totals are usually grouped as mild (0 to 7), moderate (8 to 19) or severe (20 to 35).⁴

When researching or comparing groups of patients the IPSS performs well. In the original study its questions hung together closely, and men who filled it in twice gave very similar totals (a correlation of 0.92).² That tells you the questionnaire can rank a group of men by the severity of their symptoms. But it says less about whether changes in one man’s score over time are a reliable guide to how he is doing from one appointment to the next.

Answering consistently

At a US radiotherapy clinic, 144 men with prostate cancer filled in the IPSS online, usually in the week before their appointment. At the appointment a nurse went through each answer with them.⁴ Half of the men changed at least one answer. For 10% of the men, the corrected score would have changed whether they were eligible for treatment (although a week had passed, so some men’s symptoms may really have changed). Although these were men having radiotherapy for prostate cancer, rather than managing urinary symptoms, the survey questions are the same.

Remembering symptoms

The nurse’s check catches one source of error: men misreading the questions. But even a man who understands every question has to remember a month’s worth of symptoms in one sitting. Research on memory shows how hard that is.

Much of what’s known about recall of symptoms comes from studies on pain and fatigue. For example, Arthur Stone, Joan Broderick and colleagues compared what people later remembered with ratings they gave at the time, on an electronic diary several times a day. In 83 patients with rheumatic conditions, people consistently remembered their symptoms as worse than they had rated them at the time.⁵ Recall of the past day matched well, but patients found it increasingly hard to remember symptom levels beyond the past several days. A later study found end-of-day ratings closely matched the day’s in-the-moment ratings.⁶

Urinary symptoms have also been tested directly. In the LURN Recall Study, 254 men and 261 women at six US specialist clinics answered questions about their symptoms at the end of each day. Some did this for a whole month, then answered the same questions about the past month.⁷ Across the group, the month they remembered matched the average of their daily answers well. It’s possible that the practice of thinking about symptoms every day to answer questions means that there’s a more consistent answer at the end of a month. The study didn’t measure recall of a month without the daily check-ins, and it had a well educated and steady group, as anyone planning a change of treatment was excluded.

Even so, at least a quarter of people over- or under-reported on some questions, including some similar to those in the IPSS on getting up at night, urgency, weak streams, not emptying fully and dribbling.⁸

Leaving out treatment changes matters. Symptoms shift when a treatment starts or stops, and memory is least reliable when symptoms are shifting. In a study of 95 women recording premenstrual symptoms every day, recall over 7 days was further from the daily record when symptoms were rising or falling than when they were steady.⁹ As far as I can find, nobody has tested this for urinary symptoms.

Noticeable changes

An error in someone reporting symptoms matters because the error can be as big as a genuine change in symptoms that a patient would notice. The IPSS scores go from 0 to 35. Estimates of the smallest change a patient notices range from about 3 to about 5 points, and depend on how high the score was to begin with. In a trial of 1,218 men with an enlarged prostate, those who said they felt slightly better after 13 weeks had improved by 3.1 points on average.¹⁰ A later study, of men starting a new treatment, put it at about 5 points: about 3 for men whose symptoms were moderate, and about 7 for men whose symptoms were severe.¹¹ Doctors use the lower figure as a rule of thumb. In the US, one of Medicare’s measures of the quality of urology care counts how many newly diagnosed men improve by at least 3 points within a year.¹²

The score can also move when nothing has changed. The same study estimated this measurement noise as well: any single score could easily be 3 or 4 points above or below where the man’s symptoms really were.¹¹ In the US radiotherapy study above, the scores of one in five men moved by 3 points or more between answering on their own and going through the questions with a nurse, up to a week later.⁴ What this means is that a 3 to 5 point swing in the score could either be a genuine change or a result of people struggling to remember symptoms or being inconsistent in their answers.

Bladder diaries

The usual answer to the memory problem is to ask people to keep a detailed diary. A bladder diary records each trip to the toilet as it happens, usually for three days, often with the amount passed. It is a lot to ask. In another study by the LURN research network, 40% of the 290 people enrolled had no usable diary: half returned none, and the other half kept it on days that didn’t match the study’s other questionnaires.¹³ In a larger LURN study, 85% returned a diary but only half of the diaries were complete (although 88% had usable content).¹⁴ At urogynaecology clinics, 83% of women completed a one-day diary, and of those only 54% wrote each visit down at the time.¹⁵

Shorter daily questions do better. In the LURN Recall Study described above, which asked a few questions at the end of each day, 90% of the 569 people who signed up completed the study.⁷ Asking less, and more simply, seems to help people keep going.

The questionnaire in the NHS

In England, NICE’s guidance starts with a diary, not the questionnaire. It asks GPs to have men with bothersome symptoms keep track of drinks and toilet trips for a few days.¹

NICE suggests the IPSS only once a man is thinking about treatment, so there is a starting score to compare against later. The committee that wrote the guidance felt that asking every man to fill it in “was time consuming and did not add much” to talking with him. They saw its value as providing “a baseline score to monitor their response to treatment”.¹⁶

Even then, it isn’t always used. When the Royal College of Physicians audited NHS records in 2010, the year the guidance came out, a symptom score had been written down for about a quarter of men seen by their GP, and fewer in hospital.¹⁷ My own experience has been the opposite: GPs and urologists have asked me to fill in the IPSS many times, and a diary only once (and then they forgot to send the instructions).

Despite the score’s bands (mild, moderate and severe) providing a vocabulary for doctors and patients, in NICE’s guidance decisions about treatment mostly turn on whether the symptoms bother the man and whether a medicine has helped.¹

Guidelines elsewhere use the questionnaire more. In Canada, urologists are advised to use it with every man at his first appointment, and again to judge whether a treatment is working.¹⁸ In the US, there is the Medicare quality measure described above, which counts how many men improve by at least 3 points.

So in the NHS and abroad, the questionnaire’s main job is to track how one man’s symptoms change once he starts a treatment. That is the use the earlier sections show to be least reliable.

Asking at random moments

Psychology has a long-established method for measuring how people feel without needing people to have great memories or be diligent diary-keepers. It’s called ecological momentary assessment.

In the 1970s the psychologist Mihaly Csikszentmihalyi, later known for his work on “flow”, gave 25 teenagers pagers that bleeped at random times during a normal week. At each bleep they noted what they were doing and how they felt.¹⁹ Arthur Stone and Saul Shiffman brought the approach into health research in the 1990s and named it ecological momentary assessment.²⁰ Shiffman, Stone and Michael Hufford describe it as “repeated sampling of subjects’ current behaviors and experiences in real time, in subjects’ natural environments”, designed “to minimize recall bias”, often by random time sampling.²¹

Once we all had phones, these studies became much easier. The Harvard psychologists Matthew Killingsworth and Daniel Gilbert built an iPhone app that asked people at random moments what they were doing and how happy they were, and collected nearly a quarter of a million samples from about 5,000 people.²² An example of what they found, from 2,250 of those people: people’s minds wandered almost half the time, and that’s when they felt less happy. It’s not the kind of thing you’d have seen from a diary or a questionnaire looking back over a month.

Cornelia Wrzus and Andreas Neubauer reviewed 477 studies using the method, covering more than 677,000 participants.²³ The typical study prompted people six times a day for a week, and people answered 79% of prompts on average. A common worry about the method is that being asked too often wears people down, and they stop answering. Across these studies that did not happen: the number of prompts made no difference to how many were answered, or to how many people dropped out. Those studies were more often with younger people (average age was 31) so it may not work as well for men in their 50s and 60s. Response rates also drop when a study stretches over many months.²⁴

The method has started to reach urinary symptoms. In two studies, people with pelvic pain or an overactive bladder had a phone app ask them four times a day how strong their urge to urinate was, at random or set times of day.²⁵²⁶ The typical person replied to nearly two-thirds of the prompts. Their answers showed how much the urge comes and goes. For the same person, it changed more over the course of a day than from one day to the next. A question about the past month asks for one answer to cover all of that variation.

Neither study was of men with an enlarged prostate. The closest is a trial that Scott Bauer, a doctor who researches urinary problems in older men at the University of California, San Francisco, published with colleagues in 2026.²⁷ The 31 men, who had been taking tamsulosin, a common medicine for an enlarged prostate, switched every couple of weeks between their usual tablets and dummy ones, without knowing which they were taking. Every day for about 11 weeks, they answered the seven IPSS questions about the past 24 hours instead of the past month. On average the medicine made a difference of about 3 points, but that hid a wide range: from an improvement of about 11 points for one man to a slight worsening for another. The trial didn’t ask at random moments, but it did ask about a short, recent stretch of time, again and again, and that was enough to see how each man responded.

Putting it together

Each way of measuring has a weak point. The monthly questionnaire is easy to administer, but relies on memory, which lets some men down and has not been tested while a treatment is changing. A diary avoids the memory problem, but is a lot to ask. Asking a few questions at random moments, about a short and recent stretch of time, is well established in other areas of health. The studies of urinary symptoms so far suggest it could matter here too. Rillio is about testing that out.