Gürkan Özkan
Domain

Analytics

Thinking with data: measurement, experiments, uncertainty and decisions.

Map of the domain: AnalyticsAnalytics — Thinking with data: measurement, experiments, uncertainty and decisions.ANALYTICSModel evaluation — Methods and metrics used to estimate how well a model will perform on data it has not seen.Model evaluationHeavy tails — The property of distributions in which extreme values occur far more often than a normal distribution would predict.Heavy tailsOutliers — Observations that lie markedly far from most other observations in a dataset.OutliersPaired tests — Statistical tests that compare two conditions measured on the same units, such as two models on the same problems, using the differences within each pair rather than the two totals.Paired testsPre-registration — Recording a study's hypotheses, data, tests and decision rules before the outcome data are seen, so that confirmatory results can be told apart from exploratory ones.Pre-registrationStatistical power — The probability that a test detects an effect of a given size when the effect is really there; it grows with the sample size and the size of the effect.Statistical powerSummary statistics — Measures that summarise the location, spread and shape of a dataset in a few numbers, such as the mean, the median and the quartiles.Summary statisticsFifteen ways my coding agent and I mis-measured two models on one DGX Spark — Before you compare models on your own hardware, fix the sampling, the token budget, the harness and the number of runs: on one DGX Spark, three of the four moved a score by more than the gap between the models.Fifteen ways my coding agent and I…Small agent teams win by trying more, not by coordinating — A six-worker GLM-5.3 team proved more Lean statements than one agent of the same model, but four copies that could not talk to each other matched it. Here there is no evidence that coordination adds: try independent attempts with a verifier first.Small agent teams win by trying more, not…
Map of the domain

Concepts

Analytics

  1. Heavy tails
  2. Outliers
  3. Paired tests1 essay
  4. Pre-registration1 essay
  5. Statistical power2 essays
  6. Summary statistics

Essays

Notes

No notes yet.

The first piece will appear here once it is published.