Gürkan Özkan

Atlas

5 essays · 1 note · 30 concepts

Map of the atlas: 30 concepts, 6 essays and notesAI — Models, systems and the practice of bringing AI into real work.AIAnalytics — Thinking with data: measurement, experiments, uncertainty and decisions.ANALYTICSFormal verification — Establishing with a mechanical checker, such as the kernel of a proof assistant like Lean, that a statement or a program meets a precise mathematical specification.Formal verificationMachine learning — The field that studies how computers improve their performance on a task by learning patterns from data rather than being explicitly programmed.Machine learningLarge language models — Neural network models, typically with billions of parameters, trained on very large text corpora to predict the next token.Large language modelsHybrid architectures — Language model designs that interleave attention layers with layers of another kind, typically state-space or linear-attention layers, to lower the cost of long contexts.Hybrid architecturesInference serving — Running a trained model as a service that accepts requests and returns outputs, scheduling computation and memory so that many requests can be answered with acceptable latency and throughput.Inference servingBatching — Processing several requests together in one pass of a model, so that each read of the weights from memory serves more than one sequence, raising total throughput at some cost to the latency of each request.BatchingCold start — The period after a service or model server starts, before its caches are filled and its kernels compiled or loaded, during which the first requests run slower or behave differently from later ones.Cold startKV cache — The attention keys and values of the tokens a transformer has already processed, kept in memory so that they are not recomputed for every new token it generates.KV cachePrefix caching — Reusing the key-value cache already computed for a prompt prefix, such as a shared system prompt or the earlier turns of a conversation, for later requests that begin with the same tokens.Prefix cachingSpeculative decoding — A decoding method in which a cheaper drafter proposes several tokens ahead and the target model checks them in one forward pass, accepting a prefix of them without changing the target model's output distribution.Speculative decodingLLM agents — Systems in which a language model works in a loop, choosing actions such as tool calls, observing their results and deciding the next step towards a goal.LLM agentsMulti-agent systems — Arrangements of several agents, usually separate instances of one or more language models, that divide, share or coordinate the work on a task.Multi-agent systemsMixture of experts — A model design in which a router sends each token to a small subset of specialised sub-networks, the experts, so that only a fraction of the total parameters is active for any one token.Mixture of expertsRepeated sampling — Drawing many independent attempts from a model for the same problem and keeping any that a verifier accepts, so that the share of problems solved grows with the number of attempts.Repeated samplingSampling temperature — A parameter that rescales a model's output probabilities before a token is drawn; lower values make generation more deterministic, and a temperature of zero always takes the most likely token (greedy decoding).Sampling temperatureState-space models — Sequence models, such as Mamba, that carry a fixed-size recurrent state through a sequence instead of attending to every earlier token, so that their memory per sequence does not grow with context length.State-space modelsModel evaluation — Methods and metrics used to estimate how well a model will perform on data it has not seen.Model evaluationBenchmark saturation — The state of a benchmark on which current models score so close to the maximum that it no longer separates them.Benchmark saturationData leakage — Information that would not be available in real use reaching the model during training or evaluation, making performance look better than it is.Data leakageEvaluation harness — The software that runs a model on a benchmark: it builds the prompts, calls the model, executes or scores the outputs and aggregates the results.Evaluation harnessOverfitting — A model learning the noise in its training data as well, and therefore generalising worse to new data.OverfittingQuantisation — Storing a model's weights, and sometimes its activations or cache, with fewer bits per number, such as 8 or 4 instead of 16, to cut memory use and speed up computation at some possible cost in accuracy.QuantisationMemory bandwidth — The rate at which data can be moved between memory and the processor, usually given in gigabytes per second; generating text one token at a time is often limited by it rather than by arithmetic.Memory bandwidthUnified memory — A memory architecture in which the CPU and the GPU share one physical pool of memory instead of having separate system and device memory.Unified memoryHeavy tails — The property of distributions in which extreme values occur far more often than a normal distribution would predict.Heavy tailsOutliers — Observations that lie markedly far from most other observations in a dataset.OutliersPaired tests — Statistical tests that compare two conditions measured on the same units, such as two models on the same problems, using the differences within each pair rather than the two totals.Paired testsPre-registration — Recording a study's hypotheses, data, tests and decision rules before the outcome data are seen, so that confirmatory results can be told apart from exploratory ones.Pre-registrationStatistical power — The probability that a test detects an effect of a given size when the effect is really there; it grows with the sample size and the size of the effect.Statistical powerSummary statistics — Measures that summarise the location, spread and shape of a dataset in a few numbers, such as the mean, the median and the quartiles.Summary statisticsCompiling company documents into an ontology for agents: the model added 0.017 — On two held-out invented companies a rule-based reader scored 0.833 and the same pipeline with a local language model 0.850. On a real public handbook it fully found 58 of the 166 items that blind reader agents listed. Measure what each part adds before you build on it.Compiling companydocuments into anontology for agentsFifteen ways my coding agent and I mis-measured two models on one DGX Spark — Before you compare models on your own hardware, fix the sampling, the token budget, the harness and the number of runs: on one DGX Spark, three of the four moved a score by more than the gap between the models.Fifteen ways mycoding agent and Imis-measured two…Small agent teams win by trying more, not by coordinating — A six-worker GLM-5.3 team proved more Lean statements than one agent of the same model, but four copies that could not talk to each other matched it. Here there is no evidence that coordination adds: try independent attempts with a verifier first.Small agent teams winby trying more, not bycoordinatingSpeculative decoding on GB10: a wider draft speeds up one stream and slows sixteen — Choose the speculative-decoding draft width for the number of users you serve: in darkdatter's sweep on a DGX Spark, 16 draft tokens instead of 8 made one stream 28% faster and sixteen streams 10% slower.Speculative decodingon GB10What 128 GB of unified memory buys you: serving LLMs on a DGX Spark — On a GB10 the GPU memory fraction is a share of the whole machine: size the KV cache from the workload, and expect bandwidth and prefill, not capacity, to set the limits.What 128 GB of unifiedmemory buys youThe 1,802-second hang: serving a hybrid linear-attention MoE under agent load — In July I ran coding-agent evaluations on SWE-bench Verified against Qwen3.6-35B-A3B, served by vLLM 0.21 ( ) on one DGX Spark, four requests at a time. The…The 1,802-second hangMap of the atlas: 30 concepts, 6 essays and notesAI — Models, systems and the practice of bringing AI into real work.AIAnalytics — Thinking with data: measurement, experiments, uncertainty and decisions.ANALYTICSFormal verification — Establishing with a mechanical checker, such as the kernel of a proof assistant like Lean, that a statement or a program meets a precise mathematical specification.Formal verificationMachine learning — The field that studies how computers improve their performance on a task by learning patterns from data rather than being explicitly programmed.Machine learningLarge language models — Neural network models, typically with billions of parameters, trained on very large text corpora to predict the next token.Large language modelsHybrid architectures — Language model designs that interleave attention layers with layers of another kind, typically state-space or linear-attention layers, to lower the cost of long contexts.Hybrid architecturesInference serving — Running a trained model as a service that accepts requests and returns outputs, scheduling computation and memory so that many requests can be answered with acceptable latency and throughput.Inference servingBatching — Processing several requests together in one pass of a model, so that each read of the weights from memory serves more than one sequence, raising total throughput at some cost to the latency of each request.BatchingCold start — The period after a service or model server starts, before its caches are filled and its kernels compiled or loaded, during which the first requests run slower or behave differently from later ones.Cold startKV cache — The attention keys and values of the tokens a transformer has already processed, kept in memory so that they are not recomputed for every new token it generates.KV cachePrefix caching — Reusing the key-value cache already computed for a prompt prefix, such as a shared system prompt or the earlier turns of a conversation, for later requests that begin with the same tokens.Prefix cachingSpeculative decoding — A decoding method in which a cheaper drafter proposes several tokens ahead and the target model checks them in one forward pass, accepting a prefix of them without changing the target model's output distribution.Speculative decodingLLM agents — Systems in which a language model works in a loop, choosing actions such as tool calls, observing their results and deciding the next step towards a goal.LLM agentsMulti-agent systems — Arrangements of several agents, usually separate instances of one or more language models, that divide, share or coordinate the work on a task.Multi-agent systemsMixture of experts — A model design in which a router sends each token to a small subset of specialised sub-networks, the experts, so that only a fraction of the total parameters is active for any one token.Mixture of expertsRepeated sampling — Drawing many independent attempts from a model for the same problem and keeping any that a verifier accepts, so that the share of problems solved grows with the number of attempts.Repeated samplingSampling temperature — A parameter that rescales a model's output probabilities before a token is drawn; lower values make generation more deterministic, and a temperature of zero always takes the most likely token (greedy decoding).Sampling temperatureState-space models — Sequence models, such as Mamba, that carry a fixed-size recurrent state through a sequence instead of attending to every earlier token, so that their memory per sequence does not grow with context length.State-space modelsModel evaluation — Methods and metrics used to estimate how well a model will perform on data it has not seen.Model evaluationBenchmark saturation — The state of a benchmark on which current models score so close to the maximum that it no longer separates them.Benchmark saturationData leakage — Information that would not be available in real use reaching the model during training or evaluation, making performance look better than it is.Data leakageEvaluation harness — The software that runs a model on a benchmark: it builds the prompts, calls the model, executes or scores the outputs and aggregates the results.Evaluation harnessOverfitting — A model learning the noise in its training data as well, and therefore generalising worse to new data.OverfittingQuantisation — Storing a model's weights, and sometimes its activations or cache, with fewer bits per number, such as 8 or 4 instead of 16, to cut memory use and speed up computation at some possible cost in accuracy.QuantisationMemory bandwidth — The rate at which data can be moved between memory and the processor, usually given in gigabytes per second; generating text one token at a time is often limited by it rather than by arithmetic.Memory bandwidthUnified memory — A memory architecture in which the CPU and the GPU share one physical pool of memory instead of having separate system and device memory.Unified memoryHeavy tails — The property of distributions in which extreme values occur far more often than a normal distribution would predict.Heavy tailsOutliers — Observations that lie markedly far from most other observations in a dataset.OutliersPaired tests — Statistical tests that compare two conditions measured on the same units, such as two models on the same problems, using the differences within each pair rather than the two totals.Paired testsPre-registration — Recording a study's hypotheses, data, tests and decision rules before the outcome data are seen, so that confirmatory results can be told apart from exploratory ones.Pre-registrationStatistical power — The probability that a test detects an effect of a given size when the effect is really there; it grows with the sample size and the size of the effect.Statistical powerSummary statistics — Measures that summarise the location, spread and shape of a dataset in a few numbers, such as the mean, the median and the quartiles.Summary statisticsCompiling company documents into an ontology for agents: the model added 0.017 — On two held-out invented companies a rule-based reader scored 0.833 and the same pipeline with a local language model 0.850. On a real public handbook it fully found 58 of the 166 items that blind reader agents listed. Measure what each part adds before you build on it.Compiling companydocuments into anontology for agentsFifteen ways my coding agent and I mis-measured two models on one DGX Spark — Before you compare models on your own hardware, fix the sampling, the token budget, the harness and the number of runs: on one DGX Spark, three of the four moved a score by more than the gap between the models.Fifteen ways mycoding agent and Imis-measured two…Small agent teams win by trying more, not by coordinating — A six-worker GLM-5.3 team proved more Lean statements than one agent of the same model, but four copies that could not talk to each other matched it. Here there is no evidence that coordination adds: try independent attempts with a verifier first.Small agent teams winby trying more, not bycoordinatingSpeculative decoding on GB10: a wider draft speeds up one stream and slows sixteen — Choose the speculative-decoding draft width for the number of users you serve: in darkdatter's sweep on a DGX Spark, 16 draft tokens instead of 8 made one stream 28% faster and sixteen streams 10% slower.Speculative decodingon GB10What 128 GB of unified memory buys you: serving LLMs on a DGX Spark — On a GB10 the GPU memory fraction is a share of the whole machine: size the KV cache from the workload, and expect bandwidth and prefill, not capacity, to set the limits.What 128 GB of unifiedmemory buys youThe 1,802-second hang: serving a hybrid linear-attention MoE under agent load — In July I ran coding-agent evaluations on SWE-bench Verified against Qwen3.6-35B-A3B, served by vLLM 0.21 ( ) on one DGX Spark, four requests at a time. The…The 1,802-second hang

EssayNoteConceptrelations appear on hover

Drag to pan, use the wheel or +/− to zoom. Click selects, double-click opens the page; on the keyboard use Tab, the arrow keys, Enter and Esc.

Index

Domains and their concepts, by broader concept, with the number of essays on each. The pin shows a concept on the map.

AI

  1. Formal verification1 essayShow Formal verification on the map
  2. Machine learningShow Machine learning on the map
  3. Large language models1 essayShow Large language models on the map
  4. Hybrid architectures1 essayShow Hybrid architectures on the map
  5. Inference serving3 essaysShow Inference serving on the map
  6. Batching2 essaysShow Batching on the map
  7. Cold startShow Cold start on the map
  8. KV cache2 essaysShow KV cache on the map
  9. Prefix caching1 essayShow Prefix caching on the map
  10. Speculative decoding1 essayShow Speculative decoding on the map
  11. LLM agents2 essaysShow LLM agents on the map
  12. Multi-agent systems1 essayShow Multi-agent systems on the map
  13. Mixture of experts1 essayShow Mixture of experts on the map
  14. Repeated sampling1 essayShow Repeated sampling on the map
  15. Sampling temperature1 essayShow Sampling temperature on the map
  16. State-space models1 essayShow State-space models on the map
  17. Model evaluation4 essaysShow Model evaluation on the map
  18. Benchmark saturation1 essayShow Benchmark saturation on the map
  19. Data leakage2 essaysShow Data leakage on the map
  20. Evaluation harness2 essaysShow Evaluation harness on the map
  21. Overfitting1 essayShow Overfitting on the map
  22. Quantisation1 essayShow Quantisation on the map
  23. Memory bandwidth2 essaysShow Memory bandwidth on the map
  24. Unified memory1 essayShow Unified memory on the map

Analytics

  1. Heavy tailsShow Heavy tails on the map
  2. OutliersShow Outliers on the map
  3. Paired tests1 essayShow Paired tests on the map
  4. Pre-registration1 essayShow Pre-registration on the map
  5. Statistical power2 essaysShow Statistical power on the map
  6. Summary statisticsShow Summary statistics on the map
Concepts